Developer Journal 140

It's come back to bite me. Changing from genome version alpha to genome version bravo.

In version alpha, you retrieve the gene you want using a mathematical formula. There was an order to the sequence of genes. The advantage of this is that retrieving the gene you want is fast. The disadvantage is that adding new genes is hard because you have to modify the mathematical formula.

In version bravo, you retrieve genes by doing a search. There is no order to the sequence of genes. Each gene has a number of tags to uniquely identify it. The advantage of this is that adding new genes is easy. The disadvantage is that retrieving the gene you want is slow.

Starting up the simulation with a new population has become very slow. Not only that, when I ran the simulation last night, all of the robots died. Argh! I don't know what it is and it's hard to check because running the simulation takes time.

You have to make your changes small. You have to make sure your changes work. I wish I could have figured out a way to go from version alpha to bravo in smaller steps.

You have to keep an eye on how long things take. You have to time them.

Hopefully the population died because the gender constraint was active.

I added a few timers to look at how long things took to start up. The program takes 36 seconds for 8 robots. 6 seconds is for things like starting up the graphics and physics engine. Each robots takes about 3.75 seconds to start up. That doesn't sound like much but when you have about 128 robots that's 480 seconds or 8 minutes!

The majority of the time for creating a robot is creating the neural network and a large part of that is due to the searching for the required gene through the genome again and again.

I can't fix that now, indeed I'm not even sure of how to fix it other than to go back to version alpha but that has it's own problems, namely inserting new genes is difficult.

The good thing about this is that I know what's going, how long things are taking, and that it's not taking too long. This frees me from having to tinker with the genome for now. That means I can tinker with the neural network code to add neurons and synapses on the fly.

The easiest part of projects is at the start where you're day dreaming. The hard scary part is when you get to it and actually try to finish it.

Adding a neuron group looks easy. Adding a neuron to an existing group looks hard but it's not something I have to work on now. I'll have to work on it when I do things like increase the number of input neurons for vision on the fly.

Developer Journal 139

Arg! Another day on converting genome data from version alpha to version bravo.

Developer Journal 138

Today's task, update genome data when the genome configuration file changes.

My plan is to go through one gene at a time in the configuration file, and see whether it is in the genome data. If it is not, then I'll make the appropriate changes. For example, if there is are new genes for a new input group in the configuration file, I'll add the genes to the genome and generate what I'm calling the secondary genes.

Each neuron group has a bias gene and a bias learning rate gene.

In addition each neuron group has potential connections to every other group and vice versa. For each potential connection there is a connection density gene, a learning rate gene, and a topological distortion gene.

Whoa, each robot has about a megabyte of data.

The following error took me a long time to work out because of a serialization.

for( int j = 0; i < numGene; ++j )

Developer Journal 137

Spent most of my day writing the crossover and mutation code.

Developer Journal 136

Just spent a day fixing bugs that caused the simulation to crash when converting genome data from version alpha to bravo. Seems to be good for now. Just went over the completed and next actions list. There are still so man things left to go.

Next up

• fix how robots are not getting visual data
• when genome configuration file changes, update genome
• when genome changes, update neural network
• when connection file changes, update connection
• implement gender touch sensor
• run gender simulations
• balance gender over time

Developer Journal 135

Having had a good night's sleep feels so much better. After the fire alarms stopped yesterday, I had a look at the neural network input and output over time. The output was abnormally small which explained why the robots weren't moving. The robots also didn't seem to be receiving visual input. I've got one or two hours to work today before a relaxing weekend.

Aah, I printed out the minimum efficacy, maximum efficacy, and decay rate. The maximum efficacy was the same as the minimum efficacy. Yep that was it. A small bug that freaked me out on a rainy, warm, Friday night with blaring fire alarms.

I was worried that I'd made too many big changes, without testing. At the same time I couldn't really make the changes smaller in this case. More an more I'm appreciating the power of a well designed code interface.

Developer Journal 134

So sleepy. Woke up at 3 with stomach and back pain. Too much sitting and not enough training. Nutrition, getting better. Finally I've got a few days to focus on programming and writing, and I'm tempted to goof off. I can feel myself being less motivated and less intelligent.

I've got a few things to do today. Make sure that genome version 2 works with brain version 1. Then make sure that I'm converting genome version 1 data to genome version 2.

After a day of slow programming, genome version2 works with brain version 1. It seems to anyway, I'm going to have to let it run for a while to see what happens.

I'll also have to do some profiling because the constant genome look ups are slowing down the system.

Now to check the differences that between genomes that are the result of the normal creation code and the conversion code. I hate working with the serialization code enabled, it takes a lot more time to compile, link, and load.

27 seconds extra to compile. 56 seconds extra to link. 6 minutes 6 seconds extra to load. That might not sound like much but it is.

When the program loads, class Genome converts the data from pre-pre-alpha to version pre-alpha. Then the data gets converted to version alpha.

Pre-pre alpha is when the data was in class Genome and was a vector of floats. This was how things were originally coded up. Pre-alpha when the data gets converted to a vector of class Gene. This was when I was feeling my way around.

Version alpha is when the data is moved to class GenomeAlpha. I moved it over here so that I could preserve the old code in class GenomeAlpha, use class Genome as a common interface and try new code in class GenomeBravo.

Doh. I think I broke something. When I loaded up a previous save file, the robots no longer moved. I didn't change anything major, I changed the genome code but that shouldn't affect the architecture of the neural network.

I think the conversion from class Genome to class GenomeAlpha is not quite right.

Grr. Fire alarms have gone off. I'm sure it's a drill but they're so loud and annoying!

Developer Journal 133

Slight change to the gene format.

<gene>
    tag        input
    tag        random
    tag        numExc
    min        1
    max        1
</gene>

I removed "value" and "isVariable". I figured that I can let "value" be whatever and I could set a gene to not be variable by making the min and the max the same. I'm still worried that this is going to come back to bite me but the format looks and feels good.

I should look up a good XML parser but I don't want to involve new code. I'm going write a little code to read the settings in.

Years ago when I started working on developing AI, I did not think I'd be spending time writing up an XML like file to initialise a genome structure.

Finished writing the code to create genome version 2 from a configuration file. Reading in the primary genes was easy. Creating the secondary genes that depend on the primary genes took a fair bit longer to do.

Still a fair bit of testing to do. I have to make sure genome version 2 works with brain version 1, then write brain version 2.

The main difference with brain version 2 is that it can grow new neurons and synapses when the genome changes. For example, I'm going to need to add a new input and output group to the genome and then I need the brain to grow the new input and output groups. Yee haw!

Hopefully I can get the version 2 gender experiments done soon and have some time to work on the robot arms and legs a bit more. The new feature of being able to add input and output groups on the fly is going to help a lot. Each joint in the robot body is going to need a brain output groups. In addition, I'm thinking of pumping limb data into the brain, so I'll need brain input groups.

Developer Journal 132

Oh XML, XML. Another technology I ignored during my undergraduate studies that's coming back to haunt me.

The settings file for the genome class has lots of different types of data and so much data. I've come up with a format that I hope will be extensible and won't come back to bite me later.

<gene>
    tag        input
    tag        random
    tag        numExc
    min        0
    max        1
    value      1
    isVariable 0
</gene>

My long term goal is to move as much as I can to the genome so that evolutionary processes can determine the values.

Developer Journal 131

I won't be able to do too much development today, I've got a fair amount of marking and tutoring to do.

I'm still working on the code to transition from genome version alpha to bravo. I've been thinking about whether to put the conversion code in class GenomeAlpha, class GenomeBravo, or somewhere else. I've placed the conversion code in class GenomeBravo for now. The conversion function receives a pointer to class GenomeAlpha and converts the data into class GenomeBravo.

I thought I'd be able to get the new genome, neural network, embodiment code done, and run a number of simulations by Wednesday, starting from last Wednesday. The idea was to give the robots a new sensor that would allow them to detect whether nearby robots were male or female. I wanted to avoid hard coding, and to preserve older code so I could step back if I needed to, so the coding is taking longer than I thought it would.

I'm finding more and more that topics I've avoided in the past are coming back to bite me, maths, physics, databases, and networking. Working with simulated robots requires maths and physics. Working with population data over time requires databases. Scaling the simulation requires networking to to a client server architecture.

Developer Journal 130

Converting genome data from version alpha to version bravo.

Grr. Genes have a value between 0 and 1. This value then gets interpreted into a range with a specified minimum and maximum. I was using the accessors in alpha to get the gene values and those accessors return the interpreted values and not the pre-interpreted values.

I'm suspecting more and more that a database would be useful to do serialization and to migrate the data from version to version.

I'm trying hard to not make GenomeBravo have to know about how GenomeAlpha works.

I'm suspecting more and more the code I'm writing to migrate the data from genome version alpha to genome version bravo will be soon obsolete and that I would have been better off using database techniques to store and transform the data but I don't have those database skills yet.

Developer Journal 129

Some rough notes, I'll write this up in a readable form soon.

---

I've been working on adding new capabilities to the robots. My goal is to convert the hard coded values to things I can change from text files and then have the robot sub-systems change dynamically.

If you increase the number of input / output groups, you have to change the indices. I'll also have to save the indices in order to do the updating.

Create a gene class.

When the brain updates, it pulls from the sensors.
When the robot updates, it pulls from the brain.

How to convert to a new structure?
Modify text file.
Modify genome.
Modify neural network

When a user creates a robot, they have to create the following files.
robotAlpha.cfg
+ bodyAlpha.cfg
+ brainAlpha.cfg
+ genomeAlpha.cfg
+ connectAlpha.cfg

You can change body, brain, genome, connect files by modifying robot. You can change which robot file to use through a setting the robot file.

1 shared CritParam, or 1 CritParam each

Focus on doing the minimum first

numInGroup 6

inGroup1 random
inGroup2 energy
inGroup3 red
inGroup4 green
inGroup5 blue
inGroup6 listen

inGroupRandom 1
inGroupEnergy 1
inGroupRed 3
inGroupGreen 3
inGroupBlue 3
inGroupListen 16

numOutGroup 8

outGroup1 eat
outGroup2 mate
outGroup3 fight
outGroup4 speed
outGroup5 yaw
outGroup6 light
outGroup7 focus
outGroup8 speak

outGroupEat 1
outGroupMate 1
outGroupFight 1
outGroupSpeed 1
outGroupYaw 1
outGroupLight 1
outGroupFocus 1
outGroupSpeak 16

Each genome in the text file should have a name, a variability flag, and a value

Each brain input / output has a name.
Each robot input / output has a name

// robot direction brain
visionOut > visionIn

// robot direction brain
forwardIn < forwardOut
leftIn < leftOut

I also want to be able to modify the neural network without modifying the genome.

When I serialize and after I load, I can check whether the settings file has changed.

The settings file can have a version number or I can check whether the parameters have changed. The problem with checking the parameters is that the process could take too long if each robot has their own genome param.

The reason I considered checking parameters is that I didn't want errors to come from manually incrementing the version numbers.

The genome checks the settings file.
The brain checks the genome.
The body checks the genome.

Developer Journal 128

I'm excited. I finally get to work on a system to allow me to introduce different robot types. I'm determined to avoid hard coding and to preserve previous robot designs as I go.

I've also being thinking about the nature of work and what I would do if I didn't have to work.

I wish I knew a better way to do the genome crossover process. I'll have to look around for better ways of encoding data in genomes and doing the crossover and mutation processes.

The hard part is converting from one type of robot to another.

Developer Journal 127

Gender

Introducing gender to the population didn't quite produce the results I wanted. Introducing gender should have produced different results depending on how quickly I did it. Ideally introducing gender suddenly should have resulted in the population dying and introducing gender slowly should have resulted in the population surviving. That way, I could be sure that gender had a big enough impact to cause the population to have to change their behaviour.

Unfortunately, the population survived in both cases. Now I have to look for other indicators of behaviour change. I'll be going back to check whether the rate of same gender mating attempts changed over time. In addition, I'll have to introduce different colours to the genders to help the robots identify genders. I'll have to be careful choosing colours, pink for females seems obvious but wrong.

Pheromones

This brings up new problems, what happens if robots aren't facing each other? The suggestion I was given was to imagine the robots identified gender based on scent. So when robots are near each other they can tell whether another robot is male or female if they are close enough to each other. This kind of goes against a few ideas I've had. Firstly, you shouldn't have to interpret things for the robots and this feels like some kind of interpretation is going on. Secondly, I can't fit this neatly into my idea of basing on how humans evolved. At the same time I'm going to give it a go, it won't take long to program (famous last words) and it'll provide more information about what's happening in the background.

Intention

The other thing I've been thinking about and it relates to the problem of robots facing each other is encouraging intentional behaviour. At the moment, if two robots are next to each other and happen to think about mating at the same time for a split second, they reproduce. It's like if you're sitting next to someone and if you both happen to think about sex you end up with a baby. My idea is that they have to do something that takes a prolonged effort. For example, they have to face each other for 5 seconds and in that time, both agree to reproduce. The same with food. That way, if they choose to reproduce they have to face another robot for a period of time and activate their pseudo mating mechanisms for a sufficient period of time and intensity.

Arms, Legs, and Maturation

At the same time I want to introduce the maturation condition, which makes younger robots depend on the help of older robots to survive. At the same, same time I want to introduce the arms and legs that I've been going on about for months now. Yeah, I should get to work, but writing this helps clarifies my thoughts.

The thing that would be useful to work on is the ability to introduce new neural network inputs and outputs on the fly. That's something that's useful for all three of my programming tasks. Once I have that feature, the task of implementing arms and legs becomes a lot smaller since I've done most of the physics programming.

This time, I've learned my lesson. Instead of modifying the neural network code, I'll make version two. It's weird, it's been so long since I've touched the neural network code that I don't know how the network gets set up. I suspect I'll find the physics code for arms and legs unfamiliar as well. Oh yeah, I want to do my game programming as well.

Developer Journal 126

Grr. I realised while in the shower this morning that I had set up the simulation with a 100 by 100 world instead of a 200 by 200 world.

Then when I was at uni I realised I was setting up the gender condition incorrectly. Borrowing an idea from RPG games, same gender robots have to roll below a percentage in order to reproduce. For example if the limit is 50, robots have to roll in the range 1 to 100 and get below 50.

The problem was I was rolling in the range 1 to the limit. Yep. Moving on.

To be extra geeky, I'm changing it around. Robots have to roll above the threshold. For example if the percentage is 75, robots have to roll above 75 to succeed in reproducing if they are the same gender.


So much dumbness. First, the threshold was already at my limit because of serialization so even though I wanted it to start at 1 and then increase it wouldn't. Second, I was using the wrong timer to periodically increase the limit. If you program while you're tired or distracted, you're a bl#$$y idiot.

Developer Journal 125

Grr. My approximately 10 minute load time grew to way to long because of my memory tracking code. 

• 3 hours, 33 minutes, 6 seconds, primary computer
• 3 hours, 24 minutes, 55 seconds, secondary computer

 I'll have to disable the memory tracking code for normal use and only enable it for debugging purposes.

Some things I'm learning.

• Don't code your own serialization system
• Have a serialization strategy
• Have a few global variables to makes things easier
• Don't code your own memory tracking system
• If you're starting to consider writing your own template code, you're probably on the wrong track

Developer Journal 124

Yah! I finished the introducing the gender condition. It took me a lot longer than I expected. I have to get a whole bunch of assignment marking done. I've got a lot of experiments I can run while I mark. At the same time I'd like to get the maturation condition done as well.

I'd also like to figure out how to make the code easier to work with. Things are all over the place.

Grr. I had to do a bunch of extra work to get my memory tracking code to work with Boost Serialization. When Boost Serialization loads, I have to manually register the object because I don't know how to get Boost Serialization to use my New() function that wraps around the "new" operator.

Time to test whether my changes are compatible with my ongoing save files.

Running one experiment where I suddenly introduce gender to an ongoing population. On another computer I'm gradually introducing gender.

Sigh. I think my memory tracking code is making the load process take too long. Loading a file with 128 to 256 robots takes about 10 minutes normally.

Developer Journal 123

Grr. Lot's of assignment marking today. I'm going to finish the gender constraint so that I can run the simulations in the background.

I got the following error message that took me a while to fix.

-*- mode: compilation; default-directory: "~/polyworld/" -*-
Compilation started at Wed Oct 12 14:53:54

mingw32-make 
Scanning dependencies of target OgreApp
[  2%] Building CXX object CMakeFiles/OgreApp.dir/app/Serializer.cpp.obj
In file included from D:\agi\dep\boost_1_44_0/boost/serialization/static_warning.hpp:73:0,
                 from D:\agi\dep\boost_1_44_0/boost/serialization/export.hpp:38,
                 from D:\agi\polyworld\app\Serializer.cpp:10:
D:\agi\dep\boost_1_44_0/boost/mpl/print.hpp: In instantiation of 'boost::mpl::print< boost::serialization::boost_serialization_static_warning_line< 137> >':
D:\agi\dep\boost_1_44_0/boost/serialization/static_warning.hpp:92:13:   instantiated from 'boost::serialization::static_warning_test< false, 137="">'
D:\agi\dep\boost_1_44_0/boost/serialization/export.hpp:137:1:   instantiated from 'const boost::archive::detail::< unnamed>::guid_initializer< t>& boost::archive::detail::< unnamed>::guid_initializer< t>::export_guid() const [with T = World, boost::archive::detail::< unnamed>::guid_initializer< t> = boost::archive::detail::< unnamed>::guid_initializer< world>]'
D:\agi\polyworld\app\Serializer.cpp:60:1:   instantiated from here
D:\agi\dep\boost_1_44_0/boost/mpl/print.hpp:55:5: warning: comparison between signed and unsigned integer expressions

< /world>< /unnamed>< /t>< /unnamed>< /t>< /unnamed>< /t>< /unnamed>< /false,>< /boost::serialization::boost_serialization_static_warning_line< 137>< 

The clue was T = World. I forgot to make the destructor virtual. Oh Boost Serialization, oh how I love and hate thee.

Developer Journal 122

I've almost finished the code for making the world bigger over time. I can't believe it took so long. Hopefully introducing gender and pseudo ageing won't take so long.

Damn, damn, damn. I took almost 3 hours to solve a problem.

	

	// create a motion state
	obj->motionState.reset( new btDefaultMotionState( transform ) );

	// create a collision shape
	const btVector3 halfExtents( 0.5f, 0.5f, 0.5f );
	obj->collisionShape.reset( new btBoxShape( halfExtents ) );    

	// create a rigid body
	const float mass = obj->mass;
	btVector3 localInertia( 0.0f, 0.0f, 0.0f );
	// rigidbody is dynamic if is non zero    	
	if( mass != 0.0f ) {
	    obj->collisionShape->calculateLocalInertia( mass, localInertia );
	}
	btRigidBody::btRigidBodyConstructionInfo rbInfo( 
	    mass, obj->motionState.get(), obj->collisionShape.get(), localInertia );
	shared_ptr< agiRigidBody > rigidBody( new agiRigidBody( rbInfo ) );
	rigidBody->Init( obj );
	
	// add to dynamics world
	imp->dynamicsWorld->addRigidBody( rigidBody.get() );

	// set pointer in data object
	obj->rigidBody = rigidBody;

	// scale
	btVector3 bulletScale;
	bulletScale.setX( obj->scale.x() );
	bulletScale.setY( obj->scale.y() );
	bulletScale.setZ( obj->scale.z() );
	rigidBody->getCollisionShape()->setLocalScaling( bulletScale );

I didn't know I had to initialise localInertia. Most vectors I work with default to 0.0.

I was relying on similar code elsewhere and then when I changed over to this code I had to track that error down. Grr. For three hours I had things falling through my ground for no good reason. Yeah, I need to learn more about physics and the physics library I'm using.

Developer Journal 121

I'm currently writing some code to make the world bigger over time. In particular, I'm thinking about how to save the state of the world over time. I'm wary of using more of Boost Serialization. The world is pretty simple with four walls and a floor. I can serialize the world object or the wall and ground objects.

I remember reading that I should avoid file reading in constructors. I'll need to do some file reading to get parameters to control how the world changes over time. I need to do this in the constructor if I serialize the world object or else I will need to call an initialisation function after each time I load the world object.

I don't know whether to serialize the world object or just the ground and wall objects. When in doubt and analysis paralysis, my strategy is usually to do the obvious, easy, albeit dumb thing. Save the 4 walls, the ground, and the last update time. Do everything manually for now.

I'm finding I'm spending more time trying to do things the smart way than is worthwhile.

Even simpler, save the current world size and the last world update time.

Hmm, that means the currentWorldSize and the lastWorldUpdate is inside class SimData and outside class World. When I load class World, it should be good to go. Albeit, I have to manually do stuff for physics and graphics. I'll have to copy over currentWorldSize and lastWorldUpdate from class SimData to class World. Then I'll have variables doing the same thing in different places. Grr, I'll serialize the world.

Dang! I just realized my fancy memory registry class is limited by the serialization process. My idea was to wrap new in a function that creates a note in the memory registry. When an object goes out of scope it notifies the memory registry. When the memory registry goes out of scope it should have no notes. If it has notes then it means some objects were not released and there is a memory leak.

I originally stumbled because I couldn't figure out how to get it to work with classes that were not my own. Then I realized that I didn't need to. The point of the memory register was to make sure that I avoided cyclic dependencies in my code. All good, then I realized that I didn't know how to make Boost Serialization use my New() function. I have to register objects created by Boost Serialization with the memory register manually. Not ideal, but it'll work for now.