GISMO will be a special purpose genetic algorithm capable of evolving CTRNN and CiTRuS networks only. It is well know that incorporating 'domain specific' knowledge into genetic operators, and thus producing a GA specialised for a particular task, can improve performance (see Michalewicz, Z.; Genetic algortihms + Data Structures = Evolution Programs; Springer). In a sense GeNGA used some domain specific knowledge, but remained general since this knowledge was supplied in a configuration file, not at the level of the design of genetic operators. GISMO will evolve variable topology CTRNN and CiTRuS networks using operators taylored for the exploitation of neutral networks in the fitness landscape.
See Lionel Barnett's paper Netcrawling - optimal evolutionary search with neutral networks in the CCNR publications page for information about neutral networks. In this alternative view of GA search dynamics, the population spends large amounts of time drifting randomly around at the same fitness level until by chance it finds a 'portal' genotype to a higher fitness level. The time spent optimising newly discovered innovations is small relative to that spent diffusing. Consequently an efficient search would maximise the rate of drift rather than of optimisation as such. This also suggests that to explore network topologies efficiently we should be careful about the type of mutation operators used to add, delete or modify network components. Using domain specific knowledge, we can add, modify and in some cases delete network components without affecting the phenotype. Further, non neutral mutations can then explore new areas of phenotype space.
This is in contrast to mutations where topology is mutated non neutrally. Since topological mutation can have sudden, large effects on phenotype (there is no way to add or delete a node in stages as such), many mutants will be eliminated immediately. Neutral mutants have a much greater chance of drifting to fixation in the population (particularly if netcrawler were used), allowing the network topology to drift at a faster rate.
I use the term 'smooth mutation' to mean a superset of 'neutral mutation'. The process of neutrally adding a new network component is useless if any subsequent parameter mutations on the object will cause either large disruptive changes or no changes at all. We want our neutral operations to leave new components poised to come into dynamic play easily but gradually. At least we want subsequent small parametric mutations to cause small changes to the dynamics (of course we can also apply large mutations if we wish). In the limit of infinity small creep mutations the idea is to have all changes in the number of network nodes to occur smoothly with respect to network dynamics. Here are three heuristics for designing a 'smooth' topology mutation system:
(1) Phenotypic neutrality - topology changes should have a minimal effect on the dynamics of the existing network. Adding a new node with large weight values to existing nodes will probably disrupt their behavior, and the mutant will not survive. Deleting a node with many connections is also likely to have a similar effect.
(2) Topological effectiveness - this says we shouldn't add things which cannot be brought into dynamic play quickly. For example, there is little point in adding a new node which is disconnected from the rest of the network. Mutation of its parameters will do nothing until we add weights linking it to the network proper. Until then its just network bloat, which selection cannot act upon.
(3) Mutational poise - initialise the parameters of new components carefully, e.g. use centre crossing, so that future small mutations can gradually bring the component into play. A node which is always saturated on or off because of a large bias value cannot be brought into play easily by attaching small magnitude weights to it.
An example operator involving addition of a new node might be to delete an existing weight and replace it with a weight to node to weight triplet, such that the effect of the original weight is mimiced as closely as possible (NEAT uses a similar operator). This would involve setting the new node's time constant to be as fast as possible, scaling the input onto the linear part of its activation function and scaling the output back to the original range. This is neutral as long as the parameter ranges allow the appropriate scaling and rescaling to occur, and provided the new node can respond to its inputs fast enough to avoid acting as a filter. This is a topologically effective mutation since a new non linearity etc. is now available in the network.
Deletions are more difficult to render as 'smooth operators'. Firstly, we can delete any nodes which don't have output weights. If we delete network weights piecemeal we may generate (selection willing) no-output nodes, and be able to remove these neutrally. We now need a way to delete weights neutrally. Again, we know we can already delete weights whose magnitude is zero, but this is not likely to occur by chance apart from weights we may have just added and initialised to zero. We can actively encourage the generate zero weights, selection willing, using the following 'metamutation' scheme. In this scheme each weight has a metamutation flag. When unflagged mutations are applied as normal. New weights are created unflagged, which protects then from premature deletion. When the flag is mutated to active mutation of the weight parameter is biased or constrained towards decreases in magnitude. When a mutation occurs which crosses zero (i.e. changes the sign of the weight), the weight is deleted (but only if it is flagged, to allow ordinary mutation to change the sign of weights). The point of metamutation is that selection may allow a weight magnitude to be decreased gradually where it would not have allowed an immediate deletion to occur. This is because compensatory mutations may be able to make up for the decreasing weight, until finally it is not needed, for example by changing the bias of the receiving node.
There are at least three fundamentally different ways of bringing new variables (nodes/dimensions) into a CTRNN dynamical system smoothly. These are (i) as a function, (ii) as a parameter and (iii) as a duplication. In (i) we note that a node whose dynamics are much faster than all other changes in the system will always be at equilibrium, and its output will consequently always appear to be a function of its inputs to the rest of the system (and a function does not constitute a variable). If we insert a new node into the network such that it acts as an identity function (as we did in the Example Addition Operator section above), the insertion will be neutral, and a gradual reduction of the time constant would make the new variable felt gradually. In (ii) we note that a node whose value never changes will appear as a fixed parameter to the rest of the network. If we introduce a new node which acts to add an input of zero to other nodes, the addition will be neutral, and small changes in node output will gradually bring it into play. In (iii) we note that replacing a node by two identical copies, each with the same inputs and outputs, save that the output weights are halved, would be a neutral operation.
The reverse of these three classes of operation are also smooth, and could in principle form the basis for corresponding deletion operators.