CiTRuS - The Continuous Time Recurrent System

This is a superset of the popular CTRNN (Continuous Time Recurrent Neural Network) model. Having failed to evolved path integration using CTRNN, but having suceeded with ModCTRNN, I have now undertaken a systematic study of CTRNN model supersets, including ModCTRNN. Counter intuitively, the more complex ModCTRNN model produced a more readily understandable path integration model than the partial solutions I was able to produce using CTRNN. I now wish to investigate if this apparent advantage applies to other simulated tasks, such as those addressed by Randall Beer's work.

The CTRNN model is a model of neurons (cells of the brain or nervous system of animals) which describes their behaviour using spiking rates. This is different from spiking models which generate spikes explicitly. The use of the CTRNN model here and in related research is not intended to model neurons accurately, but rather to use it as a dynamical system, capable of responding to input from sensors, of performing internal processing including memory and of producing appropriate motor output signals.

With this abstraction in mind, it should be clear that we no longer need to restrict our study to models derived directly from neurons. Some reasons for using CTRNN are probably that it has been shown to be capable in principle of approximating any dynamical system (given sufficient neurons), that it is derived from models of real brains (which are known to be good controllers), that it is widely used and therefore known to work on may problems in practise and that it is about the simplest system likely to be capable of such general dynamical behaviour. CiTRuS is an attempt to really probe the suitablity of CTRNN, by testing it head to head on the same tasks with other models constructed according to a set of design heuristics intended to make them powerful, evolvable agent controllers.

CiTRuS consists of five models in a nested hierachy, starting from the CTRNN model. Each higher model is a superset of the last. This means that a simulator only needs to implement the top level model and translate all other models into it. It also means that the generality results applicable to CTRNN also apply to all models, and that we could if neccessary seed a population with any existing CTRNN network but allow it to evolve using any of the higher models.

The models, in order, are called: citron (CTRNN translated into the CiTRuS formalism), mandarin (ModCTRNN in CiTRuS form), lemon, satsuma and grapefruit. Three design heuristics were used to guide model construction: plasticity, availability and neurality.

Plasticity says that we always want evolution to be able to modify a parameter in the network by adding a new interaction to it and turn it into a variable. This must apply to all network parameters we define as plastic, and must KEEP ON applying. Eg, if we have a fixed weight, and we want network weights to be plastic, we must be able to input a signal to the weight, via a weight-weight. The weight-weight must also be modifiable, via a weight-weight-weight and so on. The idea is to not just add an adhoc mechanism to modify a certain network parameter, but to get a maximum increase in generality by making the mechanism apply recursively. This may sound frivolous, but the path integration network evolved to use 4th order weights. Also consider that a series of higher order weights can implement multiplication and approximate Ezequiel Di Paolo's homeostatic oscillator neurons and multiple forms of Hebbian learning (i.e. mandarin can do all this).

Availablity says that any network variable must be able to take part in any number of interactions anywhere in the network, limited only by its type. eg. weights only act as weights, not conductances or potentials. This allows global variables. NB only plastic network properties need be made available. The idea is that useful information could evolve to be stored in any network variable. In mandarin weights are plastic but not available, so weight information is trapped into a single network location. Availability only applies what is already the case in CTRNN with respect to cell potentials, i.e. any cell potential can output any number of times to any neuron.

Neutrality says that when you add a new component to the network during evolution (this whole investigation assumes a variable length encoding scheme since full connectivity is not possible once we have plastic weights) the addition should not alter existing network dynamics. This must be implemented by a simple mechanism such as setting the initial interaction strength (e.g. weight) to zero. Later on creep mutations can take this value away from zero and so gradually cause the new component to have an effect. This is an attempt to create a smooth 'fitness landscape'.

Model summary (all networks implement neutrality):
A = available
P = plastic

All models use a single equation with the same form as the CTRNN model, with minor changes. The equation is simply applied to weights and sometimes conductances as well as cell potentials. The form of the equation is always:

where X is either a potential (v), weight (w) or conductance (g). A conductance is formally the reciprocal of resistance, whereas a time constant is the membrane resistance times the capacitance. I take g = 1 / tau here by defining capacitance as unity. Hence a conductance can have a value from zero (the variable being controlled does not change with time) to some maximum consistent with the numerical integration scheme being used. I changed from time constants to conductances for two reasons:
(1) an infinite time constant is allowed since it means the value is changing so slowly that nothing happens, but zero is not since it means an infinitely fast process which cannot be handled by the numerical integrator. Zero is easier to compute with than infinity, so use 1 / tau.
(2) generally the genotype encodes ln(tau) which is converted to tau before being used. Encoding 1 / tau has a similar effect, since it compresses the larger values relative to the small ones. So if we use a conductance, just encode the conductance directly.

Network Encoding Schemes

Below a square is a constant value, a circle is a variable. Each item shown is encoded as a single gene. Dotted components are parts of other genes referred to by the gene but not encoded by it. The arrows show the direction of information flow through the network. Any of the arrows may be left unconnected by the GA such that the encoded interaction does not have any effect on the phenotype. In all cases the sigmoid function is the standard logistic activation function (I've taken the bias out of the activation function and put it into the state equation proper, because all variables need biases, but only potentials are passed through an activation function):

It is neccessary to rectify the conductance values when they are variables to prevent them from making time run backwards when they achieve negative values. Either use the sigmoid function, and scale by the maximum allowable conductance (since this function is smooth dynamical systems analysis may be more straight forward) or use the piecewise linear function:

When using numerical integration it is usually essential to limit the conductance values to some maximum positive value relative to the integration step size being used.

Citron

Here we see the standard CTRNN model (expressed in CiTRuS form) where weights are constants, connecting from one neuron (whose internal state is a potential, not a firing rate) to another. Each potential has encoded along with it a conductance (1 / time constant) which is fixed. Nothing inputs information to a weight or conductance since they are fixed.

Mandarin

This (referred to as ModCTRNN on my path integration page) was my first attempt at modifying the CTRNN. Weights are now able to receive incoming links just like potentials are, hence X e {v,w}. Conductances are still constants encoded along with their potential or weight.

Lemon

Lemon is the first model to make conductances variable. This is enough to enable latching behaviour: if a variable X has a low conductance it retains its current value. If the conductance rapidly changes to a high value X quickly moves to its current equilibrium point based on its current input. Now if the conductance drops again X retains its new value. This might be a useful trick for implementing memory in an evolved agent.

Satsuma

This model implements all of my heuristics for all interesting network variables (note biases are not considered interesting in this sense since turning a bias into a variable is exactly the same as having a variable weight from a neuron firing at a fixed rate). Since these are _only_ heuristics, full availability might just as well reduce evolvability as increase it since any variable can influence (or disrupt) a large number of others. Phi is a meta weight, and is only there to implement the neutrality property: when you create a new synaptic interaction, set phi to zero so it had no effect. Later creep mutations can alter its value slowly.

Grapefruit

Probably no more general than satsuma, but here each variable X can have multiple incoming conductance interactions, instead of being limited to one. Again phi is an interaction weight which is set to zero when a new conductance interaction is created to make the inherently discontinuous event of inserting a new discrete component phenotypically neutral. Grapefruit networks can, like CTRNN, be fully connected because we can define at the start how many of each type of node (v,w,g) we want, and then fully connect them with synaptic and conductance interactions.

One probable change I will make to the model is to put weights through a sigmoid function for the satsuma and grapefruit models, otherwise it is easy to construct a small network where a weight value grows exponentially towards infinity. If you have any comments or questions send me an email (see my CCNR page ).