From Language Games to Sensorimotor Coordination Games: Long runs of HO-MIND (higher order mutual information darwinian neurodynamics) on Lincoln

What research am I doing today?

1. Just started a run on Lincoln:  MB15 is being run to determine whether pairs of motor actions evolve to maximise MI between them and ONE sensor signal. Lets hope it works.

2. Continue design of the co-evolutionary COD-N algorithm, ... 
Opps just found this, this looks urrrrrhhh relevant.. perhaps lets read it.
http://laral.istc.cnr.it/nolfi/papers/gigliotta-pezzulo-nolfi.pdf
mmm, naaaa...

The design issues are of at least two types.

1. What is an evolvable open-ended (has systematicity and compositionally, and thus is capable of productivity, bla bla Fodor and Pylysian, bla bla) representation of action? 

A path evolution approach was considered previously, in which overlapping paths can integrate action fitness efficiently. Let action atoms be nodes. Action atoms can link with other action atoms with time delays of 0 to N, thus encoding parallel and serial concurrent and sequential activation. Each action atom takes a sensory vector (and inputs from the target state) as input and outputs a motor vector as output. This motor output can be primitive motor commands OR calls to execute other action atoms. 

Inside an action atom can be a whole plethora of different kinds of controller. Roughly, from simplest to most complex. 

a. A linear regression unit that maps s/t to m reactively. 
b. A FFN unit that maps s/t to m reactively. 
c. A CTRNN that stores STM state, mapping s/t to m. 
d. A black-box e.g. CMA-ES stochastic search algorithm that minimises a distance measure from s to t! 
e. LWPR can learn an inverse model by being exposed to the training date acquired from the above functions, and then can use local greedy planning to minimise distance from s to t, using the learned inverse model. 
f. Forward models can be learned and search through the forward models can be exploited explicitly to plan actions. 
g. Unsupervised learning atoms: Higher-level atoms that do not explicitly encode motor output primitives nor even execute other motor atoms directly can exist. These generate higher-order representations based on s/t and maybe m inputs that can be used by other motor atoms as inputs. For example, there may be a DBN atom that tries to classify some subset of inputs, or a k-means atom, or a EM-atom for example.

The evolutionary algorithm mutates these atoms, and mutates the links between atoms using a path evolution type algorithm in which there is stochastic execution of action-molecules based on link strength. 

2. What is an evolvable open-ended representation of goals? 

The representation of goals is in a sense even harder. For example, when I was a child I used to enjoy moving my hand between my eye and the central lines on the road while in the car, trying to get my finger between the gaps between the central lines of the road. This is a rather enjoyable sensorimotor coordination game that I invented all by myself for no 'good' reason. 

The goal representations should be rich enough to specify the rules of this level of sensorimotor coordination game! :) 

The specification of fitness functions/games to play can take the following forms from simple to complex. 

1. A single vector showing the desired final sensory state (subset or full set of all sensations) to be achieved throughout a fixed time trial. 
2. A more complex but still static perceptual feature to be achieved, e.g. keep the face in the middle of the screen. Such higher level perceptual goals would require access to again a set of unsupervised learning atoms that self-organize interesting representations.. Perhaps such a set of unsupervised atoms should be accessible to both populations (atomic actions and atomic goals). 
3. A dynamic specification of target behaviours. A general dynamic specification might be achieved by for example generating a target set of sensory states using the output of an unsupervised atom that contained a liquid state machine which produced some dynamic function of the sensory data stream for example? 
4. Game nodes could point to other game nodes, producing games that are hierarchical, e.g.  put your head and stroke your stomach.