Action Molecule for Visual Reaching

After a wonderful meeting in Paris, taught by Phillipe Capdepuy from Generation Robots/HumaRobotics

http://www.generationrobots.com/site/program-nao-robot/index_en.html
http://homepages.stca.herts.ac.uk/~cp6cl/

I have had some good ideas for cognitive architectures based on Darwinian neurodynamics and how to implement them. These involve the use of much higher level action atoms than I was previously considering. The techniques suggested by Phillipe for using modules and events are terribly helpful, and are surely the way to go.

One way to test the evolvability of the action atom architecture I am proposing is to try to hand design various behaviours, e.g. those behaviours exhibited by Frida, in the specification language proposed. An example is reaching for a visual target something which Frida has truly mastered by 4 months, being able to use both hands, and getting depth judged much better than before.

One possible modular framework for reaching to a visual target is shown below.

The action molecule consists of 3 atoms.

Atom O[input: visual image, output: list of object coordinates]
Atom A[input:two vectors of length any N, output: Euclidean distance between vectors]
Atom B[input:a scalar value, output: a continuous valued vector]
Atom B' (not shown/optional) [inputs: a continuous valued vector e.g. from Atom B, output: motor commands, e.g. of the arm]

The modularity of the above system of atoms allows higher level functions to be recombined during operation, and thus reused, e.g. Euclidean distance may be reused in many action molecules.

The next population is a population of games. Here the game atom is as follows

Game Atom A: [input:a scalar value (i.e. Euclidean distance), output: the inverse of the sum of these values while the above action molecule is active] In effect this measures how close the hand is to the visual target over the course of the run. The larger the value the better the reaching. This value is the fitness assigned to the action molecule. Fitness of an action molecule means that the root atom of the molecule has a higher chance of being activated subsequently and that the links between the action molecule are strengthened.

The fitness of the Game Atom is as always the variance of fitness components of actor molecules when playing this game. For now, it is undefined as there is only one actor molecule and one game atom. However, when new actor molecules and new game atoms arise in the course of evolution, then it is this fitness which will determine which games are played. As Fisher said "

"The rate of increase in fitness of any organism at any time is equal to its genetic variance in fitness at that time."^[1]

Well, the rate of increase in fitness of an idea at any time is equal to the variance in fitness at that time. And good games are those which so increase the variance in quality of solutions in a neural population of replicators! 

Another variant of the above game might for example be to move the object at the same time as the hand, i.e. to produce coordinated motion of the object and the effector. This is all specified in the mutated Euclidean distance function.

OK, so to get further insight into the possible behaviours and adjacent possible behaviours of those behaviours we can hand-design a set of behavioural prototypes based on observations of Frida. The behaviour, grip object and put in mouth is interesting, and can be generated by the following system. 

The above is a much more complex behaviour. It assumes the existence of an already learned inverse model of the arm and mouth, allowing mouth contact to be made from any arm position. This model will itself have been learned with an earlier system of action atoms which might involve an atom for classifying objects sensed by the mouth into various classes. That is, the "unsupervised clustering by the mouth" atom might be the driving force for the formation of the inverse model that moves things to the mouth. Its not certain on what selection criteria such a model is learned yet. 

Examples of Simpler Primitive Action Molecules and Associated Game Atoms

TASK A: Centralise a salient object in center of visual field 
1. Atom A [input: camera, output: object position, type]
2. Atom B [input: object position, distance from centre of visual field, process: CMA-ES using distance minimisation as the criteria for selection, output: head movement]
3. Atom B' [input: object position, distance from centre of visual field, process: CMA-ES using distance minimisation as the criteria for selection + LWPR to learn an inverse model and improve it, output: head movement]
4. Game atom: Minimize distance from centre of visual field to object. 

TASK B: Touch your toes [What makes touching your toes interesting enough to learn to do?]


TASK C: Manipulate fingers on the other hand, trying to separate each finger on one hand with the fingers on the other hand.