Action grammar.

Implementation of the following FF in MB14 

Fitness function 0: One can use a naive approach and simply consider the joint probabilities of the motors being in particular bins, and label each bin with a unique state number, and map this to the joint probability bins of the sensors that are also uniquely numbered, e.g. 

00  01 10 11 may be the possible states of two motors

and 

00 01 10 11 may be the possible states of two sensors

The MI between these states is then the new fitness. This can be extended to many motors and sensors. 

http://robince.github.com/pyentropy/examples.html

But while we are coding actors with variable output motor sizes and variable sensory prediction vector sizes, we might as well re-program them to be able to deal with hierarchical predictions as well, because that will require also a complete recoding. 

Hierarchical neural units of evolution (A general framework to obtain open-ended motor representations) 

A hierarchical unit does not only take the raw sensory inputs as input to control motors but has the following additional functionality: 

1. Inputs include motor commands from other units, issued at this time step. 

2. Inputs include predictions of sensory states made by other units (if those units contain forward models)

3. Outputs include not only predictions of raw sensory data (MI or forward models)

     a. Predictions /MI calculations of/based on motor commands of other units

     b. Predictions /MI calculations of/based on predictions of other units. 

4. In all cases fitness of a unit is the MI between its motor command vector and its prediction vector. 

Q. How is action selection and production done with a hierarchical system on units each issuing motor commands that depend on each other?

A. Currently a single unit has total control for some fixed period of time, and then another unit is assessed. This is the primitive lower level of independent sm pairs. The fitness is non-interacting. 

(Leonardo based back to front thing above) 

Each smp (sense, motor, predict) unit acts in time, serially, and independently, and its fitness is only a function of itself, and no other unit of action (except for taboo effects due to the existence of smp units stored in the archive). Its execution is certainly entirely as a single unit, with no real-time interference with other units doing execution. 

How can multiple such pairs be combined in order to produce more complex action? A noble approach is that of "Options" by Barto's group. 

http://people.cs.umass.edu/~mahadeva/papers/hrl.pdf

Each unit has an initiation sensory set, an action sequence which consists of separate actions which can also execute other option actions. Such calls could be parallel or serial calls, specifying whether the other action should be executed in parallel or series with the current action. Actions may have recursive structure by this means, requiring a called action to complete prior to its continuation. It has not escaped your attention I presume that by this means we introduce systematicity and compositionally to action representations. 

Right, it looks like this Sunday will be spent thinking about how to specify evolvable recursive action representations, and making Sunday lunch.  

God. This links very nicely with the path evolution algorithm. Units call other units and it is this CHAIN or TREE of activation which is assigned fitness. I resist the temptation to remove this potentially Nobel Prize winning idea from the internet, because I assume people will think its too crazy or stupid to bother wasting their time with :) 

So for example A calls B, B calls C, then the ABC chain is the entity to which fitness is assigned, as in my completely unread paper... 

http://www.sussex.ac.uk/Users/philh/pubs/FernandoetalPLosONE2011.pdf

The amazing thing is that D can call B which calls C, and the BC component will get fitness as a result of this. In other words, action primitives OVERLAP. The use of overlapping action is the way to "make fitness count". 

So lets consider in more detail how hierarchical/recursive/network path based actions can be specified in a rich manner. The method seems very well suited to the Python method of making behaviour calls in the NAO Naoqi.api [which is not surprising since the Aldebaran guys had to make an action specification that worked in practice. This is a reassuring convergence.]

TO DO, read: http://rstb.royalsocietypublishing.org/content/367/1585/103.abstract

quotes from the above with references I must read... 

1. "two-year- old children have been found to be able not only to parse hierarchically organized actions [3], but also to copy and reproduce such actions [4]. Complex action structure (analysed as means-end parse trees) has also been found to be represented abstractly, i.e. independently of the actual semantics of the actions [5]. More strikingly, neurobiological evidence on the nature of neural circuits in the traditionally related to language-production area of the human brain (i.e. Broca’s area) provides a growing number of sugges- tions regarding the characteristics of an action grammar, such as the role of body parts/effectors, of tools and object type, and the role of the notion of ‘goal’ in human action representation [6,7]."

2. "Broca’s area has been suggested as the neural locus of an action grammar [14], an area where goals are represented and hierarchical motor chains are planned [7]."

3. "At a behaviour level, action syntax has been shown to comprise simpler elements (motor primitives) that are connected to each other either serially or in parallel (i.e. simultaneously; [15 – 19])."

4. "Reaching movements appear to be coded in terms of direction and extent, and appear to be composed of discrete submovements, all with a similar stereotypical, serially concatenated shape and overlapping in time [20 – 22]."

5. "Human and monkey grasping and object manipulation has been studied extensively and has been described as consist- ing of sub-actions executed as a unified coordinated complex act (e.g. [23])."

6. "In our analysis, we consider a human action to be a serial or parallel conjunction of perceptible movements car- ried out by one or more actors with a certain goal. We identify three main ‘morpho-syntactic’ features that characterize human actions and that we employ for defining action terminals and non-terminals."  

I don't like the formalism they use for action grammar, it is not helpful in specifying behaviours for the NAO I don't think. But the motivation is a good one. 

First draft of a formal action grammar