11th January MotorBabbling7.py

1. Add all body sensors as inputs

       bodyAngles = self.motionProxy.getAngles("Body", self.useSensors)

2. Add all body motor actuation commands as outputs


   def set_motor_values(self, motorAngles):

         fractionMaxSpeed  = 0.3
         bodyLyingDefaultAngles = [-0.20406389236450195, 0.16256213188171387, 1.5784441232681274,
                                   0.2469320297241211, 0.5199840068817139, -0.052114009857177734,
                                   -1.1029877662658691, 0.1711999773979187, 0.013848066329956055,
                                   0.13503408432006836, 0.0061779022216796875, 0.47856616973876953,
                                   0.9225810170173645, 0.04912996292114258, 0.013848066329956055,
                                   -0.14568805694580078, 0.3021559715270996, -0.09232791513204575,
                                   0.9320058226585388, 0.0031099319458007812, 1.7718119621276855,
                                   -0.13503408432006836, 1.515550136566162, 0.12429594993591309,
                                   1.348344087600708, 0.14719998836517334]
         #motorAngles = bodyLyingDefaultAngles
         #Check that the specified angles are within limits
         for index,i in enumerate(self.Body):
              if motorAngles[index] < i[0]:
                   motorAngles[index] = i[0]
              if motorAngles[index] > i[1]:
                   motorAngles[index] = i[1]
                   
         self.motionProxy.angleInterpolationWithSpeed("Body", motorAngles, fractionMaxSpeed)

The motor angles are determined by a huge 3 layer FFN


         self.inputLength = 26+18
         self.outputLength = 26
         self.net = buildNetwork(self.inputLength,10,self.outputLength)

which is randomly initialised and not trained. The main loop is as follows.

        while i< ASSESSMENT_PERIOD: #Run behaviour and store the sensorimotor data
               sensedAngles = self.get_sensor_values()
               motorAngles  = self.net.activate(sensedAngles)
               self.set_motor_values(motorAngles)
               i = i + 1

Once the above two aspects were implemented one observes some damn weird motor babbling (in simulation) which if implemented in the real NAO would probably break it. A modification is made to limit the range of joint movements to the central 1/3rd of their overall range. 


      #Check that the specified angles are within limits
         for index,i in enumerate(self.Body):
              if motorAngles[index] < i[0]/3:
                   motorAngles[index] = i[0]/3
              if motorAngles[index] > i[1]/3:
                   motorAngles[index] = i[1]/3
                   

The movement is now less violent and extreme. Also I note that Webots is not doing collision detection of the legs properly. There are 736 weights, far more than CMA-ES is designed to be able to optimise. 

   res = cma.fmin(mb.calculateFitness, 0.0*(np.random.random_sample(736,)-0.5), 1, verb_disp=1)

3. Create a population of loops each defined discretely by a sensory input stream(state)/a motor output(action) stream/and an observed/goal/effect sensory consequence stream (e.g. which a predictor in the loop may try to predict)). The fitness of the actor may for example be some function of the observed sensory (effect) stream when that actor is acting, e.g. the granger causality between motor outputs (actions) and the observed (effect) sensory stream. The loop has access to a memory of the sensorimotor states that arose during the episode, but only to those which it is explicitly specified as having access to.

4. Use a genetic algorithm to evolve actors on the basis of the above fitness function. What kinds of actors evolve? For now, it might be sufficient just to have random actor weights which are not changed, i.e. each actor is effectively a random motion, but its a subset of random motions that are causally effective, i.e. where each actor has high GC on its observed sensory dimensions. This seems like a good first step in chunking the control function into meaningful components.

The above algorithm, but with a dummy fitness function is found in motorBabbling9.py and CuriosityLoop.py The inspyred GA library is used. These files can be found on github here...

https://github.com/ctf20/DarwinianNeurodynamics










#TODO: 1. Add touch sensors because then granger causality should reward action
#         policies that reward activation of the touch sensors... etc... Problem with Nao touch sensors, probably need to make an arduino glove with pressure sensors. 
#      2. Visualize the fitness increase and the final evolved controller functions.
#      3. Add a population of regression predictors with pruning (a la Goren Gordon) and use these to define       actor fitness, e.g. actors resulting in the most non-pruned (viable) predictors survive. 
#      4. Include Deep Belief Networks in the fitness function, where fitness is a property
#         of the quality of the DBN that is learned (????)
#