MB15 starts to implement the hierarchical extension of the MIND algorithm. It is the second developmental phase that takes place once a archive of s-m-p primitives has been evolved.
#Hierarchical Genotype Generation, points to 2 primitive archieved entities.
def generateLoopH(self, random, args):
#The hierarchical loop is defined by referring to
#1. N archieve individuals.
#2. Logical interactions between these elements,
# e.g. X then Y, or X and Y active simultaneously
#3. Restrictions.
#Choose two random primitive archieve elements.
#Load the archieve
primitives = pickle.load( open( "saveArchieve.p", "rb" ) )
print(len(primitives))
prim1 = random.randint(0,len(primitives)-1)
prim2 = random.randint(0,len(primitives)-1)
#Which primitives sensory state should MI be determined on,
#for fitness calculation.
pred = random.randint(0,l)
return list([prim1, prim2, pred])
The higher level genotype points to two primitive s-m-p units which will be activated together and have their fitness determined by how much MI there is between their motor command vector and their specified sensory dimension. A single sensory dimension is chosen as the entity under observation, not both (for now).
The following code implements the above fitness function
def getFitnessH(self, smMatrix): #Store the sm state into memory
fit = 0
#Fitness function (1) ******************************************
#Mutual Information between two motors and the first prediced sensor dimension
##
## sp = smMatrix[1:,2] #Only predicts the first sensory outcome.
## mot1 = smMatrix[1:,6]
## mot2 = smMatrix[1:,7]
## spQ = ent.quantise(sp,10)
## motQ1 = ent.quantise(mot1,10)
## motQ2 = ent.quantise(mot2,10)
### print(motQ1[0])
### print(motQ2[0])
## motQ = np.row_stack((motQ1[0],motQ2[0]))
### print(motQ)
## s = ent.DiscreteSystem(motQ,(2,10), spQ[0],(1,10))
## print(str(spQ[0]) + "\n" + str(motQ))
## s.calculate_entropies(method='plugin', calc=['HX', 'HXY'])
## mutInf = s.I()
## fit = mutInf
## print(fit)
## return fit
And the modified function which ( I think) calculates the MI between two 2 x 10 matrices ( 2 sensors, and 2 motors over 10 trials). How much information does knowing the two motors give about knowing the two sensors is the question I want to answer.
#Fitness function (2) ******************************************
#MI between both motors and both sensors
sp1 = smMatrix[1:,2]
sp2 = smMatrix[1:,5]
mot1 = smMatrix[1:,6]
mot2 = smMatrix[1:,7]
spQ1 = ent.quantise(sp1,10)
spQ2 = ent.quantise(sp2,10)
motQ1 = ent.quantise(mot1,10)
motQ2 = ent.quantise(mot2,10)
# print(motQ1[0])
# print(motQ2[0])
motQ = np.row_stack((motQ1[0],motQ2[0]))
spQ = np.row_stack((spQ1[0],spQ2[0]))
# print(motQ)
s = ent.DiscreteSystem(motQ,(2,10), spQ,(2,10))
print(str(spQ) + "\n" + str(motQ))
s.calculate_entropies(method='plugin', calc=['HX', 'HXY'])
mutInf = s.I()
fit = mutInf
print(fit)
return fit
It might be useful to try a more standard fitness function in order to test the universality/evolvability of the action specification. We want a wide range of actions to be evolvable. This I believe requires multiple encodings for action, multiple ways of representing what to do, from the reactive/reflex, to the goal-directed with CMA-ES used to do exploration towards the desired goal state, to goal-directed with using an already formed inverse model (e.g. learned through LWPR), or by planning trajectories through a forward model using search methods, etc...
But before we give up on the sm-groups consisting of 2 sm primitives, we should check that the MI calculation is working.
Currently (31st Jan 2013) I'm running the two developmental phases online with a 50 ts assessment period as suggested by Robin the designer of pyentropy. He said I was trying to use too short a number of trials to calculate MI between sensors and motors and so I extended this. We'll see what effect it has on the archives.
No comments:
Post a Comment