http://www.lri.fr/~hansen/gecco2011-CMA-ES-tutorial.pdf
Once the sm pairs have been found by the MI method, I select a random goal point in the sensor dimension and use the difference between the actual and desired sensor points squared as the error which cma aims to minimise.
I'm not sure the cma-es python is working.
Each generation the myObserver is called. Every 5 generations if the best fitness > threshold, an individual is put into the archive, and goal babbling is done on the sm dimensions described by that a MI-pair.
def myObserver(self, population, num_generations, num_evaluations, args):
best = max(population)
print('{0:6} -- {1} : {2}'.format(num_generations,
best.fitness,
str(best.candidate)))
#Store the best candidate in an archieve and use this to punish exisiting solutions in the population.
#1. On convergence, or simply every M generations, move the best individual into an archieve...
#2. Later the archieve may also be used to bias the varation operator (but not yet).
if num_generations%5 is 0 and best.fitness > 3.0:
self.cl.addToArchieve(best)
self.goalBabble(best.candidate)
The goal babbling is done in the goalBabble method
def goalBabble(self, x):
print("Doing goal babbling now\n")
p = self.net.params
p[:] = x[3]
i = 0
self.sMemory = np.array([1]*(INPUTSIZE + PREDICTSIZE))
self.mMemory = np.array([0]*OUTPUTSIZE)
self.rest()
sensedAngles = self.get_sensor_values()
pv = sensedAngles[x[2]]
#Choose a goal sensor value in the range of the predicted angle
self.randomGoal = self.rand.uniform(self.Body[x[1]][0],self.Body[x[1]][1]) [Probably wrong]
print(self.randomGoal)
#CMA-ES can be used to discover joint angles that get
#closer to the goal state!!!!!!!!
self.x1 = x[1]
self.x2 = x[2]
res = cma.fmin(self.calcGoalScore, 1.0*(np.random.random_sample(2,)-0.5), 0.1,maxfevals=500, verb_disp=1)
# cma.plot()
# cma.show()
which sets up a random goal within the range of the sensor dimension to be optimised. cma.fmin calls calcGoalScore with the genotype.
def calcGoalScore(self, j): #J[0] contains the joint position(s) to be tested.
#self.rest()
print("testing" + str(j) )
sensedAngles = self.get_sensor_values()
pv = sensedAngles[x[2]]
self.set_motor_values(j, self.x1)
sleep(0.3)
sensedAngles = self.get_sensor_values()
pv2 = sensedAngles[self.x2]
#Update the inverse and the forward models here.
f = pow(pv2-self.randomGoal,2)
#print( "Now pos = " + str(pv2) + "Goal = " + str(self.randomGoal) + ": Fitness = " + str(f)+ "\n")
print(self.x1, self.x2, pv, pv2, f)
return f
and returns the fitness. I would expect CMA-ES to slowly move the sensory state towards the goal, by choosing joint angles that move that sensory state towards to the goal. It is a bizarre idea to think that a real-time limb movement could be the result of a stochastic hill climbing (Evolution Strategy) type process operating in the brain, with the fitness function being the distance from a goal state.
In some cases it works (e.g. where the sensor is a joint sensor, and the motor controls that same joint), see below for example...
The predicted sensor = 0 = HeadYawAngle
The motor = 0 = HeadYaw
I think...
0 -- 3.32192809489 : [[37, 1], 0, 0, [0.5333654100063798, 0.933604268980669, 0.8597931173300151, 0.9526975609482713, 0.6131969957022982, -0.6731020480565504, -0.4814950775105089, -0.4020373978831122, -0.12491746195246933]]
Doing goal babbling now
-0.0987005123854
(3_w,6)-CMA-ES (mu_w=2.0,w_1=63%) in dimension 2 (seed=749847, Sat Jan 19 19:37:27 2013)
Iterat #Fevals function value axis ratio sigma minstd maxstd min:sec
1 7 2.652241967377137e-03 1.0e+00 4.20e-01 4e-01 4e-01 0:0.0
2 13 1.076389687783868e-02 1.1e+00 3.02e-01 2e-01 2e-01 0:0.0
3 19 8.246101768478750e-03 1.1e+00 2.46e-01 2e-01 2e-01 0:0.1
4 25 5.823238760622979e-05 1.1e+00 2.55e-01 2e-01 2e-01 0:0.1
5 31 1.893283604004319e-03 1.3e+00 3.38e-01 2e-01 3e-01 0:0.1
6 37 7.292876225381406e-04 1.8e+00 3.22e-01 2e-01 4e-01 0:0.1
7 43 3.441885100710548e-03 2.4e+00 2.74e-01 1e-01 3e-01 0:0.2
8 49 3.157467330610024e-05 2.5e+00 2.26e-01 9e-02 2e-01 0:0.2
9 55 4.944447075623953e-04 2.4e+00 1.99e-01 7e-02 2e-01 0:0.2
10 61 6.204718549890929e-05 2.6e+00 1.49e-01 5e-02 1e-01 0:0.2
Iterat #Fevals function value axis ratio sigma minstd maxstd min:sec
11 67 8.326635730545471e-06 2.6e+00 1.10e-01 3e-02 8e-02 0:0.3
12 73 9.983879241896835e-06 2.7e+00 1.03e-01 2e-02 8e-02 0:0.3
13 79 2.057312372794826e-05 3.1e+00 7.51e-02 2e-02 5e-02 0:0.3
14 85 1.481652545262251e-04 3.1e+00 5.90e-02 1e-02 3e-02 0:0.3
15 91 1.927489329556059e-05 3.0e+00 4.78e-02 9e-03 3e-02 0:0.3
16 97 2.621330436617522e-06 2.9e+00 3.46e-02 6e-03 2e-02 0:0.4
17 103 6.025269268839409e-06 3.0e+00 3.07e-02 5e-03 1e-02 0:0.4
18 109 9.809989730316828e-07 2.8e+00 2.71e-02 4e-03 1e-02 0:0.4
19 115 2.022772660155675e-09 2.7e+00 2.51e-02 3e-03 1e-02 0:0.4
20 121 1.434890005798992e-08 3.0e+00 2.10e-02 2e-03 7e-03 0:0.5
Iterat #Fevals function value axis ratio sigma minstd maxstd min:sec
21 127 3.162199983210820e-09 3.1e+00 1.55e-02 2e-03 5e-03 0:0.5
22 133 6.520401243270684e-07 3.2e+00 1.64e-02 2e-03 5e-03 0:0.5
23 139 5.319060020315956e-10 3.6e+00 1.19e-02 1e-03 3e-03 0:0.5
24 145 7.388057085505114e-09 3.6e+00 1.09e-02 8e-04 3e-03 0:0.6
25 151 3.694906990065009e-09 3.9e+00 9.26e-03 6e-04 3e-03 0:0.6
26 157 1.410363220497069e-08 4.7e+00 7.26e-03 4e-04 2e-03 0:0.6
27 163 4.384720205722738e-09 4.7e+00 6.77e-03 4e-04 2e-03 0:0.6
28 169 1.613254540715287e-08 5.0e+00 6.60e-03 3e-04 2e-03 0:0.6
29 175 2.066610511988407e-10 5.7e+00 4.69e-03 2e-04 1e-03 0:0.7
30 181 2.619591822919972e-09 5.7e+00 4.56e-03 2e-04 1e-03 0:0.7
Iterat #Fevals function value axis ratio sigma minstd maxstd min:sec
31 187 2.242049747341781e-11 6.5e+00 3.40e-03 1e-04 7e-04 0:0.7
32 193 3.292799960479909e-10 6.9e+00 2.61e-03 7e-05 5e-04 0:0.7
33 199 4.123637795796042e-10 6.9e+00 2.46e-03 6e-05 4e-04 0:0.8
34 205 3.611098517368258e-10 7.9e+00 2.85e-03 8e-05 5e-04 0:0.8
35 211 9.116572655428879e-10 5.7e+00 3.47e-03 1e-04 6e-04 0:0.8
36 217 1.926141605345461e-09 6.2e+00 3.96e-03 1e-04 8e-04 0:0.8
37 223 2.480934420783911e-11 8.6e+00 3.42e-03 1e-04 7e-04 0:0.9
38 229 1.699105877757048e-10 8.4e+00 2.91e-03 8e-05 5e-04 0:0.9
39 235 2.644782383871435e-09 8.5e+00 2.70e-03 7e-05 4e-04 0:0.9
40 241 9.583767581261623e-09 8.0e+00 2.70e-03 7e-05 4e-04 0:0.9
Iterat #Fevals function value axis ratio sigma minstd maxstd min:sec
41 247 1.875468924327829e-09 9.1e+00 3.07e-03 1e-04 5e-04 0:1.0
42 253 1.757778352778659e-10 8.4e+00 2.25e-03 6e-05 3e-04 0:1.0
43 259 1.115860172861457e-09 8.0e+00 2.05e-03 5e-05 3e-04 0:1.0
44 265 1.976131828544043e-09 8.6e+00 2.29e-03 5e-05 3e-04 0:1.0
45 271 1.087176499494000e-09 8.7e+00 2.95e-03 6e-05 4e-04 0:1.0
46 277 1.087176499494000e-09 7.6e+00 3.65e-03 7e-05 5e-04 0:1.1
47 283 1.087176499494000e-09 7.2e+00 3.36e-03 6e-05 4e-04 0:1.1
48 289 2.191694633615547e-09 7.9e+00 2.65e-03 4e-05 3e-04 0:1.1
49 295 2.191694633615547e-09 8.1e+00 2.79e-03 5e-05 3e-04 0:1.1
50 301 2.191694633615547e-09 7.1e+00 2.16e-03 3e-05 2e-04 0:1.2
Iterat #Fevals function value axis ratio sigma minstd maxstd min:sec
51 307 3.147970985046110e-09 6.8e+00 2.03e-03 3e-05 2e-04 0:1.2
52 313 3.147970985046110e-09 6.4e+00 2.38e-03 4e-05 2e-04 0:1.2
53 319 3.147970985046110e-09 8.0e+00 3.12e-03 5e-05 3e-04 0:1.2
54 325 2.665513726102889e-09 7.1e+00 4.83e-03 8e-05 5e-04 0:1.3
55 331 8.980944390109148e-09 6.3e+00 4.70e-03 7e-05 5e-04 0:1.3
56 337 3.713952812859466e-09 6.7e+00 4.38e-03 6e-05 4e-04 0:1.3
57 343 3.713952812859466e-09 7.3e+00 5.53e-03 8e-05 7e-04 0:1.3
58 349 3.861508563173864e-10 1.0e+01 4.75e-03 6e-05 6e-04 0:1.3
59 355 1.231190517506661e-09 1.2e+01 5.13e-03 6e-05 7e-04 0:1.4
60 361 5.034671558118855e-09 1.4e+01 4.83e-03 5e-05 7e-04 0:1.4
Iterat #Fevals function value axis ratio sigma minstd maxstd min:sec
61 367 6.637401765246271e-11 1.6e+01 4.12e-03 4e-05 5e-04 0:1.4
62 373 8.432317569117883e-10 1.7e+01 4.19e-03 4e-05 5e-04 0:1.4
63 379 8.432317569117883e-10 1.4e+01 4.87e-03 5e-05 5e-04 0:1.5
64 385 8.432317569117883e-10 1.0e+01 4.53e-03 4e-05 4e-04 0:1.5
65 391 8.432317569117883e-10 1.1e+01 3.85e-03 3e-05 4e-04 0:1.5
66 397 8.432317569117883e-10 1.1e+01 3.17e-03 3e-05 3e-04 0:1.5
67 403 4.108970888533294e-09 1.0e+01 4.26e-03 4e-05 4e-04 0:1.6
68 409 4.108970888533294e-09 1.1e+01 4.21e-03 4e-05 4e-04 0:1.6
69 415 4.108970888533294e-09 1.1e+01 4.20e-03 4e-05 3e-04 0:1.6
70 421 4.108970888533294e-09 9.4e+00 3.18e-03 3e-05 2e-04 0:1.6
Iterat #Fevals function value axis ratio sigma minstd maxstd min:sec
71 427 4.108970888533294e-09 9.1e+00 2.33e-03 2e-05 1e-04 0:1.6
72 433 4.108970888533294e-09 9.1e+00 2.31e-03 1e-05 1e-04 0:1.7
73 439 4.108970888533294e-09 1.0e+01 1.92e-03 1e-05 1e-04 0:1.7
74 445 4.108970888533294e-09 9.7e+00 1.55e-03 9e-06 8e-05 0:1.7
75 451 4.108970888533294e-09 9.5e+00 1.58e-03 1e-05 8e-05 0:1.7
76 457 4.108970888533294e-09 8.2e+00 1.61e-03 1e-05 7e-05 0:1.8
77 463 4.108970888533294e-09 6.6e+00 1.50e-03 1e-05 6e-05 0:1.8
78 469 4.108970888533294e-09 6.0e+00 1.72e-03 1e-05 8e-05 0:1.8
79 475 4.108970888533294e-09 7.8e+00 1.52e-03 8e-06 6e-05 0:1.8
80 481 4.108970888533294e-09 8.2e+00 1.44e-03 8e-06 5e-05 0:1.9
Iterat #Fevals function value axis ratio sigma minstd maxstd min:sec
81 487 4.108970888533294e-09 7.4e+00 1.25e-03 7e-06 4e-05 0:1.9
82 493 4.108970888533294e-09 6.8e+00 1.09e-03 5e-06 3e-05 0:1.9
83 499 4.108970888533294e-09 6.8e+00 1.23e-03 6e-06 4e-05 0:1.9
84 505 4.108970888533294e-09 6.8e+00 1.41e-03 7e-06 5e-05 0:1.9
termination on maxfevals=500 (Sat Jan 19 19:40:05 2013)
final/bestever f-value = 4.108971e-09 2.242050e-11
mean solution: [-0.09875262 -0.2743043 ]
std deviation: [ 6.87598035e-06 4.50002106e-05]
global variable pylab.rcParams['font.size'] set (from 12) to 10
The function is minimised pretty quickly, i.e. within a 100 or so moves. The standard CMA-ES graphs are shown below... [Note a dummy joint angle is used to make the genotype a vector not a single value (which the CMA-ES python function wouldn't tolerate)].
Other sm pairs do not allow goals to be reached. e.g.
5 -- 3.32192809489 : [[39, 40], 10, 14, [-0.35623017696864545, -0.48124979461386386, -0.17908115358018656, 0.6604877377439933, -0.9920692896664866, -0.20849330100670116, 0.8358292934691168, -0.6781476970786198, -0.7375323146549436]]
Doing goal babbling now
-0.14673090308
(3_w,6)-CMA-ES (mu_w=2.0,w_1=63%) in dimension 2 (seed=758752, Sat Jan 19 19:48:19 2013)
Iterat #Fevals function value axis ratio sigma minstd maxstd min:sec
1 7 2.578471888722357e-02 1.0e+00 5.35e-01 5e-01 5e-01 0:0.0
2 13 2.578454750521574e-02 1.3e+00 5.25e-01 5e-01 5e-01 0:0.0
3 19 2.578475747068609e-02 1.1e+00 5.09e-01 4e-01 5e-01 0:0.1
4 25 2.578532575758621e-02 1.1e+00 4.55e-01 4e-01 4e-01 0:0.1
5 31 2.578448648976927e-02 1.0e+00 4.32e-01 3e-01 4e-01 0:0.1
6 37 2.578421132296648e-02 1.2e+00 3.86e-01 3e-01 3e-01 0:0.1
7 43 2.578465308603628e-02 1.1e+00 4.56e-01 3e-01 4e-01 0:0.2
8 49 2.578458070482722e-02 1.3e+00 3.44e-01 2e-01 2e-01 0:0.2
9 55 2.578466594717083e-02 1.3e+00 2.61e-01 1e-01 2e-01 0:0.2
10 61 2.578469974505134e-02 1.3e+00 2.40e-01 1e-01 2e-01 0:0.2
Iterat #Fevals function value axis ratio sigma minstd maxstd min:sec
11 67 2.578507929976155e-02 1.5e+00 1.86e-01 8e-02 1e-01 0:0.3
12 73 2.578516125283952e-02 1.5e+00 1.90e-01 8e-02 1e-01 0:0.3
13 79 2.578466923722902e-02 1.5e+00 1.53e-01 6e-02 8e-02 0:0.3
14 85 2.578461779270677e-02 1.5e+00 1.21e-01 4e-02 5e-02 0:0.3
15 91 2.578504400614036e-02 1.4e+00 9.18e-02 3e-02 4e-02 0:0.3
16 97 2.578515317716547e-02 1.4e+00 8.42e-02 3e-02 3e-02 0:0.4
17 103 2.578517680599312e-02 1.4e+00 7.78e-02 2e-02 3e-02 0:0.4
18 109 2.578518248887733e-02 1.5e+00 9.01e-02 3e-02 4e-02 0:0.4
19 115 2.578518338617489e-02 2.1e+00 8.42e-02 2e-02 3e-02 0:0.4
20 121 2.578518368527408e-02 1.8e+00 9.94e-02 4e-02 4e-02 0:0.5
Iterat #Fevals function value axis ratio sigma minstd maxstd min:sec
21 127 2.578518398437328e-02 1.9e+00 1.06e-01 4e-02 5e-02 0:0.5
22 133 2.578518398437328e-02 2.1e+00 9.78e-02 3e-02 4e-02 0:0.5
23 139 2.578518398437328e-02 2.3e+00 9.66e-02 3e-02 4e-02 0:0.5
24 145 2.578518398437328e-02 2.6e+00 8.94e-02 3e-02 3e-02 0:0.6
25 151 2.578518398437328e-02 2.5e+00 8.08e-02 2e-02 3e-02 0:0.6
26 157 2.578518398437328e-02 2.6e+00 9.08e-02 3e-02 4e-02 0:0.6
27 163 2.578518398437328e-02 3.3e+00 7.06e-02 2e-02 3e-02 0:0.6
28 169 2.578518398437328e-02 3.3e+00 6.02e-02 2e-02 2e-02 0:0.6
29 175 2.578518398437328e-02 3.0e+00 4.51e-02 1e-02 1e-02 0:0.7
30 181 2.578518398437328e-02 3.1e+00 3.86e-02 9e-03 1e-02 0:0.7
Iterat #Fevals function value axis ratio sigma minstd maxstd min:sec
31 187 2.578518398437328e-02 3.3e+00 3.60e-02 8e-03 9e-03 0:0.7
32 193 2.578518398437328e-02 2.9e+00 2.99e-02 6e-03 6e-03 0:0.7
33 199 2.578518398437328e-02 2.9e+00 4.09e-02 1e-02 1e-02 0:0.8
34 205 2.578518398437328e-02 4.4e+00 3.48e-02 8e-03 1e-02 0:0.8
35 211 2.578518398437328e-02 4.8e+00 2.67e-02 5e-03 8e-03 0:0.8
36 217 2.578518398437328e-02 4.9e+00 1.99e-02 3e-03 5e-03 0:0.8
37 223 2.578518398437328e-02 4.8e+00 2.57e-02 4e-03 7e-03 0:0.8
38 229 2.578518398437328e-02 4.4e+00 3.23e-02 5e-03 9e-03 0:0.9
39 235 2.578518398437328e-02 3.9e+00 3.05e-02 4e-03 8e-03 0:0.9
40 241 2.578518398437328e-02 4.0e+00 3.29e-02 5e-03 9e-03 0:0.9
termination on flat fitness: please (re)consider how to compute the fitness more elaborate=None (Sat Jan 19 19:49:35 2013)
termination on tolfun=1e-11 (Sat Jan 19 19:49:35 2013)
termination on tolfunhist=1e-12 (Sat Jan 19 19:49:35 2013)
final/bestever f-value = 2.578518e-02 2.578421e-02
mean solution: [ 0.36231246 -0.19784393]
std deviation: [ 0.00487775 0.0088996 ]
5 -- 3.32192809489 : [[39, 40], 10, 14, [-0.35623017696864545, -0.48124979461386386, -0.17908115358018656, 0.6604877377439933, -0.9920692896664866, -0.20849330100670116, 0.8358292934691168, -0.6781476970786198, -0.7375323146549436]]
Doing goal babbling now
-0.14673090308
(3_w,6)-CMA-ES (mu_w=2.0,w_1=63%) in dimension 2 (seed=758752, Sat Jan 19 19:48:19 2013)
Iterat #Fevals function value axis ratio sigma minstd maxstd min:sec
1 7 2.578471888722357e-02 1.0e+00 5.35e-01 5e-01 5e-01 0:0.0
2 13 2.578454750521574e-02 1.3e+00 5.25e-01 5e-01 5e-01 0:0.0
3 19 2.578475747068609e-02 1.1e+00 5.09e-01 4e-01 5e-01 0:0.1
4 25 2.578532575758621e-02 1.1e+00 4.55e-01 4e-01 4e-01 0:0.1
5 31 2.578448648976927e-02 1.0e+00 4.32e-01 3e-01 4e-01 0:0.1
6 37 2.578421132296648e-02 1.2e+00 3.86e-01 3e-01 3e-01 0:0.1
7 43 2.578465308603628e-02 1.1e+00 4.56e-01 3e-01 4e-01 0:0.2
8 49 2.578458070482722e-02 1.3e+00 3.44e-01 2e-01 2e-01 0:0.2
9 55 2.578466594717083e-02 1.3e+00 2.61e-01 1e-01 2e-01 0:0.2
10 61 2.578469974505134e-02 1.3e+00 2.40e-01 1e-01 2e-01 0:0.2
Iterat #Fevals function value axis ratio sigma minstd maxstd min:sec
11 67 2.578507929976155e-02 1.5e+00 1.86e-01 8e-02 1e-01 0:0.3
12 73 2.578516125283952e-02 1.5e+00 1.90e-01 8e-02 1e-01 0:0.3
13 79 2.578466923722902e-02 1.5e+00 1.53e-01 6e-02 8e-02 0:0.3
14 85 2.578461779270677e-02 1.5e+00 1.21e-01 4e-02 5e-02 0:0.3
15 91 2.578504400614036e-02 1.4e+00 9.18e-02 3e-02 4e-02 0:0.3
16 97 2.578515317716547e-02 1.4e+00 8.42e-02 3e-02 3e-02 0:0.4
17 103 2.578517680599312e-02 1.4e+00 7.78e-02 2e-02 3e-02 0:0.4
18 109 2.578518248887733e-02 1.5e+00 9.01e-02 3e-02 4e-02 0:0.4
19 115 2.578518338617489e-02 2.1e+00 8.42e-02 2e-02 3e-02 0:0.4
20 121 2.578518368527408e-02 1.8e+00 9.94e-02 4e-02 4e-02 0:0.5
Iterat #Fevals function value axis ratio sigma minstd maxstd min:sec
21 127 2.578518398437328e-02 1.9e+00 1.06e-01 4e-02 5e-02 0:0.5
22 133 2.578518398437328e-02 2.1e+00 9.78e-02 3e-02 4e-02 0:0.5
23 139 2.578518398437328e-02 2.3e+00 9.66e-02 3e-02 4e-02 0:0.5
24 145 2.578518398437328e-02 2.6e+00 8.94e-02 3e-02 3e-02 0:0.6
25 151 2.578518398437328e-02 2.5e+00 8.08e-02 2e-02 3e-02 0:0.6
26 157 2.578518398437328e-02 2.6e+00 9.08e-02 3e-02 4e-02 0:0.6
27 163 2.578518398437328e-02 3.3e+00 7.06e-02 2e-02 3e-02 0:0.6
28 169 2.578518398437328e-02 3.3e+00 6.02e-02 2e-02 2e-02 0:0.6
29 175 2.578518398437328e-02 3.0e+00 4.51e-02 1e-02 1e-02 0:0.7
30 181 2.578518398437328e-02 3.1e+00 3.86e-02 9e-03 1e-02 0:0.7
Iterat #Fevals function value axis ratio sigma minstd maxstd min:sec
31 187 2.578518398437328e-02 3.3e+00 3.60e-02 8e-03 9e-03 0:0.7
32 193 2.578518398437328e-02 2.9e+00 2.99e-02 6e-03 6e-03 0:0.7
33 199 2.578518398437328e-02 2.9e+00 4.09e-02 1e-02 1e-02 0:0.8
34 205 2.578518398437328e-02 4.4e+00 3.48e-02 8e-03 1e-02 0:0.8
35 211 2.578518398437328e-02 4.8e+00 2.67e-02 5e-03 8e-03 0:0.8
36 217 2.578518398437328e-02 4.9e+00 1.99e-02 3e-03 5e-03 0:0.8
37 223 2.578518398437328e-02 4.8e+00 2.57e-02 4e-03 7e-03 0:0.8
38 229 2.578518398437328e-02 4.4e+00 3.23e-02 5e-03 9e-03 0:0.9
39 235 2.578518398437328e-02 3.9e+00 3.05e-02 4e-03 8e-03 0:0.9
40 241 2.578518398437328e-02 4.0e+00 3.29e-02 5e-03 9e-03 0:0.9
termination on flat fitness: please (re)consider how to compute the fitness more elaborate=None (Sat Jan 19 19:49:35 2013)
termination on tolfun=1e-11 (Sat Jan 19 19:49:35 2013)
termination on tolfunhist=1e-12 (Sat Jan 19 19:49:35 2013)
final/bestever f-value = 2.578518e-02 2.578421e-02
mean solution: [ 0.36231246 -0.19784393]
std deviation: [ 0.00487775 0.0088996 ]
fit mod = 0
10 -- 3.32192809489 : [[18, 1], 23, 39, [-0.3064580366589219, 0.09198015281875083, 0.4626983635299074, 1.1927013898816232, -0.26221069629622873, -0.5592647250808256, -0.7929659467484068, -1.4025487061521766, -1.1350023177411117]]
Doing goal babbling now
0.272433873931
WARNING: initial solution is out of the domain boundaries:
x0 = [-0.22447999 -0.34060958]
ldom = [ 0.03490658 0.03490658]
udom = [ 1.54461634 1.54461634]
(3_w,6)-CMA-ES (mu_w=2.0,w_1=63%) in dimension 2 (seed=733478, Sat Jan 19 19:51:53 2013)
Iterat #Fevals function value axis ratio sigma minstd maxstd min:sec
1 7 8.763450672760229e+01 1.0e+00 4.76e-01 4e-01 5e-01 0:0.0
2 13 8.704434970220727e+01 1.2e+00 4.47e-01 4e-01 4e-01 0:0.0
3 19 8.780453606351549e+01 1.2e+00 4.69e-01 4e-01 5e-01 0:0.1
4 25 8.696631789996867e+01 1.1e+00 5.04e-01 5e-01 5e-01 0:0.1
5 31 8.671649959919993e+01 1.2e+00 6.02e-01 5e-01 6e-01 0:0.1
6 37 8.690590555011211e+01 1.2e+00 8.66e-01 9e-01 1e+00 0:0.1
7 43 8.921141288347522e+01 1.5e+00 9.42e-01 9e-01 1e+00 0:0.2
8 49 8.671976775586187e+01 1.4e+00 8.52e-01 8e-01 9e-01 0:0.2
9 55 8.743193033139136e+01 1.4e+00 7.49e-01 6e-01 7e-01 0:0.2
10 61 8.737292536169839e+01 1.4e+00 6.14e-01 4e-01 6e-01 0:0.2
Iterat #Fevals function value axis ratio sigma minstd maxstd min:sec
11 67 8.664924963895793e+01 1.4e+00 5.96e-01 4e-01 5e-01 0:0.3
12 73 8.706634583230355e+01 1.5e+00 5.34e-01 3e-01 4e-01 0:0.3
13 79 8.670838275913273e+01 1.6e+00 4.29e-01 2e-01 3e-01 0:0.3
14 85 8.673159757690530e+01 1.7e+00 4.41e-01 3e-01 4e-01 0:0.3
15 91 8.705340764253134e+01 2.0e+00 3.95e-01 2e-01 3e-01 0:0.4
16 97 8.725780803797282e+01 1.9e+00 4.02e-01 2e-01 3e-01 0:0.4
17 103 8.708298714581950e+01 1.8e+00 5.94e-01 3e-01 4e-01 0:0.4
18 109 8.711812600791532e+01 2.0e+00 4.31e-01 2e-01 3e-01 0:0.4
19 115 8.667753512848648e+01 2.2e+00 3.80e-01 2e-01 3e-01 0:0.5
20 121 8.743638905612447e+01 2.3e+00 3.28e-01 2e-01 2e-01 0:0.5
Iterat #Fevals function value axis ratio sigma minstd maxstd min:sec
21 127 8.670797426299950e+01 2.5e+00 2.64e-01 1e-01 2e-01 0:0.5
22 133 8.698430160743499e+01 2.6e+00 2.29e-01 1e-01 1e-01 0:0.5
23 139 8.879092916530378e+01 2.6e+00 1.75e-01 7e-02 1e-01 0:0.5
24 145 8.725315788807058e+01 2.8e+00 2.37e-01 1e-01 2e-01 0:0.6
25 151 8.734589921472545e+01 4.3e+00 2.09e-01 1e-01 2e-01 0:0.6
26 157 8.697040898557462e+01 4.2e+00 2.46e-01 1e-01 2e-01 0:0.6
27 163 8.684067952289004e+01 5.6e+00 2.04e-01 1e-01 2e-01 0:0.6
28 169 8.703673356702852e+01 5.3e+00 2.00e-01 1e-01 1e-01 0:0.7
29 175 8.822180257384782e+01 5.5e+00 1.79e-01 9e-02 1e-01 0:0.7
30 181 8.689415274586472e+01 5.5e+00 1.61e-01 8e-02 1e-01 0:0.7
Iterat #Fevals function value axis ratio sigma minstd maxstd min:sec
31 187 8.662707548102466e+01 5.8e+00 1.67e-01 9e-02 1e-01 0:0.7
32 193 8.781316875784864e+01 7.1e+00 1.56e-01 8e-02 1e-01 0:0.8
33 199 8.690743471771579e+01 6.2e+00 1.40e-01 7e-02 9e-02 0:0.8
34 205 8.670497273396656e+01 6.8e+00 1.29e-01 6e-02 7e-02 0:0.8
35 211 8.696078619375966e+01 6.0e+00 1.68e-01 7e-02 9e-02 0:0.8
36 217 8.671280523539085e+01 4.2e+00 1.66e-01 6e-02 8e-02 0:0.9
37 223 8.736128363941772e+01 3.7e+00 1.30e-01 4e-02 6e-02 0:0.9
38 229 8.676476445441527e+01 3.8e+00 1.35e-01 4e-02 6e-02 0:0.9
39 235 8.758526867116835e+01 3.3e+00 1.17e-01 3e-02 5e-02 0:0.9
40 241 8.700705517319133e+01 3.0e+00 1.03e-01 2e-02 4e-02 0:0.9
Iterat #Fevals function value axis ratio sigma minstd maxstd min:sec
41 247 8.674756733866448e+01 2.5e+00 1.25e-01 3e-02 4e-02 0:1.0
42 253 8.856603502701232e+01 2.0e+00 1.12e-01 3e-02 3e-02 0:1.0
43 259 8.686942288996117e+01 2.1e+00 1.13e-01 3e-02 3e-02 0:1.0
44 265 8.690704353402498e+01 1.8e+00 1.33e-01 3e-02 3e-02 0:1.0
45 271 8.682724269718068e+01 1.5e+00 1.62e-01 4e-02 5e-02 0:1.1
46 277 8.677345248835832e+01 1.9e+00 2.46e-01 7e-02 7e-02 0:1.1
47 283 8.681186982662405e+01 2.0e+00 2.74e-01 7e-02 8e-02 0:1.1
48 289 8.818737320580340e+01 2.0e+00 3.34e-01 9e-02 9e-02 0:1.1
49 295 8.693407284172758e+01 2.3e+00 3.95e-01 1e-01 1e-01 0:1.2
50 301 8.759126647597510e+01 2.7e+00 6.05e-01 2e-01 2e-01 0:1.2
Iterat #Fevals function value axis ratio sigma minstd maxstd min:sec
51 307 8.696580207426696e+01 2.7e+00 5.86e-01 1e-01 2e-01 0:1.2
52 313 8.672438590831101e+01 2.8e+00 6.60e-01 2e-01 2e-01 0:1.2
53 319 8.757786095166529e+01 3.0e+00 7.69e-01 2e-01 2e-01 0:1.3
54 325 8.672818709527463e+01 3.7e+00 1.30e+00 3e-01 4e-01 0:1.3
55 331 8.738775943017144e+01 3.4e+00 1.17e+00 3e-01 3e-01 0:1.3
56 337 8.668137080448012e+01 3.2e+00 1.14e+00 2e-01 3e-01 0:1.3
57 343 8.673944904469178e+01 3.0e+00 1.51e+00 4e-01 4e-01 0:1.3
58 349 8.775541106606579e+01 3.1e+00 1.64e+00 4e-01 4e-01 0:1.4
59 355 8.729540575387844e+01 3.3e+00 1.73e+00 4e-01 4e-01 0:1.4
60 361 8.775151596519309e+01 2.8e+00 1.84e+00 3e-01 4e-01 0:1.4
Iterat #Fevals function value axis ratio sigma minstd maxstd min:sec
61 367 8.703294341832354e+01 2.7e+00 1.78e+00 3e-01 4e-01 0:1.4
62 373 8.717394621468340e+01 2.5e+00 1.46e+00 2e-01 3e-01 0:1.5
63 379 8.721359208584528e+01 2.4e+00 1.26e+00 2e-01 2e-01 0:1.5
64 385 8.791906547885630e+01 2.5e+00 1.03e+00 1e-01 2e-01 0:1.5
65 391 8.704709016881114e+01 2.5e+00 1.07e+00 1e-01 2e-01 0:1.5
66 397 8.733973156395905e+01 2.9e+00 9.20e-01 9e-02 1e-01 0:1.6
67 403 8.721111618135517e+01 3.1e+00 9.83e-01 1e-01 2e-01 0:1.6
68 409 8.689001011626792e+01 3.4e+00 8.35e-01 8e-02 1e-01 0:1.6
69 415 8.900995411522680e+01 3.2e+00 7.95e-01 8e-02 1e-01 0:1.6
70 421 8.736267418719946e+01 3.2e+00 8.73e-01 1e-01 1e-01 0:1.7
Iterat #Fevals function value axis ratio sigma minstd maxstd min:sec
71 427 8.728462453802481e+01 3.4e+00 9.30e-01 9e-02 1e-01 0:1.7
72 433 8.687536057884751e+01 2.9e+00 7.43e-01 7e-02 8e-02 0:1.7
73 439 8.747443551845343e+01 3.0e+00 7.59e-01 8e-02 8e-02 0:1.7
74 445 8.684586968627232e+01 3.7e+00 6.86e-01 6e-02 7e-02 0:1.8
75 451 8.714135999853150e+01 3.9e+00 6.78e-01 6e-02 7e-02 0:1.8
76 457 8.690901724254077e+01 3.5e+00 9.79e-01 9e-02 1e-01 0:1.8
77 463 8.701089813318978e+01 4.6e+00 1.48e+00 1e-01 2e-01 0:1.9
78 469 8.793484012690277e+01 5.0e+00 1.85e+00 2e-01 3e-01 0:1.9
79 475 8.715710033908950e+01 5.6e+00 1.94e+00 2e-01 3e-01 0:1.9
80 481 8.665082981135851e+01 5.7e+00 1.60e+00 1e-01 2e-01 0:1.9
Iterat #Fevals function value axis ratio sigma minstd maxstd min:sec
81 487 8.734286883887091e+01 5.4e+00 1.19e+00 9e-02 1e-01 0:2.0
82 493 8.710520177258194e+01 5.8e+00 1.02e+00 7e-02 1e-01 0:2.0
83 499 8.685457955551675e+01 5.5e+00 9.83e-01 6e-02 9e-02 0:2.0
84 505 8.678971040737940e+01 5.0e+00 7.29e-01 4e-02 6e-02 0:2.0
termination on maxfevals=500 (Sat Jan 19 19:54:32 2013)
final/bestever f-value = 8.940085e+01 8.662708e+01
mean solution: [ 0.99619572 1.68646145]
std deviation: [ 0.05669508 0.04256809]
This kind of failure appears to be due to the fact that the sensor is e.g. an accelerometer and it does not respond to the joint angle, but to joint velocity.
To achieve the desired accelerometer value the CMA-ES should search not over joint angles, but also over velocities.
Look into dynamical movement primitives as a motor encoding method.
http://www-clmc.usc.edu/Research/MotorPrimitives
To achieve the desired accelerometer value the CMA-ES should search not over joint angles, but also over velocities.
Look into dynamical movement primitives as a motor encoding method.
http://www-clmc.usc.edu/Research/MotorPrimitives
No comments:
Post a Comment