The directory MB10 will be the first attempt here of implementing a fitness function for a curiosity loop that is a function of the sensorimotor history of a subset of sensor inputs, motor outputs and extra sensory streams that the predictor has access to.
Possible fitness functions for a curiosity loop.
1. Granger causality between motors and extra sensory streams. [Not sure this is very effective for the very short samples of sm histories in each episode that we're working with.]
2. Use my 2013 CogSci paper's TCN (Temporal Causal Network) to identify causality between motors and sensors.
3. Simply use linear regression with regularisation and pruning as Goren Gordon has done in
http://gorengordon.com/Publications/Hierarchical%20Construction%20erlars2011.pdf
But see also... http://www.ncbi.nlm.nih.gov/pubmed/22386787
If the weights into and out of a neuron are below a threshold then that neuron is removed. If a regression unit persists, then it is indication of a genuine correlation between the input and outputs of that unit. LR is attempted for various time-points in the time-series, e.g. s(t-2) --> m(t+1), etc... so that a large variety of models (inverse, forward, etc...) can be learned.
Crucially, trivial models are selected out by saying that if only one input to a neuron is strong, then that neuron is removed, i.e. a neuron MUST have at least two inputs that are high, thus preventing trivial regressors from taking over the population.
a. Fitness assessment of a loop: Limit models to forward models with the sm(t) predicting p(t+1) for now. This training set can be constructed from the smMatrix as follows.
#Fitness function (3) *************************************************************
#Record the sm data for this loop and consider its properties
#print(smMatrix)
#print(len(smMatrix))
net = buildNetwork(3,10,1, bias = True)
ds = SupervisedDataSet(3, 1)
trainSet = []
for index_x, x in enumerate(smMatrix):
if index_x > 0 and index_x < len(smMatrix)-1:
#trainSet.append( [smMatrix[index_x][0], smMatrix[index_x][1], smMatrix[index_x][2], smMatrix[index_x+1][3] ] )
ds.addSample(([smMatrix[index_x][0], smMatrix[index_x][1], smMatrix[index_x][2]]), (smMatrix[index_x+1][3]))
#print(trainSet)
#print(ds)
trainer = BackpropTrainer(net, ds, weightdecay=0.5)
print("error = " + str(trainer.trainUntilConvergence(maxEpochs = 10)))
The training set is the current state of the sensors and motor actions of the loop plus the next state of the predicted sensory dimension. So its a 3 input, 1 output regression that must be achieved. The above uses PyBrain to train a FNN with 1 hidden layer with 10 nodes in it, by backpropogation (self-supervised learning). At the end an error is obtained. Weight decay is implemented. Pruning can be done at the end based on thresholding, as follows.
#Fitness function (3) *************************************************************
#Record the sm data for this loop and consider its properties
#print(smMatrix)
#print(len(smMatrix))
#net = buildNetwork(3,10,1, bias = True)
net = FeedForwardNetwork()
inp = LinearLayer(3)
h1 = SigmoidLayer(10)
outp = LinearLayer(1)
# add modules
net.addOutputModule(outp)
net.addInputModule(inp)
net.addModule(h1)
# create connections
iToH = FullConnection(inp, h1)
hToO = FullConnection(h1, outp)
net.addConnection(iToH)
net.addConnection(hToO)
# finish up
net.sortModules()
ds = SupervisedDataSet(3, 1)
trainSet = []
for index_x, x in enumerate(smMatrix):
if index_x > 0 and index_x < len(smMatrix)-1:
#trainSet.append( [smMatrix[index_x][0], smMatrix[index_x][1], smMatrix[index_x][2], smMatrix[index_x+1][3] ] )
ds.addSample(([smMatrix[index_x][0], smMatrix[index_x][1], smMatrix[index_x][2]]), (smMatrix[index_x+1][3]))
#print(trainSet)
#print(ds)
trainer = BackpropTrainer(net, ds, weightdecay=0.01)
err = trainer.trainUntilConvergence(maxEpochs = 100)
#Visualize the network performance and structure.
#nn = NNregression(ds, epoinc = 10)
#nn.setupNN()
#nn.runTraining()
#self.pesos_conexiones(net)
#print("Input to hidden", iToH.params)
#print("H to output", hToO.params)
#print(iToH.params)
n1 = iToH.params
n1a = zip(*[iter(n1)]*3)
n2 = hToO.params
fit = sum(n1a[:]) + sum(n2[:])
print fit
return fit
#Fitness function (3) *************************************************************
#Record the sm data for this loop and consider its properties
#print(smMatrix)
#print(len(smMatrix))
net = buildNetwork(3,10,1, bias = True)
ds = SupervisedDataSet(3, 1)
trainSet = []
for index_x, x in enumerate(smMatrix):
if index_x > 0 and index_x < len(smMatrix)-1:
#trainSet.append( [smMatrix[index_x][0], smMatrix[index_x][1], smMatrix[index_x][2], smMatrix[index_x+1][3] ] )
ds.addSample(([smMatrix[index_x][0], smMatrix[index_x][1], smMatrix[index_x][2]]), (smMatrix[index_x+1][3]))
#print(trainSet)
#print(ds)
trainer = BackpropTrainer(net, ds, weightdecay=0.5)
print("error = " + str(trainer.trainUntilConvergence(maxEpochs = 10)))
The training set is the current state of the sensors and motor actions of the loop plus the next state of the predicted sensory dimension. So its a 3 input, 1 output regression that must be achieved. The above uses PyBrain to train a FNN with 1 hidden layer with 10 nodes in it, by backpropogation (self-supervised learning). At the end an error is obtained. Weight decay is implemented. Pruning can be done at the end based on thresholding, as follows.
#Fitness function (3) *************************************************************
#Record the sm data for this loop and consider its properties
#print(smMatrix)
#print(len(smMatrix))
#net = buildNetwork(3,10,1, bias = True)
net = FeedForwardNetwork()
inp = LinearLayer(3)
h1 = SigmoidLayer(10)
outp = LinearLayer(1)
# add modules
net.addOutputModule(outp)
net.addInputModule(inp)
net.addModule(h1)
# create connections
iToH = FullConnection(inp, h1)
hToO = FullConnection(h1, outp)
net.addConnection(iToH)
net.addConnection(hToO)
# finish up
net.sortModules()
ds = SupervisedDataSet(3, 1)
trainSet = []
for index_x, x in enumerate(smMatrix):
if index_x > 0 and index_x < len(smMatrix)-1:
#trainSet.append( [smMatrix[index_x][0], smMatrix[index_x][1], smMatrix[index_x][2], smMatrix[index_x+1][3] ] )
ds.addSample(([smMatrix[index_x][0], smMatrix[index_x][1], smMatrix[index_x][2]]), (smMatrix[index_x+1][3]))
#print(trainSet)
#print(ds)
trainer = BackpropTrainer(net, ds, weightdecay=0.01)
err = trainer.trainUntilConvergence(maxEpochs = 100)
#Visualize the network performance and structure.
#nn = NNregression(ds, epoinc = 10)
#nn.setupNN()
#nn.runTraining()
#self.pesos_conexiones(net)
#print("Input to hidden", iToH.params)
#print("H to output", hToO.params)
#print(iToH.params)
n1 = iToH.params
n1a = zip(*[iter(n1)]*3)
n2 = hToO.params
fit = sum(n1a[:]) + sum(n2[:])
print fit
return fit
The fitness still has to be modified so that the prediction networks are first pruned, i.e. neurons in the hidden layer whose input weights are below a threshold are removed. The consequences of this run should be investigated carefully. But first MUTATION must be added tomorrow.
No comments:
Post a Comment