Since I live in Colorado and wasn't able to make it to the last hackathon, I
held my own last night and came up with this little example. It is the classic
text based skiing game where you have to move the skier left or right to keep
him on the ski slope without hitting the trees.
I have been very intrigued by the idea that motor control could be just another
form of prediction. So I thought that I could train a model on a perfect run
of the game and then use the predictions that it generates to move the skier in
a live game.
It starts by feeding the model 1000 lines of perfect skier positions in a
ramdomly generated ski slope. A slope line consists of 80 characters with two
trees as boundaries and the skier (hopefully) in the middle.
| H |
Tree Skier Tree
The slope line is encoded as three integer values to pass into the model: the
left tree position, the skier position, and the right tree position.
Given the current ski slope line, we ask the model to predict the next skier
position. If the position is greater than the current value, the skier is
moved to the right one space. If the position is less than the current value,
the skier is moved to the left one space. Otherwise, the skier position is
left alone.
You can download the code and play for yourself at:
https://github.com/keithcom/nta_ski
Building and running this test app has raised a few more questions for me.
1. Is there a string encoder yet? Originally, I wanted to just send the model
the full ski slope line, but converted it to the int array to pass in scaler
values.
2. The trained model seems to work for a little bit, but then stops as it keeps
learning during the live run. I would like the model to "see" the results of
it's prediction (i.e. how it moved the skier to complete the feedback loop),
but I also want to have some kind of error value so it knows that it's
prediction was not optimal. Does a mechanism like this exist in the current
code?
3. Most of the model settings are just copied directly from the hotgym example.
Should I change some of the values to work better in this scenario?
4. Any other comments or suggestions to improve the demonstration?
Thanks,
Matt
_______________________________________________
nupic mailing list
[email protected]
http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
