both the motor and sensory get recorded together in the entry point. then redundancies are found, how would a data model help you? well you have to think how memory playback could help the reinforcement learning.
> From: [email protected] > Subject: nupic Digest, Vol 15, Issue 12 > To: [email protected] > Date: Fri, 4 Jul 2014 12:00:06 -0400 > > Send nupic mailing list submissions to > [email protected] > > To subscribe or unsubscribe, visit > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > or, via email, send a message with subject or body 'help' to > [email protected] > > You can reach the person managing the list at > [email protected] > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of nupic digest..." > > > Today's Topics: > > 1. RNNs, Reinforcement Learning (RL) and NuPIC (Alexander Hirner) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 4 Jul 2014 16:29:12 +0200 > From: Alexander Hirner <[email protected]> > To: [email protected] > Subject: [nupic-discuss] RNNs, Reinforcement Learning (RL) and NuPIC > Message-ID: <[email protected]> > Content-Type: text/plain; charset=windows-1252 > > I've once made a test case to explore possibilities in learning dynamic, > context sensitive behavior. In essence it?s a virtual 2D environment where > the agent gets simple vision input and has simple motor actions to catch as > many yellow blobs as possible. > > https://www.youtube.com/watch?v=XvPdLdCOVGk ... more about the experiment and > task in the video description. > > My goal is to use ANN topologies in conjunction with NuPIC. I believe that > NuPIC has greater and more robust capacity to recognize and possibly (?) > replay patterns. Especially since the focus is on the time domain. My > question is how would one, at current state, incorporate NuPIC into RL with a > closed loop of sensor data and motor action? As I understand, the ?only? > output from NuPIC so far is detection of novelty and prediction. > > Because pybrain allows you to sandwich any configuration very rapidly, I > think it?s a great tool to prototype such systems. E.g. one could image > something like this: > > 1) sensors encoded in SDR ?> some NuPIC pooler ?> if novelty is detected, > turn on RL. Turn it off otherwise. > 2a) if reward is high and learning is off, feed motor signals AND sensors > encoded into another NuPIC pooler. > 2b) if learning is off, ask for prediction upon sensors encoded in SDR (and > motor signals?) > > Somehow this should even resemble the structure of the more complete CLA > theory, where a pybrain NN fulfills the role of the topmost layer. But I?m > not clear on this. > These are just some initial thoughts and I?d gladly hear some input and other > ideas. And more specifically, which encoders and which pools to use in such a > case. > > Alex > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > > > ------------------------------ > > End of nupic Digest, Vol 15, Issue 12 > *************************************
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
