Pedro, I wish I could advise you on the feasibility of your plan, but I don't yet know enough about CLA to be of any help. I am not part of Numenta. I am only an onlooker.
On Friday, August 30, 2013, Pedro Tabacof wrote: > Ralph, > > The problem is that the controller dynamics interfere with the system > dynamics, thus changing its overall structure. So when you apply any kind > of black-box identification procedure it can mistake the real system with > the controller, because the outputs will always seem to be chasing the > reference point. A very extensive reference can be found in [1] (I'm still > reading it). In the case of online learning, I'm not sure what is going to > happen, but I haven't had good experience with offline learning using other > techniques on closed loop data. > > For example, once I trained an Echo State Network (powerful and easy to > use recurrent neural nets) on some real closed loop data and the result > seemed very good first, but when I simulated a step response, I found out > the network had created an equilibrium point around the controller > reference, which clearly wasn't the case for the true system. Since the > controller stabilized the system around the reference, the neural net > thought it was an equilibrium point, but the real reference was a very > unstable place! (The true system was a pH neutralizer plant and the > reference was pH = 7) > > Here is the MPC algorithm I propose: > > While control is active: > > Turn NuPIC learning off > > While cost function is still changing (cost function usually is squared > difference between outputs and reference plus regularization term): > > 1) Guess next N inputs via some optimization procedure (N <= K) > > 2) Predict next K steps with the N guessed inputs (K usually around 20-30) > > Apply only the first of all optimized inputs to the real system > Retrieve output from the real system > Turn learning on and use real inputs and outputs to update NuPIC > > Does this seem feasible? I guess the basic scalar encoder can be used as > interface. The optimization part is probably very time-consuming, but we > can use warm starting to greatly increase convergence (just initialize new > input guess with last input guess). > > One problem I see is that the number of prediction steps is actually > greater than the number of applied steps to the system, so the learning > would have to be done with less than the number of prediction steps. Is > this possible? > > Pedro. > > [1] Forssell, Urban, and Lennart Ljung. "Closed-loop identification > revisited."*Automatica* 35.7 (1999): 1215-1241. > http://photon.isy.liu.se:81/research/pub-db-supplemental/phd/566/main.pdf > > > On Thu, Aug 29, 2013 at 1:34 PM, Ralph Dratman > <[email protected]<javascript:_e({}, 'cvml', > '[email protected]');> > > wrote: > >> Pedro, >> >> This is well expressed and I find it very interesting. Can you suggest a >> reference for your point about noise behavior in a closed loop? >> >> You present a situation in which a cortical algorithm ought to adapt in >> some way. But the algorithm you describe is not an elementary loop, and >> I would like to give the implications more thought. >> >> Ralph >> >> >> >> On Tuesday, August 27, 2013, Pedro Tabacof wrote: >> >>> Hello, >>> >>> I've used the old NuPic for spatial classification (achieving almost the >>> same error rate as a SVM classifier) and read the whitepaper on the CLA, >>> but I haven't used the current implementation or been up to date on recent >>> developments. >>> >>> I was reading the thread about the Ski game and started to think if it >>> would be possible to apply NuPIC for MPC (Model Predictive Control), a >>> mature and popular technology for advanced process control. The main >>> difficult associated with MPC is having a reliable model of the system to >>> be controlled, so if a smart online learner such as NuPIC could be used to >>> model the system dynamics, it would be an interesting opportunity for >>> academic or even commercial research. >>> >>> MPC is an optimization problem where the controlled inputs are chosen to >>> minimize some cost function subject to the system dynamics and restrictions >>> for a given time horizon. After the control inputs are chosen for the whole >>> time window, only the next input is actually applied to the real system, >>> the model is updated with the system response to the chosen input, and the >>> optimization is done all over again. This picture sums it up: >>> [image: Inline image 1] >>> >>> NuPIC could be used to predict the system outputs given the input/output >>> history. The search for the best inputs on the control horizon could be >>> done with a nonlinear least squares using a numerical derivative or some >>> global heuristic (such as an evolutionary strategy). This of course leads >>> to a local minimum, but sometimes this is more than enough. Learning would >>> be off during the search, and turned on only after the next input is chosen >>> and applied to the system. >>> >>> Is the CLA robust to white noise? Has anyone used NuPIC within a closed >>> stable loop? This actually changes the noise dynamics of the system, so I'm >>> not sure if the current learning algorithm can be applied indiscriminately. >>> >>> Do you think the current evaluation speed is fast enough for a local >>> search with numerical derivatives or a global heuristic search? On more >>> concrete terms, how long does it take to predict 30 time slices for 5 >>> outputs on a regular PC on a moderately sized network? >>> >>> The inputs and outputs are all real numbers, any idea on how to convert >>> this to a proper SDR? >>> >>> Thanks, >>> Pedro. >>> >>> -- >>> Pedro Tabacof, >>> State University of Campinas, Brazil. >>> >> >> _______________________________________________ >> nupic mailing list >> [email protected] <javascript:_e({}, 'cvml', >> '[email protected]');> >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> >> > > > -- > Pedro Tabacof, > Unicamp - Eng. de Computação 08. >
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
