Ralph, The problem is that the controller dynamics interfere with the system dynamics, thus changing its overall structure. So when you apply any kind of black-box identification procedure it can mistake the real system with the controller, because the outputs will always seem to be chasing the reference point. A very extensive reference can be found in [1] (I'm still reading it). In the case of online learning, I'm not sure what is going to happen, but I haven't had good experience with offline learning using other techniques on closed loop data.
For example, once I trained an Echo State Network (powerful and easy to use recurrent neural nets) on some real closed loop data and the result seemed very good first, but when I simulated a step response, I found out the network had created an equilibrium point around the controller reference, which clearly wasn't the case for the true system. Since the controller stabilized the system around the reference, the neural net thought it was an equilibrium point, but the real reference was a very unstable place! (The true system was a pH neutralizer plant and the reference was pH = 7) Here is the MPC algorithm I propose: While control is active: Turn NuPIC learning off While cost function is still changing (cost function usually is squared difference between outputs and reference plus regularization term): 1) Guess next N inputs via some optimization procedure (N <= K) 2) Predict next K steps with the N guessed inputs (K usually around 20-30) Apply only the first of all optimized inputs to the real system Retrieve output from the real system Turn learning on and use real inputs and outputs to update NuPIC Does this seem feasible? I guess the basic scalar encoder can be used as interface. The optimization part is probably very time-consuming, but we can use warm starting to greatly increase convergence (just initialize new input guess with last input guess). One problem I see is that the number of prediction steps is actually greater than the number of applied steps to the system, so the learning would have to be done with less than the number of prediction steps. Is this possible? Pedro. [1] Forssell, Urban, and Lennart Ljung. "Closed-loop identification revisited."*Automatica* 35.7 (1999): 1215-1241. http://photon.isy.liu.se:81/research/pub-db-supplemental/phd/566/main.pdf On Thu, Aug 29, 2013 at 1:34 PM, Ralph Dratman <[email protected]>wrote: > Pedro, > > This is well expressed and I find it very interesting. Can you suggest a > reference for your point about noise behavior in a closed loop? > > You present a situation in which a cortical algorithm ought to adapt in > some way. But the algorithm you describe is not an elementary loop, and I > would like to give the implications more thought. > > Ralph > > > > On Tuesday, August 27, 2013, Pedro Tabacof wrote: > >> Hello, >> >> I've used the old NuPic for spatial classification (achieving almost the >> same error rate as a SVM classifier) and read the whitepaper on the CLA, >> but I haven't used the current implementation or been up to date on recent >> developments. >> >> I was reading the thread about the Ski game and started to think if it >> would be possible to apply NuPIC for MPC (Model Predictive Control), a >> mature and popular technology for advanced process control. The main >> difficult associated with MPC is having a reliable model of the system to >> be controlled, so if a smart online learner such as NuPIC could be used to >> model the system dynamics, it would be an interesting opportunity for >> academic or even commercial research. >> >> MPC is an optimization problem where the controlled inputs are chosen to >> minimize some cost function subject to the system dynamics and restrictions >> for a given time horizon. After the control inputs are chosen for the whole >> time window, only the next input is actually applied to the real system, >> the model is updated with the system response to the chosen input, and the >> optimization is done all over again. This picture sums it up: >> [image: Inline image 1] >> >> NuPIC could be used to predict the system outputs given the input/output >> history. The search for the best inputs on the control horizon could be >> done with a nonlinear least squares using a numerical derivative or some >> global heuristic (such as an evolutionary strategy). This of course leads >> to a local minimum, but sometimes this is more than enough. Learning would >> be off during the search, and turned on only after the next input is chosen >> and applied to the system. >> >> Is the CLA robust to white noise? Has anyone used NuPIC within a closed >> stable loop? This actually changes the noise dynamics of the system, so I'm >> not sure if the current learning algorithm can be applied indiscriminately. >> >> Do you think the current evaluation speed is fast enough for a local >> search with numerical derivatives or a global heuristic search? On more >> concrete terms, how long does it take to predict 30 time slices for 5 >> outputs on a regular PC on a moderately sized network? >> >> The inputs and outputs are all real numbers, any idea on how to convert >> this to a proper SDR? >> >> Thanks, >> Pedro. >> >> -- >> Pedro Tabacof, >> State University of Campinas, Brazil. >> > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > > -- Pedro Tabacof, Unicamp - Eng. de Computação 08.
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
