Ah, yes! We renamed some of the keys in model params in https://github.com/numenta/nupic/pull/1872
"coincInputPoolPct" is now "potentialPct", for example. I've updated master in nupic.rogue with the updated params in https://github.com/numenta/nupic.rogue/pull/2/files and you should be able to pull in the latest to fix your specific problem. On Thu, Jun 4, 2015 at 7:52 AM, Michael Parco <[email protected]> wrote: > Austin this was a great rundown on the ins and outs of nupic rogue. I've > done a lot of work with rrdtool and used such agents previously to stream > metrics data collected by ganglia agents to some streaming analytics. > Although I think rrdtool is great for temporary local storage, I am looking > to possible replace it with a different backend that I can better > communicate with. > > I think the issue that I have seen thus far is that the nupic forwarder > has been giving me errors when I attempt to forward data to it. > "RuntimeError: Unknown parameter 'coincInputPoolPct' for region 'SP' of > type 'py.SPRegion'" and then gives me a list of valid parameters. This > seems to be an error from nupic within python itself and not anything to do > with nupic.rogue > > On Wed, Jun 3, 2015 at 11:12 PM, Austin Marshall <[email protected]> > wrote: > >> There are a few options, depending on what you’re trying to do. >> >> The application is structured such that a set of agents ( >> https://github.com/numenta/nupic.rogue/blob/master/avogadro/__init__.py#L39-L60) >> that >> run constantly in the background and periodically poll for metrics and save >> to a local rrdtool database. rrdtool is essentially a flat file-based >> time series database with some interesting properties. In the case of >> nupic.rogue, it’s used as a buffer between either a grok instance ( >> https://aws.amazon.com/marketplace/pp/B00I18SNQ6/ref=srh_res_product_title?ie=UTF8&sr=0-2&qid=1433386525659) >> if >> using the grok forwarder, or a nupic model if using the nupic forwarder. >> Then, the data is forwarded for analysis in a separate process. >> >> There is one major difference between the grok forwarder and nupic >> forwarder: the grok forwarder is meant to be run regularly with cron. The >> grok forwarder maintains a set of “.pos” files to keep track of position >> between runs so that it can send everything since the last run to a running >> grok instance. The nupic forwarder has no such bookkeeping and sends the >> entire batch to a freshly created model in a one-off sort of way, and saves >> the results to a cvs file locally. >> >> If you want to set it up in a streaming fashion, imagine replacing the >> rrdtool component with some sort of queue implementation (say, rabbitmq or >> redis pubsub). You could even do simple communication over a socket. The >> code is structured such that one back end can be swapped out for another >> (there only happens to be one back end right now — “rrdtool”). For >> example, each of the agents are a subclass of AvogadroAgent, which itself >> is a subclass of RRDToolClient. You could create an alternate back end >> implementation that writes to a queue, and change AvogradoAgent to be a >> subclass of your new class rather than RRDToolClient. If that’s what you’d >> like to do, I suggest starting with a copy of >> https://github.com/numenta/nupic.rogue/blob/master/avogadro/rrdtool.py, >> remove the methods prefixed with “_”, and re-implement __init__(), >> createParams(), addParseOptions(), and store() to suit your needs. Then, >> you need only write a simple script which reads from the queue and feeds >> the samples to a model you’ve created. >> >> You could also create a new forwarder which is sort of a hybrid between >> the grok and nupic forwarders. For example, use the “.pos” file approach >> of grok forwarder, and keep the script running rather than scheduled by >> cron periodically. >> >> On Jun 3, 2015, at 2:28 PM, Michael Parco <[email protected]> >> wrote: >> >> I am attempting to setup a similar system to the way grok would operate >> using nupic algorithms. Currently I git cloned the nupic.rogue github and >> built nupic.rogue using the setup python scripts. I have also built nupic >> and run some of the test examples such as hot gym and cpu predictions. >> >> I have the nupic.rogue agent running and collecting cpu I/O, network, >> memory data as rrds and I am able to execute rogue-export --prefix=var/db >> to obtain the .csv conversions of the rrd files. My next step is to feed >> the .csv files or .rrd files in an nupic model running the HTM model to run >> for predictions and anomaly scores. Ideally I would like to set this up in >> a streaming fashion on a local box. I came across the nupic_forwarder.py >> script within nupic.rogue, but I have been unable to feed in the collected >> data... any ideas? >> >> >> >
