Hi Ryan, You are correct, a human could procure model parameters without swarming, but it might take a very long time and a lot of experiments. That is what swarming was designed to do -- test out a bunch of different permutations of model parameters and see which ones are the best.
Also, a human must choose what data fields are available to a swarm in the first place. We usually do this by guessing what data might affect the values of other data over time. Swarming helps us refine exactly how much each data field contributes to the predicted field. We have found that sometimes correlations between different input data is not evident to the human eye, and swarming can uncover those correlations that a human would rarely think logical. For an interesting discussion of this, see this mailing list thread [1]. As you probably know, streams of data can have lots of different "shapes". Some have many fields, some have only one. Some fields are highly affected by the values of other fields. Some data patterns are daily, some hourly, some have no regard to time whatsoever. Finding out which input data is relevant and how to encode it is what swarming does. It also permutes over encoder parameters to find the best way to encode the input data into SDRs. This video explains it in detail [2] in case you have not seen it. Generally, you only need to swarm as a pre-processing step, and once you find good model parameters, you can feed data into an HTM model over the data's lifetime. As the patterns in the data change, the model will learn those changes online. You generally only need to re-swarm if the "shape" of the data changes. I hope this was helpful, please let me know if I answered any of your questions. If you are just getting started with NuPIC, you might want to try out Menorah [3] to run some models very quickly based upon data readily available in River View [4]. [1] http://lists.numenta.org/pipermail/nupic_lists.numenta.org/2015-October/011878.html [2] https://www.youtube.com/watch?v=xYPKjKQ4YZ0 [3] https://github.com/nupic-community/menorah [4] http://data.numenta.org/index.html Regards, --------- Matt Taylor OS Community Flag-Bearer Numenta On Fri, Dec 4, 2015 at 12:28 PM, Ryan Singer <[email protected]> wrote: > Hello NuPIC community, > > I'm just getting started. I'm excited about Nupic because the HTM model > and SDR data structure make so much sense intuitively. Finally, an approach > that doesn't feel arbitrary and over-engineered. > > However I'm confused about why swarming is necessary when configuring > models. I've been reading all the docs and I haven't yet found an > explanation of why a human can't arrive at the right model parameters > through reasoning. > > Am I missing some documentation or video that explains this? Any help is > appreciated. > > Ryan >
