Hello NuPIC,
# HTM over HTTP
This is a project that really should exist, but does not today. It
would be a great opportunity to make HTM experimentation really easy,
and something that I would be using today if it existed. Here is a
brief spec...
## Spec
- RESTful HTTP interface for creating a small number of models
- create model: allows POST of full model params, returns model guid
- if no model params given, use anomaly detection params as default
- PUT data: given model guid, runs a row of data, returns model results
- DELETE model: given model guid, delete and clear from memory
- EASY to deploy to cloud service like Heroku or Google App Engine
Initially, this doesn't need to be robust, secure, or scale-able. I
just want something I can stand up quietly in a corner of the internet
and run experiments on live data. If it proves useful, I'll make
tutorials so others can use it. Once there are users, we can discuss
making it robust, secure, or scale-able.
## Sidenotes
- This project MUST be AGPL [1]. I am happy to discuss why in another
thread if you want to talk about licenses.
- This project does not include swarming. It is assumed that model
parameter investigation will be done offline.
- Easy deployment is a big priority. Should be something a junior
programmer could deploy with the right credentials. Docker is probably
a good idea, but include detailed instructions for deployment.
- You could use either NuPIC or HTM.Java for this, it doesn't really
matter. (I'm only excluding Comportex because Felix has mentioned
several times that he does not want to consider it stable.)
## Ideas
1. You could use HTM Engine and the HTTP interface provided by the
skeleton app to get started [2]. This might be really easy, and
provide some scalability right off the bat. The work would mostly just
be figuring out the Docker configuration for deployment. However, it
would not allow users to provide model params, and adding this
functionality would need to be done in the HTM Engine itself (others
want this too [3]).
2. You could use the simple HTTP wrapper around NuPIC provided by
Jared Weiss [4]. It is just a SimpleHttpServer but it would be a fast
prototype. Again, the work would be deployment configuration.
3. You could use HTM-Moclu [5] and HTM.Java. I haven't been able to
get this running without multiple servers (because Akka), but someone
with the know-how could take it on.
4. You could wrap a Java HTTP server over HTM.Java.
## What's the Point?
Two things, actually...
1. Internet of Things
I installed SmartThings [6] IoT devices all over our office. They are
constantly streaming live data about things like energy consumption,
door state, temperature, and motion. I have created a SmartApp that
can live-stream all this data anywhere over HTTP. I just need an HTM
to push the data to, and I should be able to build an app that
displays live charts of this data and the live anomaly scores for all
of them. This could potentially be re-used by anyone with other
SmartThings setups.
2. Big Public Streaming Temporal Data (BPSTD)
Over the break, I started a project to scan the entire Socrata
platform for temporal data streams [7] with the intention of creating
HTM models for as many as possible. I'm going to end up with hundreds
or thousands of data streams by the time I'm through with this, and I
need a place to run all those models. Standing up Heroku or GAE
instances is easy and cheap, so it is a way I can analyze lots of
streaming data with lots of models and watch how well they do over
months of time. We have not had this capability before.
So, if anyone is interesting in helping design an implement HTM over
HTTP, let's discuss here?
[1] http://www.gnu.org/licenses/agpl-3.0.en.html
[2] https://github.com/nupic-community/skeleton-htmengine-app
[3] https://github.com/numenta/numenta-apps/issues/503
[4] https://github.com/numenta/nupic/tree/master/examples/opf/simple_server
[5] https://github.com/antidata/htm-moclu
[6] http://www.smartthings.com/
[7] https://github.com/rhyolight/soda-tap
---------
Matt Taylor
OS Community Flag-Bearer
Numenta
