The good thing is that HTM.java holds no state in the algorithms (Designed
like that on purpose), so the algorithms can be reused and passed in the
Connections matrices relevant to the caller's context. This needs to be
experimented with of course, but this is one of the reasons I pushed for
statel
Fergal I don't think that's what he's talking about. The MappedByteBuffer (I
believe) lets you read the file is if it's in memory even though it's on disk
(it's an in-memory representation of a file?) and the file can contain anything
and for HTM.java all we have to do is serialize the Connectio
Hi Sato,
Thanks for mentioning that. Yes, that is how you get high performance out
of Java - it's essentially replacing objects with arrays as used in NuPIC.
You'd have to redesign HTM.java to do that. I'm not sure that's a great
idea - you'd be unable to innovate the algorithms flexibly.
Regards
Hi David and Fergal,
> NuPIC can do this by storing all the state in a set of C++ unboxed
arrays, and it can persist that, free the memory, and load a different
model. That's how HTM Engine (and Grok) can run lots of models. You can't
do this in a single JVM if you have live refs to your Networks,
Hi (Mike?) I'm sorry I don't remember what the "M" stands for,
Yes, you are correct - I haven't done any optimizations and emphasis was on
compatibility with the the Python version. Can you tell me how many models
you were able to run for both the traffic tutorial and ATAD?
I will take a look at
David, I'm sure we all are in the right track, the best we can do today is
experiment with different solutions and share the experience.
The way I think that HTM.java was developed (let me know if I'm wrong) was
trying to reproduce the theory and the code behind the python version
without optimiza
Not at all. I'm only talking about this use case - appearing to run 500
non-trivial, independent models in one JVM. If you really do want to run
hundreds of models, your time is worth more than the cost of deploying
hundreds of JVM instances, and something like htm-moclu is going to save
you all th
Hi Fergal,
By "unboxed" are you referring to the term used in the Java world which
refers to primitive constructs? I just wanted to be clear what you mean. So
are you saying that HTM.java should be de-objectified?
On Tue, Dec 8, 2015 at 5:34 AM, Fergal Byrne
wrote:
> Hi David,
>
> No, I'm afra
Hi David,
No, I'm afraid this is an insurmountable problem if you use objects instead
of unboxed arrays. NuPIC can do this by storing all the state in a set of
C++ unboxed arrays, and it can persist that, free the memory, and load a
different model. That's how HTM Engine (and Grok) can run lots of
Hi Fergal,
It's not that big of a deal. I just haven't done a round of profiling yet.
Therefore there is lots of room for improvement in terms of memory
handling. There are lots of JVM applications running really data heavy
applications and the state of the art JVM GC is fully capable of handling
Hi Matt,
As Stuart Holloway explains here [1], on the JVM, it's always GC. I can
barely run a single 2048x16 HTM model on my 8Gb laptop on the hotgym hourly
data - it slows to a crawl after 1000 rows because it's thrashing the GC
trying to free space on the heap (setting -Xmx2800m as JVM params he
Hey Matt, did you try ramping up from 1 model to see if it was a capacity
issue? I would be interested to see how the system responds as an
increasing number of models are added. Anyway, I can't really comment on
moclu as I don't know what's happening there and I don't have time these
days to help
David, BTW the failure in the video is a 4m: https://youtu.be/DnKxrd4TLT8?t=4m
-
Matt Taylor
OS Community Flag-Bearer
Numenta
On Mon, Dec 7, 2015 at 7:24 PM, Matthew Taylor wrote:
> David and Mike,
>
> I've moved this to another topic to discuss.
>
> So what I tried with moclu was to tak
David and Mike,
I've moved this to another topic to discuss.
So what I tried with moclu was to take the HTM engine traffic app as shown here:
https://github.com/nupic-community/htmengine-traffic-tutorial/blob/master/images/HTM-Traffic-Architecture.jpg
And I swapped out the entire green python b
14 matches
Mail list logo
