Stewart, I want what you are describing as well! I think we all do. The only issue is that Numenta doesn't *need* this to support Grok at this point. Almost all our focus (minus one or two engineers working *part-time* on NuPIC), is on making Grok successful, so we can come back around to working on machine intelligence again after we are generating income. Numenta has been around more than 7 years now, and we've mostly been burning resources on R&D. I hope you understand why it is so important for us to monetize on NuPIC via Grok now that we finally have the opportunity.
That being said, I will support any efforts by the community to create a nupic-core C++ project, even if it's started from scratch. We can give you tooling resources, a repo on github.com/numenta (if you want it), a [nupic-core] mailing list, etc. I really do want to keep this development close to NuPIC as much as possible. If there is a movement in this direction, Grok might be able to take advantage of it in the future to build new products or streamline our current implementation. Of course, we'd need to ensure all our current regression tests and benchmarks pass, but that would be a good thing for the nupic-core project anyway. Jeff, Subutai, and all of us knew that fragmentation was inevitable, so we're mentally prepared for it. But we would love to try to keep the community together and working towards the common goal. I would especially love for the projects to be able to come back together at some point in the future to form an ecosystem of integrated parts. Cheers, --------- Matt Taylor OS Community Flag-Bearer Numenta On Sat, Nov 16, 2013 at 11:50 AM, Stewart Mackenzie <[email protected]> wrote: > Doug your words nail it precisely, that was succinct and clearly states the > problem. > > Although I don't mention APIs much in this email, it is implicit below. I'm > looking at higher, long term goals that simplify things greatly for each > party. > > In order to make your last paragraph possible (something i deeply desire to > see) we need to use the right level of abstraction. Hence my suggestion at a > major rewrite because in order to achive the last paragraph the best > 'paradigm' or concept to use is declarative concurrency. The current > codebase is shared state concurrency. Its a nightmare to manage distributed > code using this paradigm. > > Hence I suggest keeping the fast C++ core algos and embedding them (via > foreign function interface) into a language that can handle declarative > concurrency. Luckily that language runtime can be distributed as a C++ > library with which callback interaction is supported. > > This approach allows us to include NuPIC into anything that supports talking > to C++ libraries, get declarative concurrency, keep the fast c++ algos, and > give us a set of tools which allow for developing powerful abstractions over > the algos ie swarm using 100,000s of lightweight agents each able to do > constraint solving. > > This idea is most likely too extreme for Grok developers, though... you dont > get anymore extreme than reverse engineering the neocortex, anyway. I see a > dichotomy in the sense they do cutting edge science/engineering yet want to > stick with familiar tools, anyway that's the impression I got for testing > the 'rewrite water' with my previous email in this thread, nonetheless it's > feasible upon high-level examination. It'll require stepping into realms > which at first are completely foreign to a shared state concurrency > developer. > > To ease neuroscientists development I'm leaning towards an abstraction which > gives them a Flow Based Programming (FBP) environment to work in. This > implies box-and-line drag and drop. Refer to noflojs.org for an example FBP > implementation. This would essentially be used to configure NuPIC. One write > FBPs programs to automatically reconfigure NuPIC too, setup the inputs and > outputs. Then deploy the FBP graph to a NuPIC C++ library which sets up and > executes the graph. The host program written in whatever language then > interacts with with NuPIC by simply streaming data in and getting > predictions and anomalies out again (later motor control). > > Anyway I'll be pleasantly surprised if this becomes a reality. I firmly > believe distributed heirarchies that span the world, swap and hotswap > trained regions will be exponentially easier to actieve by using the right > level of abstraction for the problem at hand. > > Stewart > > > Doug King <[email protected]> wrote: >> >> Defining the API is extremely important and should be addressed now. Much >> of this conversation is a discussion about APIs directly or indirectly. >> Process (C4 or other) can be address in parallel, e.g. regression test are >> about process ensuring correctness of APIs and algorithms. >> >> Designing the API will: >> - force us to think about how code base is used / structured >> - force us to explicitly choose to surface operational parameters that >> configure algorithms >> - provide a contract for development community that encourages stability >> - provides a model for a other language implementations >> - provides a contract for public data structures (serialization, >> portable trained regions) >> - force us to make choices about where to include 'injected >> functionality' or 'plugins' - allowing the broader community to extend >> functionality without PRs or C4 process (encouraging organic growth of >> ideas). >> >> With a properly designed API we should be able to move forward with the >> current code base, re-factoring chunks instead of doing a complete rewrite. >> >> I believe that it is important to be able build distributed networks of >> regions and share trained regions. As we scale CLA systems up, the value >> that goes into tuning and training models should be captured and >> commoditized. A model and it's trained dataset should be easy to >> instantiate, run and move across the network. Then we will have a strong >> community of users and then we will have an ecosystem. My dream is to be >> able to put larger models together based on existing trained models - I want >> to say "hey Stewart, can I get a copy of your facial-emotion detector >> region? I'm building a speech recognition system and I need more inputs to >> correlate. I've already got Matt's speech-tone detector region". >> >> So starting with APIs is how I see we can make this happen. >> >> -Doug >> >> >> On Fri, Nov 15, 2013 at 5:47 PM, Scott Purdy <[email protected]> wrote: >>> >>> Can someone explain which parts of NuPIC they think are "clean" and which >>> they want to get rid of? With the exception of the current effort around the >>> spatial pooler I don't know which parts are being referred to. >>> >>> >>> On Fri, Nov 15, 2013 at 2:39 PM, Sebastian Hänisch >>> <[email protected]> wrote: >>>> >>>> Some explorative redesigning can be a true benefit to the current NuPIC, >>>> too. Some coordinated brainstorming can lead to a better design of >>>> 'NuNuPIC' >>>> with an extensive API and a more modular code basis. This would be a base >>>> for discussion whether a complete restart is the way to go or maybe it >>>> reveals easy ways to restructure the current version without robbing away >>>> the base for Grok. >>>> >>>> This can be done aside from NuPIC and wouldn't influence it currently >>>> (though some ideas may find themselve implemented there directly). >>>> >>>> As Newcomer to NuPIC I'm just throwing in my 2 cents of course. >>>> >>>> Regards, >>>> Sebastian >>>> >>>> >>>> >>>> 2013/11/15 Fergal Byrne <[email protected]> >>>>> >>>>> Hi Matt/Stewart, >>>>> >>>>> I think the radical approach is worth exploring, but keep it within >>>>> NuPIC. You'd need to start with a design, and then build it up by robbing >>>>> the clean algorithms from the current NuPIC. Eventually Grok would switch >>>>> to >>>>> the NuNuPIC. >>>>> — >>>>> Sent from Mailbox for iPhone >>>>> >>>>> >>>>> On Fri, Nov 15, 2013 at 9:26 PM, Matthew Taylor <[email protected]> >>>>> wrote: >>>>>> >>>>>> Let's not throw out the baby with the bathwater. Numenta's interests >>>>>> lie in improving NuPIC. We won't be able to get behind a >>>>>> start-from-scratch >>>>>> effort. We just don't have the manpower, and we're quite busy building >>>>>> product on top of NuPIC. >>>>>> >>>>>> That being said, there is nothing stopping anyone else from doing >>>>>> this. If someone extracts our core algorithms and pastes them into >>>>>> another >>>>>> project, however, the GPL must be respected, and Numenta will not be >>>>>> able to >>>>>> provide any commercial licensing options. >>>>>> >>>>>> --------- >>>>>> Matt Taylor >>>>>> OS Community Flag-Bearer >>>>>> Numenta >>>>>> >>>>>> >>>>>> On Fri, Nov 15, 2013 at 12:32 PM, Stewart Mackenzie >>>>>> <[email protected]> wrote: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> Been (carefully) thinking about this more, it's occupying a >>>>>>> disproportionately large amount of time. >>>>>>> >>>>>>> I'm going to go one step further: >>>>>>> >>>>>>> The proof of concept works, NuPIC has cracked it to a large degree. >>>>>>> But its based on an old codebase that went through a few fundamental >>>>>>> architectural changes. It doesn't support serialization nor can operate >>>>>>> in a >>>>>>> hierarchy. >>>>>>> Most of the issues are about compilation, incompatibilities, the code >>>>>>> base is dependency heavy yet... despite all this the algorthims have >>>>>>> got it >>>>>>> right. >>>>>>> >>>>>>> I propose we retain the core algorithms and start from scratch. Its >>>>>>> the algorithms that are important. >>>>>>> >>>>>>> What I have brewing will make a distributed HTM dramanically easier >>>>>>> to achieve, serialization becomes trivial and it is portable. If nupic >>>>>>> is >>>>>>> about to do some major move. Then lets at least think about how to make >>>>>>> it >>>>>>> easier to distribute and serialize. >>>>>>> >>>>>>> Insane as it sounds. >>>>>>> Stewart >>>>>>> >>>>>>> Matthew Taylor <[email protected]> wrote: >>>>>>> >Thanks, Stewart. >>>>>>> > >>>>>>> >I think we should put the discussion about releases to the side for >>>>>>> >awhile, >>>>>>> >because it's evident that we cannot release until we've nailed down >>>>>>> > the >>>>>>> >API >>>>>>> >for NuPIC. This should take priority. I'll start that discussion >>>>>>> > soon. >>>>>>> > >>>>>>> >This has been quite enlightening, and I really appreciate everyone's >>>>>>> >input. >>>>>>> > >>>>>>> >>>>>>> _______________________________________________ >>>>>>> nupic mailing list >>>>>>> [email protected] >>>>>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>>>> >>>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> nupic mailing list >>>>> [email protected] >>>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>>> >>>> >>>> >>>> _______________________________________________ >>>> nupic mailing list >>>> [email protected] >>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>> >>> >>> >>> _______________________________________________ >>> nupic mailing list >>> [email protected] >>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>> >> >> ________________________________ >> >> nupic mailing list >> [email protected] >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > > > -- > Sent from my Android device with K-9 Mail. Please excuse my brevity. > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > _______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
