Guys, This is great, and just shows how far we've come in just six months of Numenta opening up NuPIC. I think we can make this work for everyone, and bring NuPIC to a point where it is a commodity technology like Stewart talks about. In order to do this, there are a couple of constraints which are worth nailing down right now:
1. Grok/Numenta (as a company) states what it needs NuPIC to have API-wise. If we have this, and regardless of what we change NuPIC to do, Grok will always have "their" NuPIC as a basis for the work they're doing. Anything we do that might break this contract, we'll do with Grok's agreement because it improves their product in some way. 2. We should come up with a standardised design for interchanging models and SDR's as text files. If we do this, we can build the distributed, hierarchical architecture that Stewart is talking about. We should care less about efficiency and most about ubiquity in this endeavour. 3. We should extend the concept of the OPF to be a truly modular "brain-building' system. If we can plug in any kind of "neural region" the we can piggy-back on the work already done by other people. We might end up creating an open standard for running models and interchanging data among different ML systems. It's really important to understand what Jeff and co have done by opening up NuPIC. They've recognised that they can't get anywhere near what HTM/NuPIC etc can do by just using a small company (even if you guys are among the best at what you do!) - there just isn't enough time in the day. What Marek is doing just on his own proves that this model works. What Chetan is doing (quadcopterwise and now outreachwise) proves that we can extend this stuff exponentially. Kind Regards, Fergal Byrne On Sat, Nov 16, 2013 at 9:37 PM, Subutai Ahmad <[email protected]> wrote: > > While I don't disagree with Matt, I want to point out that the Network API > I mentioned in my NuPIC API email already fullfills most of Stewart's > requirements. It is a core C++ library for creating graph-like networks. It > will support GUI's like FBP. It can easily bind to other languages. Plus, > Grok is already using this through Python bindings. > > We now have a pure C++ implementation of the spatial pooler. We need a > full C++ implementation of the temporal pooler to enable pure C++ CLA > applications. > > --Subutai > > > On Sat, Nov 16, 2013 at 12:53 PM, Matthew Taylor <[email protected]> wrote: > >> Stewart, I want what you are describing as well! I think we all do. >> The only issue is that Numenta doesn't *need* this to support Grok at >> this point. Almost all our focus (minus one or two engineers working >> *part-time* on NuPIC), is on making Grok successful, so we can come >> back around to working on machine intelligence again after we are >> generating income. Numenta has been around more than 7 years now, and >> we've mostly been burning resources on R&D. I hope you understand why >> it is so important for us to monetize on NuPIC via Grok now that we >> finally have the opportunity. >> >> That being said, I will support any efforts by the community to create >> a nupic-core C++ project, even if it's started from scratch. We can >> give you tooling resources, a repo on github.com/numenta (if you want >> it), a [nupic-core] mailing list, etc. I really do want to keep this >> development close to NuPIC as much as possible. If there is a movement >> in this direction, Grok might be able to take advantage of it in the >> future to build new products or streamline our current implementation. >> Of course, we'd need to ensure all our current regression tests and >> benchmarks pass, but that would be a good thing for the nupic-core >> project anyway. >> >> Jeff, Subutai, and all of us knew that fragmentation was inevitable, >> so we're mentally prepared for it. But we would love to try to keep >> the community together and working towards the common goal. I would >> especially love for the projects to be able to come back together at >> some point in the future to form an ecosystem of integrated parts. >> >> Cheers, >> --------- >> Matt Taylor >> OS Community Flag-Bearer >> Numenta >> >> >> On Sat, Nov 16, 2013 at 11:50 AM, Stewart Mackenzie <[email protected]> >> wrote: >> > Doug your words nail it precisely, that was succinct and clearly states >> the >> > problem. >> > >> > Although I don't mention APIs much in this email, it is implicit below. >> I'm >> > looking at higher, long term goals that simplify things greatly for each >> > party. >> > >> > In order to make your last paragraph possible (something i deeply >> desire to >> > see) we need to use the right level of abstraction. Hence my suggestion >> at a >> > major rewrite because in order to achive the last paragraph the best >> > 'paradigm' or concept to use is declarative concurrency. The current >> > codebase is shared state concurrency. Its a nightmare to manage >> distributed >> > code using this paradigm. >> > >> > Hence I suggest keeping the fast C++ core algos and embedding them (via >> > foreign function interface) into a language that can handle declarative >> > concurrency. Luckily that language runtime can be distributed as a C++ >> > library with which callback interaction is supported. >> > >> > This approach allows us to include NuPIC into anything that supports >> talking >> > to C++ libraries, get declarative concurrency, keep the fast c++ algos, >> and >> > give us a set of tools which allow for developing powerful abstractions >> over >> > the algos ie swarm using 100,000s of lightweight agents each able to do >> > constraint solving. >> > >> > This idea is most likely too extreme for Grok developers, though... you >> dont >> > get anymore extreme than reverse engineering the neocortex, anyway. I >> see a >> > dichotomy in the sense they do cutting edge science/engineering yet >> want to >> > stick with familiar tools, anyway that's the impression I got for >> testing >> > the 'rewrite water' with my previous email in this thread, nonetheless >> it's >> > feasible upon high-level examination. It'll require stepping into realms >> > which at first are completely foreign to a shared state concurrency >> > developer. >> > >> > To ease neuroscientists development I'm leaning towards an abstraction >> which >> > gives them a Flow Based Programming (FBP) environment to work in. This >> > implies box-and-line drag and drop. Refer to noflojs.org for an >> example FBP >> > implementation. This would essentially be used to configure NuPIC. One >> write >> > FBPs programs to automatically reconfigure NuPIC too, setup the inputs >> and >> > outputs. Then deploy the FBP graph to a NuPIC C++ library which sets up >> and >> > executes the graph. The host program written in whatever language then >> > interacts with with NuPIC by simply streaming data in and getting >> > predictions and anomalies out again (later motor control). >> > >> > Anyway I'll be pleasantly surprised if this becomes a reality. I firmly >> > believe distributed heirarchies that span the world, swap and hotswap >> > trained regions will be exponentially easier to actieve by using the >> right >> > level of abstraction for the problem at hand. >> > >> > Stewart >> > >> > >> > Doug King <[email protected]> wrote: >> >> >> >> Defining the API is extremely important and should be addressed now. >> Much >> >> of this conversation is a discussion about APIs directly or indirectly. >> >> Process (C4 or other) can be address in parallel, e.g. regression test >> are >> >> about process ensuring correctness of APIs and algorithms. >> >> >> >> Designing the API will: >> >> - force us to think about how code base is used / structured >> >> - force us to explicitly choose to surface operational parameters >> that >> >> configure algorithms >> >> - provide a contract for development community that encourages >> stability >> >> - provides a model for a other language implementations >> >> - provides a contract for public data structures (serialization, >> >> portable trained regions) >> >> - force us to make choices about where to include 'injected >> >> functionality' or 'plugins' - allowing the broader community to extend >> >> functionality without PRs or C4 process (encouraging organic growth of >> >> ideas). >> >> >> >> With a properly designed API we should be able to move forward with the >> >> current code base, re-factoring chunks instead of doing a complete >> rewrite. >> >> >> >> I believe that it is important to be able build distributed networks of >> >> regions and share trained regions. As we scale CLA systems up, the >> value >> >> that goes into tuning and training models should be captured and >> >> commoditized. A model and it's trained dataset should be easy to >> >> instantiate, run and move across the network. Then we will have a >> strong >> >> community of users and then we will have an ecosystem. My dream is to >> be >> >> able to put larger models together based on existing trained models - >> I want >> >> to say "hey Stewart, can I get a copy of your facial-emotion detector >> >> region? I'm building a speech recognition system and I need more >> inputs to >> >> correlate. I've already got Matt's speech-tone detector region". >> >> >> >> So starting with APIs is how I see we can make this happen. >> >> >> >> -Doug >> >> >> >> >> >> On Fri, Nov 15, 2013 at 5:47 PM, Scott Purdy <[email protected]> >> wrote: >> >>> >> >>> Can someone explain which parts of NuPIC they think are "clean" and >> which >> >>> they want to get rid of? With the exception of the current effort >> around the >> >>> spatial pooler I don't know which parts are being referred to. >> >>> >> >>> >> >>> On Fri, Nov 15, 2013 at 2:39 PM, Sebastian Hänisch >> >>> <[email protected]> wrote: >> >>>> >> >>>> Some explorative redesigning can be a true benefit to the current >> NuPIC, >> >>>> too. Some coordinated brainstorming can lead to a better design of >> 'NuNuPIC' >> >>>> with an extensive API and a more modular code basis. This would be a >> base >> >>>> for discussion whether a complete restart is the way to go or maybe >> it >> >>>> reveals easy ways to restructure the current version without robbing >> away >> >>>> the base for Grok. >> >>>> >> >>>> This can be done aside from NuPIC and wouldn't influence it currently >> >>>> (though some ideas may find themselve implemented there directly). >> >>>> >> >>>> As Newcomer to NuPIC I'm just throwing in my 2 cents of course. >> >>>> >> >>>> Regards, >> >>>> Sebastian >> >>>> >> >>>> >> >>>> >> >>>> 2013/11/15 Fergal Byrne <[email protected]> >> >>>>> >> >>>>> Hi Matt/Stewart, >> >>>>> >> >>>>> I think the radical approach is worth exploring, but keep it within >> >>>>> NuPIC. You'd need to start with a design, and then build it up by >> robbing >> >>>>> the clean algorithms from the current NuPIC. Eventually Grok would >> switch to >> >>>>> the NuNuPIC. >> >>>>> — >> >>>>> Sent from Mailbox for iPhone >> >>>>> >> >>>>> >> >>>>> On Fri, Nov 15, 2013 at 9:26 PM, Matthew Taylor <[email protected]> >> >>>>> wrote: >> >>>>>> >> >>>>>> Let's not throw out the baby with the bathwater. Numenta's >> interests >> >>>>>> lie in improving NuPIC. We won't be able to get behind a >> start-from-scratch >> >>>>>> effort. We just don't have the manpower, and we're quite busy >> building >> >>>>>> product on top of NuPIC. >> >>>>>> >> >>>>>> That being said, there is nothing stopping anyone else from doing >> >>>>>> this. If someone extracts our core algorithms and pastes them into >> another >> >>>>>> project, however, the GPL must be respected, and Numenta will not >> be able to >> >>>>>> provide any commercial licensing options. >> >>>>>> >> >>>>>> --------- >> >>>>>> Matt Taylor >> >>>>>> OS Community Flag-Bearer >> >>>>>> Numenta >> >>>>>> >> >>>>>> >> >>>>>> On Fri, Nov 15, 2013 at 12:32 PM, Stewart Mackenzie >> >>>>>> <[email protected]> wrote: >> >>>>>>> >> >>>>>>> Hi, >> >>>>>>> >> >>>>>>> Been (carefully) thinking about this more, it's occupying a >> >>>>>>> disproportionately large amount of time. >> >>>>>>> >> >>>>>>> I'm going to go one step further: >> >>>>>>> >> >>>>>>> The proof of concept works, NuPIC has cracked it to a large >> degree. >> >>>>>>> But its based on an old codebase that went through a few >> fundamental >> >>>>>>> architectural changes. It doesn't support serialization nor can >> operate in a >> >>>>>>> hierarchy. >> >>>>>>> Most of the issues are about compilation, incompatibilities, the >> code >> >>>>>>> base is dependency heavy yet... despite all this the algorthims >> have got it >> >>>>>>> right. >> >>>>>>> >> >>>>>>> I propose we retain the core algorithms and start from scratch. >> Its >> >>>>>>> the algorithms that are important. >> >>>>>>> >> >>>>>>> What I have brewing will make a distributed HTM dramanically >> easier >> >>>>>>> to achieve, serialization becomes trivial and it is portable. If >> nupic is >> >>>>>>> about to do some major move. Then lets at least think about how >> to make it >> >>>>>>> easier to distribute and serialize. >> >>>>>>> >> >>>>>>> Insane as it sounds. >> >>>>>>> Stewart >> >>>>>>> >> >>>>>>> Matthew Taylor <[email protected]> wrote: >> >>>>>>> >Thanks, Stewart. >> >>>>>>> > >> >>>>>>> >I think we should put the discussion about releases to the side >> for >> >>>>>>> >awhile, >> >>>>>>> >because it's evident that we cannot release until we've nailed >> down >> >>>>>>> > the >> >>>>>>> >API >> >>>>>>> >for NuPIC. This should take priority. I'll start that discussion >> >>>>>>> > soon. >> >>>>>>> > >> >>>>>>> >This has been quite enlightening, and I really appreciate >> everyone's >> >>>>>>> >input. >> >>>>>>> > >> >>>>>>> >> >>>>>>> _______________________________________________ >> >>>>>>> nupic mailing list >> >>>>>>> [email protected] >> >>>>>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> >>>>>> >> >>>>>> >> >>>>> >> >>>>> >> >>>>> _______________________________________________ >> >>>>> nupic mailing list >> >>>>> [email protected] >> >>>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> >>>>> >> >>>> >> >>>> >> >>>> _______________________________________________ >> >>>> nupic mailing list >> >>>> [email protected] >> >>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> >>>> >> >>> >> >>> >> >>> _______________________________________________ >> >>> nupic mailing list >> >>> [email protected] >> >>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> >>> >> >> >> >> ________________________________ >> >> >> >> nupic mailing list >> >> [email protected] >> >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> > >> > >> > -- >> > Sent from my Android device with K-9 Mail. Please excuse my brevity. >> > >> > _______________________________________________ >> > nupic mailing list >> > [email protected] >> > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> > >> >> _______________________________________________ >> nupic mailing list >> [email protected] >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> > > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > > -- Fergal Byrne, Brenter IT <http://www.examsupport.ie>http://inbits.com - Better Living through Thoughtful Technology e:[email protected] t:+353 83 4214179 Formerly of Adnet [email protected] http://www.adnet.ie
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
