While I don't disagree with Matt, I want to point out that the Network API I mentioned in my NuPIC API email already fullfills most of Stewart's requirements. It is a core C++ library for creating graph-like networks. It will support GUI's like FBP. It can easily bind to other languages. Plus, Grok is already using this through Python bindings.
We now have a pure C++ implementation of the spatial pooler. We need a full C++ implementation of the temporal pooler to enable pure C++ CLA applications. --Subutai On Sat, Nov 16, 2013 at 12:53 PM, Matthew Taylor <[email protected]> wrote: > Stewart, I want what you are describing as well! I think we all do. > The only issue is that Numenta doesn't *need* this to support Grok at > this point. Almost all our focus (minus one or two engineers working > *part-time* on NuPIC), is on making Grok successful, so we can come > back around to working on machine intelligence again after we are > generating income. Numenta has been around more than 7 years now, and > we've mostly been burning resources on R&D. I hope you understand why > it is so important for us to monetize on NuPIC via Grok now that we > finally have the opportunity. > > That being said, I will support any efforts by the community to create > a nupic-core C++ project, even if it's started from scratch. We can > give you tooling resources, a repo on github.com/numenta (if you want > it), a [nupic-core] mailing list, etc. I really do want to keep this > development close to NuPIC as much as possible. If there is a movement > in this direction, Grok might be able to take advantage of it in the > future to build new products or streamline our current implementation. > Of course, we'd need to ensure all our current regression tests and > benchmarks pass, but that would be a good thing for the nupic-core > project anyway. > > Jeff, Subutai, and all of us knew that fragmentation was inevitable, > so we're mentally prepared for it. But we would love to try to keep > the community together and working towards the common goal. I would > especially love for the projects to be able to come back together at > some point in the future to form an ecosystem of integrated parts. > > Cheers, > --------- > Matt Taylor > OS Community Flag-Bearer > Numenta > > > On Sat, Nov 16, 2013 at 11:50 AM, Stewart Mackenzie <[email protected]> > wrote: > > Doug your words nail it precisely, that was succinct and clearly states > the > > problem. > > > > Although I don't mention APIs much in this email, it is implicit below. > I'm > > looking at higher, long term goals that simplify things greatly for each > > party. > > > > In order to make your last paragraph possible (something i deeply desire > to > > see) we need to use the right level of abstraction. Hence my suggestion > at a > > major rewrite because in order to achive the last paragraph the best > > 'paradigm' or concept to use is declarative concurrency. The current > > codebase is shared state concurrency. Its a nightmare to manage > distributed > > code using this paradigm. > > > > Hence I suggest keeping the fast C++ core algos and embedding them (via > > foreign function interface) into a language that can handle declarative > > concurrency. Luckily that language runtime can be distributed as a C++ > > library with which callback interaction is supported. > > > > This approach allows us to include NuPIC into anything that supports > talking > > to C++ libraries, get declarative concurrency, keep the fast c++ algos, > and > > give us a set of tools which allow for developing powerful abstractions > over > > the algos ie swarm using 100,000s of lightweight agents each able to do > > constraint solving. > > > > This idea is most likely too extreme for Grok developers, though... you > dont > > get anymore extreme than reverse engineering the neocortex, anyway. I > see a > > dichotomy in the sense they do cutting edge science/engineering yet want > to > > stick with familiar tools, anyway that's the impression I got for testing > > the 'rewrite water' with my previous email in this thread, nonetheless > it's > > feasible upon high-level examination. It'll require stepping into realms > > which at first are completely foreign to a shared state concurrency > > developer. > > > > To ease neuroscientists development I'm leaning towards an abstraction > which > > gives them a Flow Based Programming (FBP) environment to work in. This > > implies box-and-line drag and drop. Refer to noflojs.org for an example > FBP > > implementation. This would essentially be used to configure NuPIC. One > write > > FBPs programs to automatically reconfigure NuPIC too, setup the inputs > and > > outputs. Then deploy the FBP graph to a NuPIC C++ library which sets up > and > > executes the graph. The host program written in whatever language then > > interacts with with NuPIC by simply streaming data in and getting > > predictions and anomalies out again (later motor control). > > > > Anyway I'll be pleasantly surprised if this becomes a reality. I firmly > > believe distributed heirarchies that span the world, swap and hotswap > > trained regions will be exponentially easier to actieve by using the > right > > level of abstraction for the problem at hand. > > > > Stewart > > > > > > Doug King <[email protected]> wrote: > >> > >> Defining the API is extremely important and should be addressed now. > Much > >> of this conversation is a discussion about APIs directly or indirectly. > >> Process (C4 or other) can be address in parallel, e.g. regression test > are > >> about process ensuring correctness of APIs and algorithms. > >> > >> Designing the API will: > >> - force us to think about how code base is used / structured > >> - force us to explicitly choose to surface operational parameters that > >> configure algorithms > >> - provide a contract for development community that encourages > stability > >> - provides a model for a other language implementations > >> - provides a contract for public data structures (serialization, > >> portable trained regions) > >> - force us to make choices about where to include 'injected > >> functionality' or 'plugins' - allowing the broader community to extend > >> functionality without PRs or C4 process (encouraging organic growth of > >> ideas). > >> > >> With a properly designed API we should be able to move forward with the > >> current code base, re-factoring chunks instead of doing a complete > rewrite. > >> > >> I believe that it is important to be able build distributed networks of > >> regions and share trained regions. As we scale CLA systems up, the value > >> that goes into tuning and training models should be captured and > >> commoditized. A model and it's trained dataset should be easy to > >> instantiate, run and move across the network. Then we will have a strong > >> community of users and then we will have an ecosystem. My dream is to be > >> able to put larger models together based on existing trained models - I > want > >> to say "hey Stewart, can I get a copy of your facial-emotion detector > >> region? I'm building a speech recognition system and I need more inputs > to > >> correlate. I've already got Matt's speech-tone detector region". > >> > >> So starting with APIs is how I see we can make this happen. > >> > >> -Doug > >> > >> > >> On Fri, Nov 15, 2013 at 5:47 PM, Scott Purdy <[email protected]> wrote: > >>> > >>> Can someone explain which parts of NuPIC they think are "clean" and > which > >>> they want to get rid of? With the exception of the current effort > around the > >>> spatial pooler I don't know which parts are being referred to. > >>> > >>> > >>> On Fri, Nov 15, 2013 at 2:39 PM, Sebastian Hänisch > >>> <[email protected]> wrote: > >>>> > >>>> Some explorative redesigning can be a true benefit to the current > NuPIC, > >>>> too. Some coordinated brainstorming can lead to a better design of > 'NuNuPIC' > >>>> with an extensive API and a more modular code basis. This would be a > base > >>>> for discussion whether a complete restart is the way to go or maybe it > >>>> reveals easy ways to restructure the current version without robbing > away > >>>> the base for Grok. > >>>> > >>>> This can be done aside from NuPIC and wouldn't influence it currently > >>>> (though some ideas may find themselve implemented there directly). > >>>> > >>>> As Newcomer to NuPIC I'm just throwing in my 2 cents of course. > >>>> > >>>> Regards, > >>>> Sebastian > >>>> > >>>> > >>>> > >>>> 2013/11/15 Fergal Byrne <[email protected]> > >>>>> > >>>>> Hi Matt/Stewart, > >>>>> > >>>>> I think the radical approach is worth exploring, but keep it within > >>>>> NuPIC. You'd need to start with a design, and then build it up by > robbing > >>>>> the clean algorithms from the current NuPIC. Eventually Grok would > switch to > >>>>> the NuNuPIC. > >>>>> — > >>>>> Sent from Mailbox for iPhone > >>>>> > >>>>> > >>>>> On Fri, Nov 15, 2013 at 9:26 PM, Matthew Taylor <[email protected]> > >>>>> wrote: > >>>>>> > >>>>>> Let's not throw out the baby with the bathwater. Numenta's interests > >>>>>> lie in improving NuPIC. We won't be able to get behind a > start-from-scratch > >>>>>> effort. We just don't have the manpower, and we're quite busy > building > >>>>>> product on top of NuPIC. > >>>>>> > >>>>>> That being said, there is nothing stopping anyone else from doing > >>>>>> this. If someone extracts our core algorithms and pastes them into > another > >>>>>> project, however, the GPL must be respected, and Numenta will not > be able to > >>>>>> provide any commercial licensing options. > >>>>>> > >>>>>> --------- > >>>>>> Matt Taylor > >>>>>> OS Community Flag-Bearer > >>>>>> Numenta > >>>>>> > >>>>>> > >>>>>> On Fri, Nov 15, 2013 at 12:32 PM, Stewart Mackenzie > >>>>>> <[email protected]> wrote: > >>>>>>> > >>>>>>> Hi, > >>>>>>> > >>>>>>> Been (carefully) thinking about this more, it's occupying a > >>>>>>> disproportionately large amount of time. > >>>>>>> > >>>>>>> I'm going to go one step further: > >>>>>>> > >>>>>>> The proof of concept works, NuPIC has cracked it to a large degree. > >>>>>>> But its based on an old codebase that went through a few > fundamental > >>>>>>> architectural changes. It doesn't support serialization nor can > operate in a > >>>>>>> hierarchy. > >>>>>>> Most of the issues are about compilation, incompatibilities, the > code > >>>>>>> base is dependency heavy yet... despite all this the algorthims > have got it > >>>>>>> right. > >>>>>>> > >>>>>>> I propose we retain the core algorithms and start from scratch. Its > >>>>>>> the algorithms that are important. > >>>>>>> > >>>>>>> What I have brewing will make a distributed HTM dramanically easier > >>>>>>> to achieve, serialization becomes trivial and it is portable. If > nupic is > >>>>>>> about to do some major move. Then lets at least think about how to > make it > >>>>>>> easier to distribute and serialize. > >>>>>>> > >>>>>>> Insane as it sounds. > >>>>>>> Stewart > >>>>>>> > >>>>>>> Matthew Taylor <[email protected]> wrote: > >>>>>>> >Thanks, Stewart. > >>>>>>> > > >>>>>>> >I think we should put the discussion about releases to the side > for > >>>>>>> >awhile, > >>>>>>> >because it's evident that we cannot release until we've nailed > down > >>>>>>> > the > >>>>>>> >API > >>>>>>> >for NuPIC. This should take priority. I'll start that discussion > >>>>>>> > soon. > >>>>>>> > > >>>>>>> >This has been quite enlightening, and I really appreciate > everyone's > >>>>>>> >input. > >>>>>>> > > >>>>>>> > >>>>>>> _______________________________________________ > >>>>>>> nupic mailing list > >>>>>>> [email protected] > >>>>>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >>>>>> > >>>>>> > >>>>> > >>>>> > >>>>> _______________________________________________ > >>>>> nupic mailing list > >>>>> [email protected] > >>>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >>>>> > >>>> > >>>> > >>>> _______________________________________________ > >>>> nupic mailing list > >>>> [email protected] > >>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >>>> > >>> > >>> > >>> _______________________________________________ > >>> nupic mailing list > >>> [email protected] > >>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >>> > >> > >> ________________________________ > >> > >> nupic mailing list > >> [email protected] > >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > > > > > > -- > > Sent from my Android device with K-9 Mail. Please excuse my brevity. > > > > _______________________________________________ > > nupic mailing list > > [email protected] > > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > > > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
