Doug your words nail it precisely, that was succinct and clearly states the
problem.
Although I don't mention APIs much in this email, it is implicit below. I'm
looking at higher, long term goals that simplify things greatly for each party.
In order to make your last paragraph possible (something i deeply desire to
see) we need to use the right level of abstraction. Hence my suggestion at a
major rewrite because in order to achive the last paragraph the best 'paradigm'
or concept to use is declarative concurrency. The current codebase is shared
state concurrency. Its a nightmare to manage distributed code using this
paradigm.
Hence I suggest keeping the fast C++ core algos and embedding them (via foreign
function interface) into a language that can handle declarative concurrency.
Luckily that language runtime can be distributed as a C++ library with which
callback interaction is supported.
This approach allows us to include NuPIC into anything that supports talking to
C++ libraries, get declarative concurrency, keep the fast c++ algos, and give
us a set of tools which allow for developing powerful abstractions over the
algos ie swarm using 100,000s of lightweight agents each able to do constraint
solving.
This idea is most likely too extreme for Grok developers, though... you dont
get anymore extreme than reverse engineering the neocortex, anyway. I see a
dichotomy in the sense they do cutting edge science/engineering yet want to
stick with familiar tools, anyway that's the impression I got for testing the
'rewrite water' with my previous email in this thread, nonetheless it's
feasible upon high-level examination. It'll require stepping into realms which
at first are completely foreign to a shared state concurrency developer.
To ease neuroscientists development I'm leaning towards an abstraction which
gives them a Flow Based Programming (FBP) environment to work in. This implies
box-and-line drag and drop. Refer to noflojs.org for an example FBP
implementation. This would essentially be used to configure NuPIC. One write
FBPs programs to automatically reconfigure NuPIC too, setup the inputs and
outputs. Then deploy the FBP graph to a NuPIC C++ library which sets up and
executes the graph. The host program written in whatever language then
interacts with with NuPIC by simply streaming data in and getting predictions
and anomalies out again (later motor control).
Anyway I'll be pleasantly surprised if this becomes a reality. I firmly believe
distributed heirarchies that span the world, swap and hotswap trained regions
will be exponentially easier to actieve by using the right level of abstraction
for the problem at hand.
Stewart
Doug King <[email protected]> wrote:
>Defining the API is extremely important and should be addressed now.
>Much
>of this conversation is a discussion about APIs directly or indirectly.
>Process (C4 or other) can be address in parallel, e.g. regression test
>are
>about process ensuring correctness of APIs and algorithms.
>
>Designing the API will:
> - force us to think about how code base is used / structured
> - force us to explicitly choose to surface operational parameters that
>configure algorithms
>- provide a contract for development community that encourages
>stability
> - provides a model for a other language implementations
>- provides a contract for public data structures (serialization,
>portable
>trained regions)
> - force us to make choices about where to include 'injected
>functionality' or 'plugins' - allowing the broader community to extend
>functionality without PRs or C4 process (encouraging organic growth of
>ideas).
>
>With a properly designed API we should be able to move forward with the
>current code base, re-factoring chunks instead of doing a complete
>rewrite.
>
>I believe that it is important to be able build distributed networks of
>regions and share trained regions. As we scale CLA systems up, the
>value
>that goes into tuning and training models should be captured and
>commoditized. A model and it's trained dataset should be easy to
>instantiate, run and move across the network. Then we will have a
>strong
>community of users and then we will have an ecosystem. My dream is to
>be
>able to put larger models together based on existing trained models - I
>want to say "hey Stewart, can I get a copy of your facial-emotion
>detector
>region? I'm building a speech recognition system and I need more inputs
>to
>correlate. I've already got Matt's speech-tone detector region".
>
>So starting with APIs is how I see we can make this happen.
>
>-Doug
>
>
>On Fri, Nov 15, 2013 at 5:47 PM, Scott Purdy <[email protected]> wrote:
>
>> Can someone explain which parts of NuPIC they think are "clean" and
>which
>> they want to get rid of? With the exception of the current effort
>around
>> the spatial pooler I don't know which parts are being referred to.
>>
>>
>> On Fri, Nov 15, 2013 at 2:39 PM, Sebastian Hänisch
><[email protected]>wrote:
>>
>>> Some explorative redesigning can be a true benefit to the current
>NuPIC,
>>> too. Some coordinated brainstorming can lead to a better design of
>>> 'NuNuPIC' with an extensive API and a more modular code basis. This
>would
>>> be a base for discussion whether a complete restart is the way to go
>or
>>> maybe it reveals easy ways to restructure the current version
>without
>>> robbing away the base for Grok.
>>>
>>> This can be done aside from NuPIC and wouldn't influence it
>currently
>>> (though some ideas may find themselve implemented there directly).
>>>
>>> As Newcomer to NuPIC I'm just throwing in my 2 cents of course.
>>>
>>> Regards,
>>> Sebastian
>>>
>>>
>>>
>>> 2013/11/15 Fergal Byrne <[email protected]>
>>>
>>>> Hi Matt/Stewart,
>>>>
>>>> I think the radical approach is worth exploring, but keep it within
>>>> NuPIC. You'd need to start with a design, and then build it up by
>robbing
>>>> the clean algorithms from the current NuPIC. Eventually Grok would
>switch
>>>> to the NuNuPIC.
>>>> —
>>>> Sent from Mailbox <https://www.dropbox.com/mailbox> for iPhone
>>>>
>>>>
>>>> On Fri, Nov 15, 2013 at 9:26 PM, Matthew Taylor
><[email protected]>wrote:
>>>>
>>>>> Let's not throw out the baby with the bathwater. Numenta's
>interests
>>>>> lie in improving NuPIC. We won't be able to get behind a
>start-from-scratch
>>>>> effort. We just don't have the manpower, and we're quite busy
>building
>>>>> product on top of NuPIC.
>>>>>
>>>>> That being said, there is nothing stopping anyone else from doing
>this.
>>>>> If someone extracts our core algorithms and pastes them into
>another
>>>>> project, however, the GPL must be respected, and Numenta will not
>be able
>>>>> to provide any commercial licensing options.
>>>>>
>>>>> ---------
>>>>> Matt Taylor
>>>>> OS Community Flag-Bearer
>>>>> Numenta
>>>>>
>>>>>
>>>>> On Fri, Nov 15, 2013 at 12:32 PM, Stewart Mackenzie
><[email protected]
>>>>> > wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Been (carefully) thinking about this more, it's occupying a
>>>>>> disproportionately large amount of time.
>>>>>>
>>>>>> I'm going to go one step further:
>>>>>>
>>>>>> The proof of concept works, NuPIC has cracked it to a large
>degree.
>>>>>> But its based on an old codebase that went through a few
>fundamental
>>>>>> architectural changes. It doesn't support serialization nor can
>operate in
>>>>>> a hierarchy.
>>>>>> Most of the issues are about compilation, incompatibilities, the
>code
>>>>>> base is dependency heavy yet... despite all this the algorthims
>have got it
>>>>>> right.
>>>>>>
>>>>>> I propose we retain the core algorithms and start from scratch.
>Its
>>>>>> the algorithms that are important.
>>>>>>
>>>>>> What I have brewing will make a distributed HTM dramanically
>easier to
>>>>>> achieve, serialization becomes trivial and it is portable. If
>nupic is
>>>>>> about to do some major move. Then lets at least think about how
>to make it
>>>>>> easier to distribute and serialize.
>>>>>>
>>>>>> Insane as it sounds.
>>>>>> Stewart
>>>>>>
>>>>>> Matthew Taylor <[email protected]> wrote:
>>>>>> >Thanks, Stewart.
>>>>>> >
>>>>>> >I think we should put the discussion about releases to the side
>for
>>>>>> >awhile,
>>>>>> >because it's evident that we cannot release until we've nailed
>down
>>>>>> the
>>>>>> >API
>>>>>> >for NuPIC. This should take priority. I'll start that discussion
>soon.
>>>>>> >
>>>>>> >This has been quite enlightening, and I really appreciate
>everyone's
>>>>>> >input.
>>>>>> >
>>>>>>
>>>>>> _______________________________________________
>>>>>> nupic mailing list
>>>>>> [email protected]
>>>>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
>>>>>>
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> nupic mailing list
>>>> [email protected]
>>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
>>>>
>>>>
>>>
>>> _______________________________________________
>>> nupic mailing list
>>> [email protected]
>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
>>>
>>>
>>
>> _______________________________________________
>> nupic mailing list
>> [email protected]
>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
>>
>>
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>nupic mailing list
>[email protected]
>http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
_______________________________________________
nupic mailing list
[email protected]
http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
