I agree with Chetan that they are both useful in different scenarios. Circuit diagrams are great for explaining how our neurons work but trying to describe e.g. a new sequence memory algorithm would be much easier in pseudocode than a circuit diagram IMHO.
On Thu, Jan 16, 2014 at 10:46 AM, Chetan Surpur <[email protected]> wrote: > Francisco, > > I agree that the CLA would very naturally be represented as a circuit, and > it would be useful to do so. On the other hand, an algorithmic description > would offer an intuitive understanding that gets at the heart of the nature > of the process more quickly and explicitly. I don't think the two are > mutually exclusive though, and suggest that we target both representations > when trying to describe the CLA. > > Thanks, > Chetan > On Jan 16, 2014 9:26 AM, "Francisco Webber" <[email protected]> wrote: > >> hi Jeff, >> >> On 16.01.2014, at 06:44, Jeff Hawkins <[email protected]> wrote: >> >> > Thank you for all these thoughts, I am still digesting them. >> > >> > Part of what is motivating me to tackle documentation this year is that >> I might be closing in on a broader understanding of what all the layers in >> the cortex are doing. The CLA is just part of that. I need to figure out >> the best way(s) to communicate these new ideas. >> >> This is great news (actually I was hoping and waiting for this message >> ;-). But this makes the issue even more urgent. >> >> > >> > One question is what kind of language to use. In the white paper we >> used prose, pseudo-code, and neuroscience. The pseudo-code was intended to >> be a clear and somewhat formal description, immune to the messiness of the >> actual code. Yes, you need to have some programming skills but not much. >> Do you think that pseudo-code isn't sufficient as a formal description? >> >> I would not say that pseudo-code is not formal but it might not be >> appropriate. As if we would like to draw a oil painting with chalk. >> I am not sure if the CLA could really be seen as algorithm, to me it >> consists of components with inputs and outputs, some wiring and signals >> that run across them. >> By trying to model this with software, we do two things at the same time: >> we build an analog circuit simulator, and we build a parametrized (and >> serialized) schematic within. >> It took several decades of academic and industrial efforts to come up >> with reliable simulators. Why not use this achievement and focus on the >> components and the schematic of the CLA? >> Another advantage would be the fact that circuits in these simulators can >> be operated and probed virtually. This would allow to actually test the >> behavior of the setup. >> But don’t get me wrong, I am not claiming to build a “real” virtual brain >> using PSpice. Rather would I do the abstraction-step at the level of the >> components. One component could be for example a synapse with two switch >> inputs and a control-voltage input. Regardless how complicated a real >> synapse would be. Another component could be a “CLA-cell” also independent >> of how many neurons might be behind this concept in the real brain. >> When the circuit is captured as schematic, it is much easier to >> collectively discuss it by pointing at it. There are also program >> generators that can convert such a circuit model into software. But the >> circuit could probably also be rendered as hardware …. at some point. >> And after all, a schematic of a CLA circuit would be an unambiguous model >> for deriving generic software versions of the CLA which could be directly >> compared to the signal response of the hardware-model. >> > >> > The other question is where to publish. We could just write a new >> white paper, the current one is getting old anyway. The nice thing about a >> white paper is it can be as comprehensive as needed. But I am also feeling >> some pressure to publish in a peer reviewed journal. Some people don't >> take you seriously without peer review. As you suggest we could do a white >> paper and then a series of smaller papers. >> >> In order to make the step towards a peer reviewed publication, it will be >> a prerequisite to also publish some data with it, that has been evaluated >> against a known (scientific) standard outcome, and that (if you target a >> high impact journal) the provided data should be reproducible by >> independent peers. >> But this is only partially the reason why I am advocating to also >> consider the development of a reference-evaluation framework, for the >> continuous collective improvement of the CLAs. >> >> all the best from Hong Kong >> >> Francisco >> >> > >> > -----Original Message----- >> > From: nupic [mailto:[email protected]] On Behalf Of >> Stewart Mackenzie >> > Sent: Tuesday, January 14, 2014 8:58 PM >> > To: NuPIC general mailing list. >> > Subject: Re: [nupic-discuss] Hong Kong meeting outcome >> > >> > please read responses inline: >> > >> > Francisco Webber <[email protected]> wrote: >> >> Jeff, Steward, >> >> I think what is actually missing is a formal description of the >> >> algorithm that allows non software programmers to understand and >> >> analyze it in full detail. Fergal and I were even thinking of an >> >> electronic wiring schematic to try and capture the full detail of CLA >> >> in a concrete formal way. I fear that without a central formal model >> >> the theoretical research, the practical implementation and the >> >> documentation will slowly drift apart without anyone noticing. >> >> Having a software-development-independent formal description of the >> >> algorithm will also help to avoid the mix-up of code-parts that are >> >> there because the CLA algorithm says so, parts that are there because >> >> no better engineering solution has been found yet for a specific system >> >> behavior and even parts that might be there because of the specific >> >> expressiveness of a programming language. >> > >> > We're designing and developing a flow-based programming environment on >> Mozart Oz. >> > >> > -To get to the point quickly: this tool allow us to expose a >> 'box-and-line' user interface to the non-programmer that can be programmed >> by dragging a finger on the screen. One box might be a neuron, another a >> dendrite another a synapse and one can 'drill down' and open up components >> within components (ie composite components). This allows high level display >> of logic without getting bogged down in implementation code. >> > - the programming language (oz) used to program components has some 30 >> odd different language concepts and is _more_ that expressive enough for >> this challenge. >> > >> > The example below is in 'flow-based programming' for the AND logic >> gate, it is executable: >> > >> > a => a nand(nand) out -> in not(not) out => out b => b nand() >> > >> > The above represents the application's logic graph. >> > the Nand is a component and is implemented in Oz. >> > the Not gate on the other hand looks like this on the inside: >> > >> > The not gate is a composite-component. >> > NOT gate: >> > >> > in => a nand(nand) out => out >> > in => b nand() >> > >> > In the above example the 'in' on the left of the => is the input >> interface, the out to the right of the => is the output interface. The >> stuff between the => and => is the logic of the graph. nand(nand) equates >> to 'name_of_component(type_of_component)'. The second 'nand()' does not >> need a type as it has already been instantiated. >> > Again the nand gate is implemented in Oz. >> > >> > Now the above representation is very easy to create a high level >> application logic specific view that is _executable!_ most importantly, the >> components are being designed so that we may create a GUI: >> > >> > --------- -------- >> > a -----> a| | | | >> > | nand | out -------->in | not | out ---------> out >> > b -----> b| | | | >> > --------- -------- >> > >> > We are currently tying this together with an HTML implementation. So >> it'll be interactable with a finger. >> > >> > So writing the paper, then implementing it in this interface one >> creates an executable specification that non-programmers can easily change. >> This serves as a perfect highly flexible example for implementers to >> reference for more optimal implementations in say c++ (ie nupic-core). >> > >> > Francisco is this what you imagined? >> > >> > While you're still in town you should come over for dinner so I can >> demo this to you ;-) >> > >> >> - The “language” of the formal model should allow people from different >> >> (maybe even VERY different) areas to gain full insight into the >> >> fundamental algorithm. >> >> - The formal model should also allow to think, discuss and communicate >> >> concepts at different scales of resolution (from the birds-eye >> >> perspective down to the synapse level) >> >> >> >> I am well aware that this is not an easy thing to do, but I think it is >> >> indispensable for the development of any theoretic research field to >> >> have its own formal notation framework. >> >> >> >> Francisco >> >> >> >> On 15.01.2014, at 07:33, Stewart Mackenzie <[email protected]> wrote: >> >> >> >>> Very good point Jeff, I suspect this material is ready for peer >> >> review (in the idealistic sense of the phrase peer review). I'll admit >> >> I've lost faith in the peer review process of today. For many reasons, >> >> too many to go into at the moment. Though despite my tangible dislike >> >> for journals which needs to be disrupted off the face of the earth, >> >> publishing with them most likely brings more academics onboard. >> >>> >> >>> The main driver for this paper is to get everyone on the same page. >> >> I'd prefer seeing a comprehensive (white) paper, with a series of >> >> smaller papers focusing on smaller areas being published in the >> >> journals. All the detail is in the comprehensive. The smaller papers >> >> are to move academia along using a language and publication process >> >> they are familiar with. I suspect new names should be made to describe >> >> newly discovered phenomenon, with as emphasis on describing the right >> >> 'altitude' one needs to approach this problem. This conditions the >> >> academics to start adjusting the mindset to a certain level. Eventually >> >> causing a resonance which hopefully can be conducted into and through >> >> NuPIC. So we as a community had better be sure NuPIC is in shape such >> >> that the resonance won't shatter it. >> >>> >> >>> Jeff Hawkins <[email protected]> wrote: >> >>>> When you say "canonical paper" are you thinking a peer reviewed >> >> paper >> >>>> or an >> >>>> updated white paper? Is it more important to be comprehensive >> >> (white >> >>>> paper) >> >>>> or published in peer reviewed journals? Or are you thinking >> >> something >> >>>> else? >> >>>> Jeff >> >>> >> >>> Kind regards >> >>> Stewart >> >>> >> >>> -- >> >>> Please excuse my typos and brevity >> >>> >> >>> _______________________________________________ >> >>> nupic mailing list >> >>> [email protected] >> >>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> >> >> >> >> >> _______________________________________________ >> >> nupic mailing list >> >> [email protected] >> >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> > >> > Kind regards >> > Stewart >> > >> > -- >> > Please excuse my typos and brevity >> > >> > _______________________________________________ >> > nupic mailing list >> > [email protected] >> > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> > >> > >> > _______________________________________________ >> > nupic mailing list >> > [email protected] >> > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> >> >> _______________________________________________ >> nupic mailing list >> [email protected] >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
