Thanks Scott, The wiki for OPF is simply great :)
I agree, we can have README.md for each class (entity) to explain the logic in the repo itself and wiki for some type of integrated (framework) level concepts. Also, this README file could organized with index, so that, in the code we could place that index number in the comments to posses two way reference. On Thu, Aug 1, 2013 at 11:03 PM, Scott Purdy <[email protected]> wrote: > That is possible and very good feedback. In general, I think it is best > not to require developers to look outside the repo to understand what is > going on in the implementation. So making sure we add good README.md files > explaining the contents of directories and writing proper file/module and > class doc strings should go a long way in helping developers understand > what is going on. That said, I do agree that some things make sense in the > wiki, such as the OPF document: > https://github.com/numenta/nupic/wiki/Online-Prediction-Framework > > > On Wed, Jul 31, 2013 at 11:32 PM, Ramesh Ganesan <[email protected]>wrote: > >> Thanks Scott :) >> >> we could understand something from the code and comments. If logic is >> simple and obvious, we can understand that using one or two lines of >> comments. But if we are coding complex logics, then code and one/two lines >> of comments won't help here. In this case, in the code, we can give short >> desc and could provide the [reference doc, index number] on the comment >> section for the detailed explanation. In this way, code is readable with >> less comments and if i want more explanation i will refer the documents >> for detailed explanation. So, whoever writing the code, they could place >> the detailed explanation on the wiki and reference that wiki in the code >> comments with short desc. >> >> >> Regards, >> Ramesh Ganesan >> >> >> >> On Wed, Jul 31, 2013 at 11:22 PM, Scott Purdy <[email protected]> wrote: >> >>> WRT #4, the implementation documentation is the code and comments in the >>> code. We hope to have a section detailing the differences between the CLA >>> white paper and implementation soon but it isn't up yet: >>> >>> https://github.com/numenta/nupic/wiki/Cortical-Learning-Algorithm#differences-between-white-paper-and-implementation >>> >>> >>> On Wed, Jul 31, 2013 at 3:58 AM, Ramesh Ganesan <[email protected]>wrote: >>> >>>> Thanks Subutai & Marek for your interesting explanation, >>>> >>>> 1. It means, i can take some regions (for ex. region1 and regon 5) from >>>> network1 and some regions (for ex. region4 and region6) from network2 and >>>> create new network (network3) with >>>> >>>> some more empty regions +region1 from network1+ region5 from >>>> network1+ region4 from network2+ region6 from network6? >>>> >>>> 2. its interesting. so, we have two types of asyncs >>>> a. clock synchronization between regions (or column level). This >>>> approach useful for openCL based implementation. Actually this is not >>>> really asyncs, at the end everything based on the clock tick >>>> b. run all regions (or columns) asynchronously without any clock >>>> synchronization. Looks like this case is challenging. what will happen if >>>> one particular region continuously runs and other regions not even >>>> scheduled to run, we can miss one or two phases not all the phases. so, >>>> here also, i hope we need some clock ticks based synchronization. Lets >>>> say, clock tick threshold is 10, then all the regions (columns) could be >>>> executed concurrently until 10 phases and after that scheduler needs to >>>> check all non-executed regions (columns) and initiate that to run. After >>>> this, next clock tick (with threshold) continues... >>>> - could some explain, how we can correlate this to brains 100 >>>> step processing, hierarchical, thalamus controlled etc.,? >>>> >>>> >>>> 4. Do we have any docs for current algorithm's implementation? I hope, >>>> we have some difference between whitepaper and implementation. >>>> >>>> >>>> >>>> >>>> On Tue, Jul 30, 2013 at 11:46 PM, Subutai Ahmad <[email protected]>wrote: >>>> >>>>> Hi Ramesh, >>>>> >>>>> Thanks for putting this on the wiki - that is great! >>>>> >>>>> To add to Marek's answers: >>>>> >>>>> 1) Yes, it can handle that. We used to do this quite a lot too. >>>>> >>>>> 2) In theory there is nothing in the CLA that prevents everything >>>>> running asynchronously. We haven't really tested that - it would be cool >>>>> if >>>>> someone wanted to test that. From a NuPIC standpoint, the client code can >>>>> order the calls to compute's any way they want. Today, NuPIC2 does not >>>>> automatically handle threading, or distributing the computation across >>>>> cores. The older NuPIC used to do that but we did not see much benefit >>>>> from >>>>> it. >>>>> >>>>> 3) I think that's right. >>>>> >>>>> 4) Column level stuff is handled within the algorithm / Region >>>>> implementations. NuPIC doesn't impose that. From a speed standpoint it's >>>>> much better to deal with it inside the algorithm classes. Our temporal >>>>> pooler for example, uses all sorts of across-column sparse structures to >>>>> get large speedups. >>>>> >>>>> Cheers, >>>>> >>>>> --Subutai >>>>> >>>>> >>>>> >>>>> On Tue, Jul 30, 2013 at 6:30 AM, Marek Otahal <[email protected]>wrote: >>>>> >>>>>> Thanks Ramesh for wikifying this and very good questions! >>>>>> >>>>>> >>>>>> On Tue, Jul 30, 2013 at 2:31 PM, Ramesh Ganesan >>>>>> <[email protected]>wrote: >>>>>> >>>>>>> I have copied the content into wiki[yet to insert images]. Some >>>>>>> questions raised in my brain during my copy process >>>>>>> >>>>>>> 1. we have network level export (save/serialization) process. Also, >>>>>>> we have network level de-serialization process. If i want to construct >>>>>>> new >>>>>>> network with some of those previously trained network regions, will it >>>>>>> support? >>>>>>> >>>>>> +1 on this. If I spent week(s) training a network, the persistance is >>>>>> essential >>>>>> ...ok, and as I read further, Network serialization handles this. >>>>>> >>>>>>> >>>>>>> 2. we are running the networks with phases. And these phases are >>>>>>> executed sequentially. we couldnot run the phases asynchronously? For >>>>>>> ex. >>>>>>> if i have region1 and its executed in phase1 and region2 executed in >>>>>>> phase2 >>>>>>> and if i combine these results in region3 which is executed in phase2, >>>>>>> what >>>>>>> will happen? will it result in unexpected behavior? In our brain, >>>>>>> everything executed in asynchronously or do we have any synchronous >>>>>>> activity? >>>>>>> >>>>>> Yes, neurons in our brains fire asynchronously, brains are highly >>>>>> parallel devices (unlike nupic now - can we have multithreading support >>>>>> in >>>>>> nupic, say pool of threads per -region? And openCL later?), and robust to >>>>>> noice (same as nupic). >>>>>> >>>>>> For single threaded (hw-wise) use this is not relevant. For >>>>>> multithreading - clock-synchronization and input/output persistence need >>>>>> to >>>>>> be taken into account: >>>>>> >>>>>> A) sync to clock: eg 2 threads, you define first are executed reg1,2, >>>>>> they write out their output and reg3 can execute. >>>>>> >>>>>> B) async mode: each region is executed on (available) threads, writes >>>>>> out a value when finished, this val is kept (and read by others) until >>>>>> overwritten by the next round of this thread. >>>>>> >>>>>> So, for purposes of distributed computing, I believe regions should >>>>>> be executed in a parallel manner. >>>>>> >>>>>> >>>>>> Other implementation thing is OpenCL, where even input (huge index >>>>>> array) of a single region is executed on all available CPU/GPU cores. >>>>>> >>>>>> >>>>>>> >>>>>>> 3. In wiki, the last section >>>>>>> >>>>>>> "Currently a Region is logically composed of one or more Nodes. The >>>>>>> Node class may be removed as it does not appear to be necessary with the >>>>>>> new algorithms. If Node is removed, Dimensions and NodeSet will also be >>>>>>> removed." >>>>>>> >>>>>>> It means, If we remove node class, dimensions will be achieved >>>>>>> through linkPolicy? >>>>>>> >>>>>>> 4. In the doc, i couldn't find column level architecture >>>>>>> information. do we have any docs for this? >>>>>>> >>>>>>> >>>>>>> Thanks >>>>>>> Ramesh Ganesan >>>>>>> >>>>>>> Cheers, Mark >>>>>> >>>>>> -- >>>>>> Marek Otahal :o) >>>>>> >>>>>> _______________________________________________ >>>>>> nupic mailing list >>>>>> [email protected] >>>>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>>>> >>>>>> >>>>> >>>>> _______________________________________________ >>>>> nupic mailing list >>>>> [email protected] >>>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> nupic mailing list >>>> [email protected] >>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>>> >>>> >>> >>> _______________________________________________ >>> nupic mailing list >>> [email protected] >>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >>> >>> >> >> _______________________________________________ >> nupic mailing list >> [email protected] >> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org >> >> > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
