Hideaki your PDF is a good starting point for implementing a GPU accelerated spatial pooler.
I have pointed to this before - some work has been done assessing the CLA (based on CLA whitepaper 1.0) and opportunities to parallelize it with GPU. http://pdxscholar.library.pdx.edu/open_access_etds/202/ RE: Cuda 6, Latency is still an issue. Cuda 6 makes memory management simpler. But memory copies between the system and GPU memory still need to be done and the latencies remain. On the upside CUDA 6 handles the programming of data block transfers for developers transparently. In contrast, AMD's HSA architecture implements truly unified memory access and reduces latency. I am sure Nvidia will be close behind and next iteration will have a similar unified architecture that reduces latency. Moving memory form CPU to GPU and back kills a lot of the performance gains when it comes to the CLA. Hideaki, you are correct, the synapse traversals in the CLA make it a poor fit for current GPU architecture. You have to constantly reach out to other memory blocks to get the data you need to perform updates. Spatial pooler is a better fit for current generation of GPU's and we could start with spatial pooler GPU acceleration and gain some benefits right now. On Wed, May 7, 2014 at 1:10 PM, Hideaki Suzuki <[email protected]> wrote: > Hi Sergey, > > Thank you for the link. This is really a cool idea. The recent high end > GPU has a few GB of memory on the device even for gaming like GTX 780 Ti. > It looks like the main memory would be L3 or L4 cache for the memory on > the GPU device and CPU is like a co-processor for some kind of > computational tasks. ;-) > > > Unfortunately, I'm not aware of a progress about Nupic on GPU. I guess > the difficulty comes from 1) the parallel link (=synapse) traversals in CLA > and 2) a solid C code base that can be ported to GPU kernel. > > Once nupic.core becomes ready, it may be a good time to try this porting > out. In addition, CUDA 6 has been released recently which will make > programmer's lives easier. So the situation is better than ever. I > believe... [*] > > A few months ago, I draw a sketch on a piece of paper about how to run SP > on GPU. I picked it up and transferred my dirty hand writing to a pdf > file, in hope that you may find it interesting... PFA. > (note: my sketch is not about a porting design but about a GPU-oriented > design unrelated to Nupic) > > [*] https://developer.nvidia.com/cuda-toolkit > > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
