Hi Oreste san,
Thank you for your comment.
Oh, I don't have a solid idea. I'm sorry for blurry information.
My concern was, the improvement might bring too less workload to utilize
GPU during inhibition.
The loop is like {
find_max(preactive columns);
give_penalty(a few neighbor columns);
synchronize;
} repeat.
find_max can be performed by 1000 threads if we have 1000 columns to
process for the better throughput. give_penalty + synchronize force most
of GPU to stay idle for some time. So, the average workload might become
low.
But I have another thought. If one GPU can accommodate many regions linked
together, we can hopefully utilize GPU all the time. NVidia Kepler
provides very such feature called Hyper-Q, you probably know.
I hope the logic can be made more asynchronous or the load is more evenly
distributed for GPU (the current logic is much better in that perspective),
but it may not be significant anyway if we think of the total amount of
computation. I'm not really sure.
Best Regards,
Hideaki Suzuki.
2013/9/10 Oreste Villa <[email protected]>
> Hi Hideaki,
>
> Thanks for sharing, I want to confirm that this is very useful even
> for people not working on the code (as me).
> Could you please articulate, if possible, on the "more GPU friendly" in
> the last slide?
> Thanks,
>
> Oreste
>
>
> On Mon, Sep 9, 2013 at 4:41 AM, Fergal Byrne
> <[email protected]>wrote:
>
>> Hi Hideaki,
>>
>> Thanks for sharing that with us, it has the advantage that it is a very
>> significant optimisation of the SP, with the real bonus of having a strong
>> neurological basis. Local inhibition is carried out in this fashion in the
>> neocortex.
>>
>> I have a couple of questions/comments.
>>
>> In your presentation, you're showing comparisons with the global
>> inhibition strategy, rather than the old local strategy. Your example
>> application is a vision system. Does your connection mapping (pixels to
>> columns) embody strong topology? In other words, do the neighbouring
>> columns of a candidate column overlap its retinal location?
>>
>> The global inhibition strategy does not perform (semantically) as well as
>> the old local one when the connection mapping (and the data) is topological
>> and the task is truly spatial patterns. NuPIC uses global and
>> non-topological mappings currently because it's optimised for scalar and
>> category data, for which the global inhibition is a (non-biological)
>> optimisation. Global works better because neighbours in the region are
>> meaningless semantically, and the input-column connections are randomly
>> assigned.
>>
>> The old local inhibition strategy is really a divide-and-conquer strategy
>> which replaces the global region with a number of sub-regions and performs
>> the global algorithm on each in parallel. Your method seems to match more
>> closely the natural algorithm, which must always be utterly local at all
>> times (ie there is no control program in the neocortex, each column reacts
>> autonomously to its local environment).
>>
>> In order to incorporate your innovation into NuPIC, we'll need to have a
>> development activity aimed at resurrecting the vision (or other spatial)
>> architecture which has been moribund for some time. The SP would have to
>> have a switch or mode allowing a topological bias for columns and pixels
>> (or other spatial "locations"), and the "local inhibition" strategy would
>> also be switchable.
>>
>> I'm pretty sure that any real vision or spatial applications would, in
>> addition, require hierarchy to be added to NuPIC in order to be useful.
>> Otherwise we would just have another perceptron, albeit with added
>> complexity and a better biological justification. The now very old demo
>> vision program shows that even a simple hierarchy can achieve apparently
>> impressive results, but again this did not use the real CLA (as we
>> currently know it) and it did not operate in the manner we believe the
>> neocortex works.
>>
>> Regards,
>>
>> Fergal Byrne
>>
>>
>>
>>
>> On Sun, Sep 8, 2013 at 11:35 PM, Marek Otahal <[email protected]>wrote:
>>
>>> Thanks a lot for sharing Hideaki!
>>>
>>> on a quick glance, those results are impressive!
>>> How did you do the models (matlab or something?), will you be planning
>>> to implement the feature to NuPIC code?
>>> Either way, please fill in an issue request at JIRA with the slides
>>> included.
>>>
>>> Thanks a lot for your work, Mark
>>>
>>>
>>> On Mon, Sep 9, 2013 at 12:24 AM, Hideaki Suzuki <[email protected]>wrote:
>>>
>>>> Hi,
>>>>
>>>> I wrote a ppt to summarize one topic I did in my study of SP.
>>>> I have a few other topics with SP, but I'm shifting to TP.
>>>>
>>>> The idea speeds up the local inhibition x20 or better without
>>>> parallelism (6ms -> 0.3ms in my PC). As learning in SP proceeds, it will
>>>> even get close to the speed of global inhibition.
>>>>
>>>>
>>>> I'm sending this just because I don't want to be a free rider.
>>>> Whatever I feel it might be useful for someone, I like to share it. So,
>>>> please don't get offended because this is not so much biological (but
>>>> software engineering stuff).
>>>>
>>>> I hope this kind of mail is okay in this mailing list...
>>>>
>>>> Best Regards,
>>>> Hideaki Suzuki.
>>>>
>>>> _______________________________________________
>>>> nupic mailing list
>>>> [email protected]
>>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
>>>>
>>>>
>>>
>>>
>>> --
>>> Marek Otahal :o)
>>>
>>> _______________________________________________
>>> nupic mailing list
>>> [email protected]
>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
>>>
>>>
>>
>>
>> --
>>
>> Fergal Byrne
>>
>> ExamSupport/StudyHub [email protected]
>> http://www.examsupport.ie
>> Dublin in Bits [email protected] http://www.inbits.com +353 83
>> 4214179
>> Formerly of Adnet [email protected] http://www.adnet.ie
>>
>> _______________________________________________
>> nupic mailing list
>> [email protected]
>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
>>
>>
>
> _______________________________________________
> nupic mailing list
> [email protected]
> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
>
>
_______________________________________________
nupic mailing list
[email protected]
http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
