Hi Fergal san,
It is really nice to hear that the idea is not going away from neocortex.
That was my major worries.
For your query, I hope my setup follows the white paper precisely, although
my wording might not be so precise.
The test program uses a fixed 256x256 pixel small image patch, which is
connected to the 32x32 column region. I expressed a portion of the input
image that one column connects to as "fanout area". So, on page 11 for
example, I mean by "fanout radius 38" that one column is linked to 77x77
pixel square area (2*38+1). This area contains 5929 pixels. Among them,
1024 pixels are randomly chosen and connected to a column via potential
synapses. Among them, 50% of potential synapses are above connected Perm
threshold with bias toward the natural center. On the other hand, because
32x32 columns are mapped to 256x256 pixels, the centers of fanout areas of
columns are evenly distributed and are apart by 8 pixels all the time.
For the global inhibition strategy, I totally agree with you that local
inhibition reduces its behavior to that of global inhibition for
topology-less data (the bottom case of page 12) and that is the rationale
for Nupic to set global inhibition as the default choice. Local inhibition
is only a sluggish way to achieve the same thing for such kind of data.
Vision is a vast research area and resurrecting it may be a little too big
a topic for now. Having a switch or a mode sounds like a fascinating idea
to me. If we can swam over to find the best topology and hierarchy by
somehow in some day, it would become very fascinating solution for users?
;-)
Best Regards,
Hideaki Suzuki.
2013/9/9 Fergal Byrne <[email protected]>
> Hi Hideaki,
>
> Thanks for sharing that with us, it has the advantage that it is a very
> significant optimisation of the SP, with the real bonus of having a strong
> neurological basis. Local inhibition is carried out in this fashion in the
> neocortex.
>
> I have a couple of questions/comments.
>
> In your presentation, you're showing comparisons with the global
> inhibition strategy, rather than the old local strategy. Your example
> application is a vision system. Does your connection mapping (pixels to
> columns) embody strong topology? In other words, do the neighbouring
> columns of a candidate column overlap its retinal location?
>
> The global inhibition strategy does not perform (semantically) as well as
> the old local one when the connection mapping (and the data) is topological
> and the task is truly spatial patterns. NuPIC uses global and
> non-topological mappings currently because it's optimised for scalar and
> category data, for which the global inhibition is a (non-biological)
> optimisation. Global works better because neighbours in the region are
> meaningless semantically, and the input-column connections are randomly
> assigned.
>
> The old local inhibition strategy is really a divide-and-conquer strategy
> which replaces the global region with a number of sub-regions and performs
> the global algorithm on each in parallel. Your method seems to match more
> closely the natural algorithm, which must always be utterly local at all
> times (ie there is no control program in the neocortex, each column reacts
> autonomously to its local environment).
>
> In order to incorporate your innovation into NuPIC, we'll need to have a
> development activity aimed at resurrecting the vision (or other spatial)
> architecture which has been moribund for some time. The SP would have to
> have a switch or mode allowing a topological bias for columns and pixels
> (or other spatial "locations"), and the "local inhibition" strategy would
> also be switchable.
>
> I'm pretty sure that any real vision or spatial applications would, in
> addition, require hierarchy to be added to NuPIC in order to be useful.
> Otherwise we would just have another perceptron, albeit with added
> complexity and a better biological justification. The now very old demo
> vision program shows that even a simple hierarchy can achieve apparently
> impressive results, but again this did not use the real CLA (as we
> currently know it) and it did not operate in the manner we believe the
> neocortex works.
>
> Regards,
>
> Fergal Byrne
>
>
>
>
> On Sun, Sep 8, 2013 at 11:35 PM, Marek Otahal <[email protected]>wrote:
>
>> Thanks a lot for sharing Hideaki!
>>
>> on a quick glance, those results are impressive!
>> How did you do the models (matlab or something?), will you be planning to
>> implement the feature to NuPIC code?
>> Either way, please fill in an issue request at JIRA with the slides
>> included.
>>
>> Thanks a lot for your work, Mark
>>
>>
>> On Mon, Sep 9, 2013 at 12:24 AM, Hideaki Suzuki <[email protected]>wrote:
>>
>>> Hi,
>>>
>>> I wrote a ppt to summarize one topic I did in my study of SP.
>>> I have a few other topics with SP, but I'm shifting to TP.
>>>
>>> The idea speeds up the local inhibition x20 or better without
>>> parallelism (6ms -> 0.3ms in my PC). As learning in SP proceeds, it will
>>> even get close to the speed of global inhibition.
>>>
>>>
>>> I'm sending this just because I don't want to be a free rider. Whatever
>>> I feel it might be useful for someone, I like to share it. So, please
>>> don't get offended because this is not so much biological (but software
>>> engineering stuff).
>>>
>>> I hope this kind of mail is okay in this mailing list...
>>>
>>> Best Regards,
>>> Hideaki Suzuki.
>>>
>>> _______________________________________________
>>> nupic mailing list
>>> [email protected]
>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
>>>
>>>
>>
>>
>> --
>> Marek Otahal :o)
>>
>> _______________________________________________
>> nupic mailing list
>> [email protected]
>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
>>
>>
>
>
> --
>
> Fergal Byrne
>
> ExamSupport/StudyHub [email protected] http://www.examsupport.ie
> Dublin in Bits [email protected] http://www.inbits.com +353 83
> 4214179
> Formerly of Adnet [email protected] http://www.adnet.ie
>
> _______________________________________________
> nupic mailing list
> [email protected]
> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
>
>
_______________________________________________
nupic mailing list
[email protected]
http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
