Thanks Marek for another great question.
In the current implementation, boosting is necessary at least in part because
the SP is ignorant of the TP's predictions. Thus, the same 40 columns will
always win, and they'll strengthen their monopoly (or quarentolopy!) each time
they win.
This is contrary to both the reality in the cortex and Jeff's theory. The cells
which already are partially depolarised will beat those which aren't, and
(assuming similar feed forward responses) the 40 columns should include many
predictive ones. Thus the SP's SDRs will vary depending on the sequence you're
in, and you may not need boosting to counteract the entrenching quarentopolies.
Regards,
Fergal Byrne
—
Sent from Mailbox for iPhone
On Fri, Dec 13, 2013 at 10:38 PM, Michael Ferrier
<[email protected]> wrote:
> Hi Marek,
> Good question... This paper talks about the biological basis of BCM's
> floating threshold, on pp. 1515-1516:
> http://www.izhikevich.org/publications/bcm.pdf
> Apparently it's been shown that the long term level of postsynaptic
> activity changes the ratio between two different types of subunits of NMDA
> receptors, which has the effect of sliding the threshold separating LTP and
> LTD.
> -Mike
> _____________
> Michael Ferrier
> Department of Cognitive, Linguistic and Psychological Sciences, Brown
> University
> [email protected]
> On Fri, Dec 13, 2013 at 6:44 AM, Marek Otahal <[email protected]> wrote:
>> Thank you all for the ideas! It's a really interesting discussion.
>>
>> Ian and Dennis, am I correct to summarise your points that it's not
>> possible to explain boosting at the {neuron,synapses} level (I don't agree
>> on 100%, as per Mike's msg) but it's caused by things at a lower level
>> {chemicals, hormones} (agree on that one, good point!) ..?
>>
>> Some more questions in follow the text please..
>>
>> On Thu, Dec 12, 2013 at 5:53 PM, Ian Danforth
>> <[email protected]>wrote:
>>
>>> All,
>>>
>>> Great discussion. I want to frame it a bit and say that there are *many*
>>> reasons that a neuron, having been inactive for some period of time would
>>> fire, or fire more readily. This is a complex business. For example
>>>
>>> The various 'opsins (rhodopsin, photopsin) are continually being produced
>>> and consumed in the rods and cones of the retina. If the proteins are not
>>> consumed they build up, allowing the cells to become more sensitive to
>>> smaller amounts of light. If you were just looking at the activity of a
>>> retinal ganglion cell, you might think that its increased activity over
>>> time in a dark room was due to some intrinsic property, but it would be (at
>>> least in part) a function of input adaptation.
>>>
>>> Further, as Dennis mentioned the vesicles that contain and
>>> bind-to-release neurotransmitters are nearly-continually being manufactured
>>> and so a neuron that hasn't been active in a while could produce a much
>>> stronger response in a cell onto which it synapsed by contributing all that
>>> "pent up" glutamate.
>>>
>>> Finally resting potential of a neurons membrane is governed by an
>>> extremely complex interplay of the movement and balance of ions across the
>>> membrane. One type of movement, the 'leak' currents account for the basic
>>> resting potential by letting K+ ions out of the cell, but other leak
>>> currents can slowly depolarize cells over time
>>>
>>
>> Would this leak be the cause of what we call a decay in the algorithm? As
>> with time, unused synapses' weights decrease.
>>
>>
>>> (see http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3247702/) this can be
>>> useful for cells that have a minimum firing rate or help in regulation of
>>> rhythmic activity like your heartbeat. Moreover, environmental factors like
>>> temperature, and the concentrations of hormones in the larger system all
>>> directly impact the resting potential of cells making them more or less
>>> likely to fire at a given moment.
>>>
>>> So, is there one biologically supported reason we might want to 'boost'
>>> inactive cells? No. And because we don't have a 1 to 1 comparison of the
>>> time scale for CLA to the real world, it is very difficult to guess *which*
>>> of the many processes are most closely related to boosting. In biology it
>>> matters a great deal if 5 minutes pass between inputs as opposed to 5 ms,
>>> but it's all the same to CLA.
>>>
>>
>> This is an important thought imho. It should belong to a 'Timing in NuPIC'
>> discussion that pops out every now and then, but I'll just write here. At
>> the CLA region level, we don't have a notion of time. This is good in a
>> sense, as neither do neurons, but bad in other.
>>
>> This should be handeled at a HTM level, where senses are taken into
>> account. There are 2 ways to express time in Nupic, implicit: ignoring the
>> time and the TP handles the sequential order of items.
>> And explicit, in OPF, where the 'time field' is added as another value to
>> the input.
>>
>> Now, neither of the two concepts covers the idea you mentioned (you meant
>> it slightly different, but anyway..):
>> If I present a sensor a cat, and leave it there for an hour, and then
>> change it for a dog, just the change is noticed and the two values {dog,
>> cat} are same probable. In reality, dog would be a strong anomaly. A
>> solution would be: add a required field 'frequency/timing' to the sensors,
>> which would then resend the values in input to the system with the given
>> frequency.
>>
>>
>>>
>>> The more carefully we define the problems we're trying to solve, and the
>>> more closely we can relate them to biology, the smoother this comparison
>>> will be. Until then though, the truth of the matter is "we boost because it
>>> seems to help in many cases."
>>>
>>> Ian
>>>
>>>
>> Thanks all for the replies!
>> Cheers, Mark
>>
>> --
>> Marek Otahal :o)
>>
>> _______________________________________________
>> nupic mailing list
>> [email protected]
>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
>>
>>_______________________________________________
nupic mailing list
[email protected]
http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
