William Pearson wrote:
On 13/04/07, Richard Loosemore <[EMAIL PROTECTED]> wrote:
To convey this subtlety as simply as I can, I would suggest that you ask
yourself how much intelligence is being assumed in the preprocessing
system that does the work of (a) picking out patterns to be considered
by the system, and (b) picking the particular patterns that are to be
rewarded, according to some "success" criterion. Here is the problem:
if you are not careful you will assume MORE intelligence in the
preprocessor than you were hoping to get the core of the system to
learn. There are other issues, but that is one of the main ones.
For the record I agree with this critique of some of the neuroscience
views of reinforcement learning in the brain.
What I find tremendously frustrating is the fact that people are still
so dismally unaware of these issues that they come out with statement
such as the one in the quote: speaking as if the idea of reward
assigment was a fantastic idea, and as if the neuroscience discovery of
a possible mechanism really meant anything. The neuroscience discovery
was bound to collapse: I said that much of it the first time I heard of
it, and I am glad that it has now happened so quickly. The depressing
part is that the folks who showed it to be wrong think that they can
still tinker with the mechanism and salvage something out of it.
It think they do this because they haven't found a better hypothesis
and have too much invested in the previous status quo. I'd be curious
to know if your hypothesis for a motivation system has the potential
for the same simple signal given to systems, with different histories,
to cause the system to attempt to get the same signal again (addiction
being the pure example of this). This is one of the important
phenomenon I require a motivational system to explain.
This is an interesting question.
Addiction is clearly a pathology of the human motivational system, but
the explanation for it could lie at a number of levels. Maybe the
system is designed in such a way that a high-level imbalance occurs in
some cases ... and then again, in other cases, it might be caused by a
low-level problem.
Example of high level. Suppose the system is designed to work by means
of a checks and balances mechanism, where too much of a given type of
desirable activity starts to cause the eward for that activity to
decrease, making the system susceptible to new ideas for what it would
like to do. The default settings for these habituation effects would
veary between individuals, but overall we might expect that some would
be highly tolerated because whenever a normal environment provided that
source of activity, the environment always stops supplying the activity
before the person can get enough of it. Example: playing computer
games. When the natural environment supplies a fascinating game
situation, there are great benefits to getting good at it, so the system
is wired to let the human try to get as much as it wants of the game
playing, but only if the game is very complex and engaging. Such games
that involve a quick series of challenges by the environment (usually
another human being) are so rare that the human system basically says
"get as much of this as you possibly can, because it is rare, and you
can never get enough of this". But when the system says "no limits" to
this activity, it does not take account of the invention of computer
games which can go on indefinitely. Thus there is the possibility of
addiction because the design of the system wrongly assumed the
environment would never allow the human to get an infinite amount of
this activity.
Example of the low level. Chemical addiction can cause one part of the
brain to generate vast quantities of (a) novel stimuli (e.g.
hallucinogenics) or (b) satisfaction signals attached to an arbitrary
activity (e.g. drugs that cause people to feel exceptionally good for no
reason whatsoever). In both these cases the wiring of the system has
been subverted, causing a situation where the normal process starts to
require the drug because the satisfaction caused by the drug is greater
than any other activity.
Clearly, these are very simple types of explanation. Quite mundane,
really. The problem with explaining any given addiction is that we
cannot yet be sure at what level it is happening, so good explanations
will have to wait until we can do more advanced work.
Stepping back a bit from your question: note that my main interest is
in building artiifical motivation systems inspired by the human system,
so the goal of precisely explaining breakdowns of the human system is
not directly in my sights. Subtle difference, but a big one.
Richard Loosemore.
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?member_id=231415&user_secret=fabd7936
