Mike,
Thanks for the thoughtful and detailed reply.
I'm going to have to be brief because of time pressure, alas.
My overall feeling is that there is an important nugget of truth beneath
the surface of what you say, but I think that when you develop the idea
further, you might find yourself drifting away from the strict use of
the "picture" idea (which seems very important to your position right
now, but I think you might end up lessening its importance later), and
move the concept towards a "representations" or "mental models" idea
instead.
The reason I say this is that you need a very abstract notion of
"picture" to get some aspects of the idea to work. For example, when
you say:
> How does the brain know that:
>
> "The man climbed the penny "
>
> is nonsense? Because it will literally draw a picture in graphics of a
> man trying to climb a penny - and from the graphics deduce that
> that is impossible.
... can you really say that it is the *graphic* that determines the
impossibility? Is it the iumpossibiility of the attempt to draw that
graphic, that leads to the "impossible" verdict? Now that doesn't seem
quite right, because I can draw at least two images of a person climbing
a penny, both of which work, in a way (even though they are peculiar).
What I am trying to get at is that what sounds more plausible is that
the person builds some kind of mental model of the situation, and the
components of this model are well-connected enough that *their* attempts
to form a coherent structure tells the rest of the brain that something
is not quite right with the sentence. Do those mental model components
make something that should be described as an image, or a picture? I am
not so sure: they really do not have to go all the way down to an
internal canvas on which lines are positioned (simply no need for them
to get that specific: they can do their job without the lines), so
perhaps it would not be right to call them a 'picture'.
The way I am pushing your idea is in the direction of a cluster of ideas
that are common to the work of me, and people like Phil Johnson-Laird
(his book Mental Models was about how people build models when trying to
do reasoning tasks, among other things) and Doug Hofstadter (especially
see his cute little paper called "Jumbo" which is in the Fluid Analogies
book).
These ideas are also closely related to many very general ideas about
internal representation: you could stretch them to cover Goertzel's
work, and probably everyone else building a system on this list! The
idea that a mental representation is built, then tested in some way to
make deductions about the outside world, is a very common one.
What is interesting about your idea is a subtle background hint of
simultaneous constraint satisfaction (common to Hofstadter, me, many
people in the "parallel distributed processing" school of thought, etc).
That part I like.
Be good if you could write your ideas up in essay form.
[Be good if you could do that yourself Loosemore. Okay, okay. ;-) ]
Richard Loosemore.
Mike Tintner wrote:
Richard,
My apologies for coming on too strong.
Re psychosemiotics, if there were such a science, yes, it would probably
come under cognitive psychology. Is there a need for such a science?
Yes. See my "picture theory" below. But regardless of that, there is an
obvious need to look at human sign systems and media as a totality, what
they reflect about the brain's and human system's needs to apprehend the
world, how they change the human nervous system, and how they will
evolve in future - human sign systems are manifestly continuously
evolving. It should be obvious that such a science would fit in a timely
way with the new multimedia age and the new multimedia (as opposed to
literate) mentality (but I can expand on that another time).
Re Kosslyn, etc, I presume what you are really after is: what am I
personally saying, if anything, that is different from Kosslyn etc
and/or of any interest? Well, I am saying something v. different, and
if nothing else, it serves to place the imagery debate in context.
(Thanks BTW for all the links - which I found stimulating).
What my "picture tree" theory - to put it EXTREMELY briefly - says is :
the brain continually tries to process all incoming info. on at least 3
levels of abstractness/ concreteness simultaneously - as:
*symbols - totally abstract, non-referential signs like words and numbers
*graphics - which are principally the OUTLINES of objects -
exemplified by children's drawings, icons, cartoons, maps, et
*images - which, as I use the word, are more detailed representations
of objects - exemplified by photos, realistic drawings, paintings,
statues - but also involve all sensory forms not just visual
(obviously there is a difference between the brain's internal picture
tree, with its internal symbols, graphics & images, and the external
cultural forms of them).
The brain literally tries to "make sense" of all incoming info. - the
primary object being to test the truth of all info.
So when the brain reads words, it continually tries to convert them into
graphics and images - although much or all of this last activity may be
taking place UNCONSCIOUSLY.
How does the brain know that:
"The man climbed the penny "
is nonsense? Because it will literally draw a picture in graphics of a
man trying to climb a penny - and from the graphics deduce that that is
impossible - although it will also be able to say in some instances:
"well, maybe if it were a specially built outsize penny"... and the way
it will arrive at that is by graphically drawing an outsize penny, and
concluding from both graphics and images, that that is a physical though
unusual possibility.
The brain's graphics are principally dynamic, metamorphic (flexible
shape) graphics - not so much still graphics. The brain then I am
suggesting is continually drawing composite, dynamic graphic pictures to
make sense of sentences, and, where relevant, testing them against the
still more detailed images of its actual experience. (Remember that the
prime form that our sensory images of the world take are moving "VIDEOS"
not still images - you can see why I find Hawkins interesting).
What is distinctive, and unquestionably important about this theory, is
that it introduces a new category of signs. Both semiotics and the
traditional psychological/philosophical debates divide signs into two :
symbols vs icons or images. I'm pointing out what is obvious - that
"graphics" should be treated separately, and form an important separate
category of sign systems.
What's also distinctive about it is that it connects up how the brain
works with the great many different external sign systems that humans
have produced and use - there has to be value in that.
The imagery debate is v. different to all this - it's about in what form
images are laid down in the brain, and what happens when we consciously
form mental images, and imagine composite scenes and scenarios.
There is an obvious longstanding philosophical debate that forms a
background to the imagery debate, about whether symbols are grounded in
images, but it's all very general.
What I'm proposing is an overall, far more comprehensive theory of how
the brain continuously processes information - the brain's "picture
tree". And it has major consequences and applications. For example,
putting this v. simplistically, it says that those subjects which humans
find most difficult to understand and confusing, are precisely those
subjects like quantum mechanics, philosophy, or genetics which are most
abstract and literally "do not make sense" as presented - are not
adequately illustrated.
The theory also has applications to AGI - if it's correct, it's saying
that the central source of human adaptivity is the use of graphics
(backed by images). It's mainly although by no means exclusively
graphics that enable the human brain to connect just about anything to
anything else. (Plato said "Let no one without geometry enter here" -
I'm suggesting that "no one without graphics" (of which geometry is only
one form) can enter AGI). How can the brain think of a virtual infinity
of ways to "move" a given object? Probably because it first draws "move"
as something like a moving arrow. And then it can metamorphically
convert that arrow into, and associate it with, an infinite variety of
more detailed graphic forms of movement like "hammer", "poke", "shovel",
"nail", etc. etc. ad infinitum.
(In case you are interested, a couple of points about the imagery debate
from a brief reading:
*propositions - a still graphic may not be, but a MOVING graphic or
image IS a proposition - it says, for example, this object is moving
here - the cat is sitting on the mat - has anyone pointed this out?
*storing info - the whole nodes thing strikes me as v. unlikely; I
would argue that a graphic or image is a supremely more efficient way to
store info. - a standard map, for example, does not just contain info.
say about how four or five towns are geographically related to each
other, it tells you how every single point on that map is related to
every other point - it enables the planning of a virtual infinity of
journeys - I don't see how that info can be represented symbolically/
formulaically - do you? Ditto, an image of a face enables you to compare
every part of that face to every other part - I don't see how all those
comparisons can be contained in symbolic form.
*homunculus - I find the fear re a homunculus weird for this reason -
every image CONTAINS the viewer - when you look at a photo of a chair
or a mountain, that photo doesn't just show you the chair, it tells you
how far you the viewer are from it, and at what angle you are looking at
it - has this point been made? (Consciousness in other words is an
"itheatre" in which every sensory image is framed by the self viewing or
sensing it).
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?member_id=231415&user_secret=fabd7936
