Mike Tintner wrote:
----- Original Message ----- From: "Richard Loosemore" <[EMAIL PROTECTED]>
Mike,
Thanks for the thoughtful and detailed reply.
I'm going to have to be brief because of time pressure, alas.
My overall feeling is that there is an important nugget of truth
beneath the surface of what you say, but I think that when you develop
the idea further, you might find yourself drifting away from the
strict use of the "picture" idea (which seems very important to your
position right now, but I think you might end up lessening its
importance later), and move the concept towards a "representations" or
"mental models" idea instead.
[On ths subject of pictures and uhhh, the brain: I had to copy your
message across to a text processor and convert to lowercase to read it:
the ALL CAPS are giving me a headache... ;-) ]
"PICTURE" is v. important & v. deliberately chosen to force you, the
reader, to realise that you cannot understand that which you cannot
"see" or literally "make sense of". I THINK YOU SHOULD BEAR IN MIND WHEN
CONSIDERING THESE IDEAS THAT THE RESISTANCE TO THE IMPORTANCE OF IMAGES
IS SIMPLY ENORMOUS. THE LITERATE MIND BELIEVES THAT THOUGHT AND
RATIONALITY ARE LARGELY A MATTER OF THE 3 R'S - OF USING LANGUAGE PLUS
MATHS/ NUMBERS/ALGEBRA - AND IMAGES OF ALL KINDS ARE AN EXTRA. THE
SERIOUS THINKING AND REPRESENTATION GETS DONE SYMBOLICALLY.
EVERYWHERE IN OUR CULTURE BATTLES ARE CURRENTLY RAGING BETWEEN THE
LITERATE AND THE COMING MULTIMEDIATE MIND - BETWEEN, FOR EXAMPLE,
SYMBOLIC AI AND EMBODIED, SITUATED ROBOTICS, BETWEEN RATIONAL, COGNITIVE
SCIENCE AND EMBODIED COGNITION, RATIONAL PHILOSOPHY AND EMBODIED
PHILOSOPHY.
THE IMAGERY DEBATE IS ONE MORE EXAMPLE OF THAT BATTLE.
THE LITERATE MENTALITY WILL SIMPLY BE SWEPT AWAY WITHIN THE NEXT DECADE.
IT'S A NONSENSE.
>
NOTE: IT'S NOT A MATTER HERE OF EITHER/OR - OF GRAPHICS AND IMAGES, FOR
EXAMPLE, BEING MORE IMPORTANT THAN SYMBOLS - THAT WOULD BE A NONSENSE TOO.
THE BRAIN AND OUR ENTIRE CULTURE, I AM SUGGESTING, ARE AN INTERDEPENDENT
PICTURE TREE. YOU CAN NO MORE HAVE ABSTRACT WITHOUT CONCRETE, OR VICE
VERSA, THAN YOU CAN HAVE GENERALISATIONS WITHOUT PARTICULARISATIONS - IF
YOU ARE TO UNDERSTAND THE WORLD, AND REPRESENTATIONS OF THE WORLD.
The reason I say this is that you need a very abstract notion of
"picture" to get some aspects of the idea to work. For example, when
you say:
> How does the brain know that:
>
> "The man climbed the penny "
>
> is nonsense? Because it will literally draw a picture in graphics of a
> man trying to climb a penny - and from the graphics deduce that
> that is impossible.
... can you really say that it is the *graphic* that determines the
impossibility? Is it the iumpossibiility of the attempt to draw that
graphic, that leads to the "impossible" verdict? Now that doesn't
seem quite right, because I can draw at least two images of a person
climbing a penny, both of which work, in a way (even though they are
peculiar).
I THINK THAT WAS MY POINT. THE ATTEMPT TO DRAW A MAN CLIMBING A PENNY
WILL PRODUCE A) "IT'S IMPOSSIBLE, IF IT'S A NORMAL PENNY" AND B) BUT IT
MIGHT BE AN OUTSIZE PENNY.
THE EXACT DETAILS OF HOW THE BRAIN MIGHT PROCEED TO DRAW THE SCENE HERE
ARE OBVIOUSLY OPEN TO DEBATE . IT MIGHT ONLY NEED TO START PUTTING A MAN
BESIDE A PENNY TO SEE THE IMPOSSIBILITY AND NOT LITERALLY NEED TO GO TO
THE TROUBLE OF DRAWING THE "CLIMBING." ON THE OTHER HAND, (THINKING
ABOUT IT), BETTER SAFE THAN SORRY. WHY NOT DRAW THE WHOLE SCENE? THE
BRAIN CERTAINLY HAS THE CAPACITY TO DO THIS AND MANY VASTLY MORE ARDUOUS
VISUAL TASKS ON THE FLY.
There is ONE BIG THING HERE THAT YOU ARE NOT GETTING. If you were to
sit down and try to implement an actual system that did the above, how
would you get it actually DO the drawing? What mechanisms would be in
there that, after looking at the WORDS, would conclude from the words
"the man climbed the penny" that a drawing of a penny and a man were
involved? How would those mechanisms choose what kind of man, what kind
of penny, what amount of detail in both? Would it spend a lot of time
deciding what clothes the man was wearing? And what his voice sounded
like? What color to use for the lines? How big to make the picture?
Whether to use an animation rather than a still picture? How would it
decide that there was something wrong with the picture of a man standing
on a regular sized penny? (I can do that: there, I just did, and it
was easy. Hey!! You just visualized me getting up and doing it. Cut
that out! :-)). Where in the picture you just visualized, of me doing
it, was the bit of the picture that said 'this is wrong, so classify the
sentence "man climbs on penny" as a meaningless sentence.
If not, what mechanisms would have been responsible for deciding that
the voice and the clothes, etc. were not relevant? How would this
mechanism know which bits were relevant and which not? If it made all
these decisions using pictures, what pictures would THEY be, and how
would it decide on the relevant bits of THOSE pictures?
If you think *everything* is done with pictures, you have to answer all
these issues.
When you try, you will run it to the most immense brick wall.
Please, rather than arguing in the abstract, start expanding on the
model you have suggested and really answer the questions I just posed.
As for your comments about not understanding the mental models idea, I
seriously doubt that it is only because the ideas are meaningless, and
need to be translated into pictures. There are a lot of people who do
understand them, who do not think that they do so in terms of pictures.
Perhaps there is one small point you are missing: the idea of a mental
model is NOT the same as a "symbolic" or "propositional" model. It is
not a literal picture, and not a propositional representation, but
somewhere in between. (I am using propositional and symbolic in their
standard sense, which is a little subtle).
Richard Loosemore
What I am trying to get at is that what sounds more plausible is that
the person builds some kind of mental model of the situation, and the
components of this model are well-connected enough that *their*
attempts to form a coherent structure tells the rest of the brain that
something is not quite right with the sentence. Do those mental model
components make something that should be described as an image, or a
picture? I am not so sure: they really do not have to go all the way
down to an internal canvas on which lines are positioned (simply no
need for them to get that specific: they can do their job without the
lines), so perhaps it would not be right to call them a 'picture'.
WHY DON'T YOU SPELL OUT OR DRAW OUT WHAT YOU MEAN BY THIS MENTAL MODEL
BUSINESS? I CAN'T UNDERSTAND WHAT YOU MEAN - BECAUSE, AS PER MY
THEORY, I LITERALLY CAN'T "SEE" WHAT YOU'RE TALKING ABOUT. AND I SUSPECT
THAT IF WE DID A SURVEY OTHERS WOULD SIMILARLY HAVE DIFFICULTY
UNDERSTANDING YOUR SUGGESTIONS.
(YOU UNDERSTAND HERE, I HOPE, THAT I'M NOT IN ANY WAY TRYING TO GET AT
YOU, BUT MERELY TO REINFORCE MY THEORY!).
The way I am pushing your idea is in the direction of a cluster of
ideas that are common to the work of me, and people like Phil
Johnson-Laird (his book Mental Models was about how people build
models when trying to do reasoning tasks, among other things) and Doug
Hofstadter (especially see his cute little paper called "Jumbo" which
is in the Fluid Analogies book).
These ideas are also closely related to many very general ideas about
internal representation: you could stretch them to cover Goertzel's
work, and probably everyone else building a system on this list! The
idea that a mental representation is built, then tested in some way to
make deductions about the outside world, is a very common one.
What is interesting about your idea is a subtle background hint of
simultaneous constraint satisfaction (common to Hofstadter, me, many
people in the "parallel distributed processing" school of thought,
etc). That part I like.
Be good if you could write your ideas up in essay form.
[Be good if you could do that yourself Loosemore. Okay, okay. ;-) ]
Richard Loosemore.
Mike Tintner wrote:
Richard,
My apologies for coming on too strong.
Re psychosemiotics, if there were such a science, yes, it would
probably come under cognitive psychology. Is there a need for such a
science? Yes. See my "picture theory" below. But regardless of that,
there is an obvious need to look at human sign systems and media as a
totality, what they reflect about the brain's and human system's
needs to apprehend the world, how they change the human nervous
system, and how they will evolve in future - human sign systems are
manifestly continuously evolving. It should be obvious that such a
science would fit in a timely way with the new multimedia age and the
new multimedia (as opposed to literate) mentality (but I can expand
on that another time).
Re Kosslyn, etc, I presume what you are really after is: what am I
personally saying, if anything, that is different from Kosslyn etc
and/or of any interest? Well, I am saying something v. different,
and if nothing else, it serves to place the imagery debate in context.
(Thanks BTW for all the links - which I found stimulating).
What my "picture tree" theory - to put it EXTREMELY briefly - says is :
the brain continually tries to process all incoming info. on at least
3 levels of abstractness/ concreteness simultaneously - as:
*symbols - totally abstract, non-referential signs like words and
numbers
*graphics - which are principally the OUTLINES of objects -
exemplified by children's drawings, icons, cartoons, maps, et
*images - which, as I use the word, are more detailed
representations of objects - exemplified by photos, realistic
drawings, paintings, statues - but also involve all sensory forms not
just visual
(obviously there is a difference between the brain's internal picture
tree, with its internal symbols, graphics & images, and the external
cultural forms of them).
The brain literally tries to "make sense" of all incoming info. - the
primary object being to test the truth of all info.
So when the brain reads words, it continually tries to convert them
into graphics and images - although much or all of this last activity
may be taking place UNCONSCIOUSLY.
How does the brain know that:
"The man climbed the penny "
is nonsense? Because it will literally draw a picture in graphics of
a man trying to climb a penny - and from the graphics deduce that
that is impossible - although it will also be able to say in some
instances: "well, maybe if it were a specially built outsize
penny"... and the way it will arrive at that is by graphically
drawing an outsize penny, and concluding from both graphics and
images, that that is a physical though unusual possibility.
The brain's graphics are principally dynamic, metamorphic (flexible
shape) graphics - not so much still graphics. The brain then I am
suggesting is continually drawing composite, dynamic graphic pictures
to make sense of sentences, and, where relevant, testing them against
the still more detailed images of its actual experience. (Remember
that the prime form that our sensory images of the world take are
moving "VIDEOS" not still images - you can see why I find Hawkins
interesting).
What is distinctive, and unquestionably important about this theory,
is that it introduces a new category of signs. Both semiotics and the
traditional psychological/philosophical debates divide signs into two
: symbols vs icons or images. I'm pointing out what is obvious - that
"graphics" should be treated separately, and form an important
separate category of sign systems.
What's also distinctive about it is that it connects up how the brain
works with the great many different external sign systems that humans
have produced and use - there has to be value in that.
The imagery debate is v. different to all this - it's about in what
form images are laid down in the brain, and what happens when we
consciously form mental images, and imagine composite scenes and
scenarios.
There is an obvious longstanding philosophical debate that forms a
background to the imagery debate, about whether symbols are grounded
in images, but it's all very general.
What I'm proposing is an overall, far more comprehensive theory of
how the brain continuously processes information - the brain's
"picture tree". And it has major consequences and applications. For
example, putting this v. simplistically, it says that those subjects
which humans find most difficult to understand and confusing, are
precisely those subjects like quantum mechanics, philosophy, or
genetics which are most abstract and literally "do not make sense" as
presented - are not adequately illustrated.
The theory also has applications to AGI - if it's correct, it's
saying that the central source of human adaptivity is the use of
graphics (backed by images). It's mainly although by no means
exclusively graphics that enable the human brain to connect just
about anything to anything else. (Plato said "Let no one without
geometry enter here" - I'm suggesting that "no one without graphics"
(of which geometry is only one form) can enter AGI). How can the
brain think of a virtual infinity of ways to "move" a given object?
Probably because it first draws "move" as something like a moving
arrow. And then it can metamorphically convert that arrow into, and
associate it with, an infinite variety of more detailed graphic forms
of movement like "hammer", "poke", "shovel", "nail", etc. etc. ad
infinitum.
(In case you are interested, a couple of points about the imagery
debate from a brief reading:
*propositions - a still graphic may not be, but a MOVING graphic or
image IS a proposition - it says, for example, this object is moving
here - the cat is sitting on the mat - has anyone pointed this out?
*storing info - the whole nodes thing strikes me as v. unlikely; I
would argue that a graphic or image is a supremely more efficient way
to store info. - a standard map, for example, does not just contain
info. say about how four or five towns are geographically related to
each other, it tells you how every single point on that map is
related to every other point - it enables the planning of a virtual
infinity of journeys - I don't see how that info can be represented
symbolically/ formulaically - do you? Ditto, an image of a face
enables you to compare every part of that face to every other part -
I don't see how all those comparisons can be contained in symbolic
form.
*homunculus - I find the fear re a homunculus weird for this reason -
every image CONTAINS the viewer - when you look at a photo of a
chair or a mountain, that photo doesn't just show you the chair, it
tells you how far you the viewer are from it, and at what angle you
are looking at it - has this point been made? (Consciousness in
other words is an "itheatre" in which every sensory image is framed
by the self viewing or sensing it).
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?&;
--
No virus found in this incoming message.
Checked by AVG Free Edition. Version: 7.5.467 / Virus Database:
269.6.4/789 - Release Date: 04/05/2007 17:49
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?&;
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?member_id=231415&user_secret=fabd7936
