The connection between intelligence and compression is not obvious.
The connection between intelligence and compression *is* obvious -- but
compression, particularly lossless compression, is clearly *NOT*
intelligence.
Intelligence compresses knowledge to ever simpler rules because that is an
effective way of dealing with the world. Discarding ineffective/unnecessary
knowledge to make way for more effective/necessary knowledge is an effective
way of dealing with the world. Blindly maintaining *all* knowledge at
tremendous costs is *not* an effective way of dealing with the world (i.e.
it is *not* intelligent).
1. What Hutter proved is that the optimal behavior of an agent is to guess
that the environment is controlled by the shortest program that is
consistent with all of the interaction observed so far. The problem of
finding this program known as AIXI.
2. The general problem is not computable [11], although Hutter proved
that if we assume time bounds t and space bounds l on the environment,
then this restricted problem, known as AIXItl, can be solved in O(t2l)
time
Very nice -- except that O(t2l) time is basically equivalent to incomputable
for any real scenario. Hutter's proof is useless because it relies upon the
assumption that you have adequate resources (i.e. time) to calculate AIXI --
which you *clearly* do not. And like any other proof, once you invalidate
the assumptions, the proof becomes equally invalid. Except as an
interesting but unobtainable edge case, why do you believe that Hutter has
any relevance at all?
----- Original Message -----
From: "Matt Mahoney" <[EMAIL PROTECTED]>
To: <[email protected]>
Sent: Wednesday, November 15, 2006 2:54 PM
Subject: Re: [agi] A question on the symbol-system hypothesis
Richard, what is your definition of "understanding"? How would you test
whether a person understands art?
Turing offered a behavioral test for intelligence. My understanding of
"understanding" is that it is something that requires intelligence. The
connection between intelligence and compression is not obvious. I have
summarized the arguments here.
http://cs.fit.edu/~mmahoney/compression/rationale.html
-- Matt Mahoney, [EMAIL PROTECTED]
----- Original Message ----
From: Richard Loosemore <[EMAIL PROTECTED]>
To: [email protected]
Sent: Wednesday, November 15, 2006 2:38:49 PM
Subject: Re: [agi] A question on the symbol-system hypothesis
Matt Mahoney wrote:
Richard Loosemore <[EMAIL PROTECTED]> wrote:
"Understanding" 10^9 bits of information is not the same as storing 10^9
bits of information.
That is true. "Understanding" n bits is the same as compressing some
larger training set that has an algorithmic complexity of n bits. Once
you have done this, you can use your probability model to make predictions
about unseen data generated by the same (unknown) Turing machine as the
training data. The closer to n you can compress, the better your
predictions will be.
I am not sure what it means to "understand" a painting, but let's say that
you understand art if you can identify the artists of paintings you
haven't seen before with better accuracy than random guessing. The
relevant quantity of information is not the number of pixels and
resolution, which depend on the limits of the eye, but the (much smaller)
number of features that the high level perceptual centers of the brain are
capable of distinguishing and storing in memory. (Experiments by Standing
and Landauer suggest it is a few bits per second for long term memory, the
same rate as language). Then you guess the shortest program that
generates a list of feature-artist pairs consistent with your knowledge of
art and use it to predict artists given new features.
My estimate of 10^9 bits for a language model is based on 4 lines of
evidence, one of which is the amount of language you process in a
lifetime. This is a rough estimate of course. I estimate 1 GB (8 x 10^9
bits) compressed to 1 bpc (Shannon) and assume you remember a significant
fraction of that.
Matt,
So long as you keep redefining "understand" to mean whatever something
trivial (or at least, something different in different circumstances),
all you do is reinforce the point I was trying to make.
In your definition of "understanding" in the context of art, above, you
specifically choose an interpretation that enables you to pick a
particular bit rate. But if I chose a different interpretation (and I
certainly would - an art historian would never say they understood a
painting just because they could tell the artist's style better than a
random guess!), I might come up with a different bit rate. And if I
chose a sufficiently subtle concept of "understand", I would be unable
to come up with *any* bit rate, because that concept of "understand"
would not lend itself to any easy bit rate analysis.
The lesson? Talking about bits and bit rates is completely pointless
.... which was my point.
You mainly identify the meaning of "understand" as a variant of the
meaning of "compress". I completely reject this - this is the most
idiotic development in AI research since the early attempts to do
natural language translation using word-by-word lookup tables - and I
challenge you to say why anyone could justify reducing the term in such
an extreme way. Why have you thrown out the real meaning of
"understand" and substituted another meaning? What have we gained by
dumbing the concept down?
As I said in previously, this is as crazy as redefining the complex
concept of "happiness" to be "a warm puppy".
Richard Loosemore
Landauer, Tom (1986), “How much do people
remember? Some estimates of the quantity
of learned information in long term memory”, Cognitive Science (10) pp.
477-493
Shannon, Cluade E. (1950), “Prediction and
Entropy of Printed English”, Bell Sys. Tech. J (3) p. 50-64.
Standing, L. (1973), “Learning 10,000 Pictures”,
Quarterly Journal of Experimental Psychology (25) pp. 207-222.
-- Matt Mahoney, [EMAIL PROTECTED]
----- Original Message ----
From: Richard Loosemore <[EMAIL PROTECTED]>
To: [email protected]
Sent: Wednesday, November 15, 2006 9:33:04 AM
Subject: Re: [agi] A question on the symbol-system hypothesis
Matt Mahoney wrote:
I will try to answer several posts here. I said that the knowledge
base of an AGI must be opaque because it has 10^9 bits of information,
which is more than a person can comprehend. By opaque, I mean that you
can't do any better by examining or modifying the internal
representation than you could by examining or modifying the training
data. For a text based AI with natural language ability, the 10^9 bits
of training data would be about a gigabyte of text, about 1000 books. Of
course you can sample it, add to it, edit it, search it, run various
tests on it, and so on. What you can't do is read, write, or know all of
it. There is no internal representation that you could convert it to
that would allow you to do these things, because you still have 10^9
bits of information. It is a limitation of the human brain that it can't
store more information than this.
"Understanding" 10^9 bits of information is not the same as storing 10^9
bits of information.
A typical painting in the Louvre might be 1 meter on a side. At roughly
16 pixels per millimeter, and a perceivable color depth of about 20 bits
that would be about 10^8 bits. If an art specialist knew all about,
say, 1000 paintings in the Louvre, that specialist would "understand" a
total of about 10^11 bits.
You might be inclined to say that not all of those bits count, that many
are redundant to "understanding".
Exactly.
People can easily comprehend 10^9 bits. It makes no sense to argue
about degree of comprehension by quoting numbers of bits.
Richard Loosemore
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303
