Philip Hunt wrote:
2008/11/29 Matt Mahoney <[EMAIL PROTECTED]>:
The general problem of detecting overfitting is not computable. The
principle according to Occam's Razor, formalized and proven by
Hutter's AIXI model, is to choose the shortest program (simplest
hypothesis) that generates the data. Overfitting is the case of
choosing a program that is too large.
Can someone explain AIXI to me? My understanding is that you've got
some black-box process emitting output, and you generate all possible
programs that emit the same output, then choose the shortest one.
You then run this program and its subsequent output is what you
predict the black-box process will do. This has the minor drawback,
of course, that it requires infinite processing power and is
therefore slightly impractical.
I've read Hutter's paper "Universal algorithmic intelligence, A
mathematical top->down approach" which amusingly describes itself as
"a gentle introduction to the AIXI model".
Hutter also describes AIXItl of computation time Ord(t*2^L) where I
assume L is the length of the program and I'm not sure what t is. Is
AIXItl something that could be practically written or is it purely a
theoretical construct?
In short, is there something to AIXI or is it something I can safely
ignore?
It is something that, if you do not ignore it, will waste every second
of brain cpu time that you devote to it ;-).
Matt comes has a habit of repeating some version of the above statement
"... according to Occam's Razor, [which was] formalized and proven by
Hutter's AIXI model..." on a semi-periodic basis. The first n times I
took the trouble to explain why this statement is nonsense. Now I don't
bother.
AIXI is mathematical abstraction taken to the point of absurdity and
beyond. By introducing infinite numbers of copies of all possible
universes into your formalism, and by implying that functions can be
computed on such structures, and by redefining common terms like
"intelligence" to be abstractions based on that formalism, you can prove
anything under the sun.
That fact seems to escape some people.
Richard Loosemore
-------------------------------------------
agi
Archives: https://www.listbox.com/member/archive/303/=now
RSS Feed: https://www.listbox.com/member/archive/rss/303/
Modify Your Subscription:
https://www.listbox.com/member/?member_id=8660244&id_secret=120640061-aded06
Powered by Listbox: http://www.listbox.com
