Who is talking about efficiency? An infinite sequence of uncomputable values is still just as uncomputable. I don't disagree that AIXI and Solomonoff induction are not computable. But you are also arguing that they are wrong.
-- Matt Mahoney, [email protected] ________________________________ From: Jim Bromer <[email protected]> To: agi <[email protected]> Sent: Wed, July 7, 2010 6:40:52 PM Subject: Re: [agi] Solomonoff Induction is Not "Universal" and Probability is not "Prediction" Matt, But you are still saying that Solomonoff Induction has to be recomputed for each possible combination of bit value aren't you? Although this doesn't matter when you are dealing with infinite computations in the first place, it does matter when you are wondering if this has anything to do with AGI and compression efficiencies. Jim Bromer On Wed, Jul 7, 2010 at 5:44 PM, Matt Mahoney <[email protected]> wrote: Jim Bromer wrote: >> But, a more interesting question is, given that the first digits are 000, >> what >>are the chances that the next digit will be 1? Dim Induction will report .5, >>which of course is nonsense and a whole less useful than making a rough guess. > > >Wrong. The probability of a 1 is p(0001)/(p(0000)+p(0001)) where the >probabilities are computed using Solomonoff induction. A program that outputs >0000 will be shorter in most languages than a program that outputs 0001, so 0 >is >the most likely next bit. > > >More generally, probability and prediction are equivalent by the chain rule. >Given any 2 strings x followed by y, the prediction p(y|x) = p(xy)/p(x). > > -- Matt Mahoney, [email protected] > > > > > ________________________________ From: Jim Bromer <[email protected]> >To: agi <[email protected]> >Sent: Wed, July 7, 2010 10:10:37 AM >Subject: [agi] Solomonoff Induction is Not "Universal" and Probability is not >"Prediction" > > > >Suppose you have sets of "programs" that produce two strings. One set of >outputs is 000000 and the other is 111111. Now suppose you used these sets of >programs to chart the probabilities of the output of the strings. If the two >strings were each output by the same number of programs then you'd have a .5 >probability that either string would be output. That's ok. But, a more >interesting question is, given that the first digits are 000, what are the >chances that the next digit will be 1? Dim Induction will report .5, which of >course is nonsense and a whole less useful than making a rough guess. > >But, of course, Solomonoff Induction purports to be able, if it was feasible, >to >compute the possibilities for all possible programs. Ok, but now, try >thinking >about this a little bit. If you have ever tried writing random program >instructions what do you usually get? Well, I'll take a hazard and guess (a >lot >better than the bogus method of confusing shallow probability with >"prediction" >in my example) and say that you will get a lot of programs that crash. Well, >most of my experiments with that have ended up with programs that go into an >infinite loop or which crash. Now on a universal Turing machine, the results >would probably look a little different. Some strings will output nothing and >go >into an infinite loop. Some programs will output something and then either >stop >outputting anything or start outputting an infinite loop of the same >substring. >Other programs will go on to infinity producing something that looks like >random >strings. But the idea that all possible programs would produce well >distributed >strings is complete hogwash. Since Solomonoff Induction does not define what >kind of programs should be used, the assumption that the distribution would >produce useful data is absurd. In particular, the use of the method to >determine the probability based given an initial string (as in what follows >given the first digits are 000) is wrong as in really wrong. The idea that >this >crude probability can be used as "prediction" is unsophisticated. > >Of course you could develop an infinite set of Solomonoff Induction values for >each possible given initial sequence of digits. Hey when you're working with >infeasible functions why not dream anything? > >I might be wrong of course. Maybe there is something you guys haven't been >able >to get across to me. Even if you can think for yourself you can still make >mistakes. So if anyone has actually tried writing a program to output all >possible programs (up to some feasible point) on a Turing Machine simulator, >let >me know how it went. > >Jim Bromer > >agi | Archives | Modify Your Subscription >agi | Archives | Modify Your Subscription agi | Archives | Modify Your Subscription ------------------------------------------- agi Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: https://www.listbox.com/member/?member_id=8660244&id_secret=8660244-6e7fb59c Powered by Listbox: http://www.listbox.com
