Abram, I don't think you are right. The reason is that Solomonoff Induction does not produce a true universal probability for any given first digits. To do so it would have to be capable of representing the probability of any (computable) sequence that follows any (computable) string of given first digits.
Yes, if a high proportion of programs produce 000000, it will be able to register that as string as more probable, but the information on what the next digits will be, given some input, will not be represented in anything that resembled "compression". For instance, if you had 62 bits and wanted to know what the probability of the next two bits were, you would have to have done the infinite calculations of a Solomonoff Induction for each of the 2^62 possible combination of bits that represented the possible input to your problem. I might be wrong, but I don't see where all this is "information" is being hidden if I am. On the other hand, if I am right (or even partially right) I don't understand why seemingly smart people are excited about this as a possible AGI method. We in AGI specifically want to know the answer to the kind of question: Given some partially defined situation, how could a computer best figure out what is going on. Most computer situations are going to be represented by kilobytes or megabytes these days, not in strings of 32 bits or less. If there was an abstraction that could help us think about these things, it could help even if the ideal would be way beyond any feasible technology. And there is an abstraction like this that can help us. Applied probability. We can think about these ideas in the terms of strings if we want to but the key is that WE have to work out the details because we see the problems differently. There is nothing that I have seen in Solomonoff Induction that suggests that this is an adequate or even useful method to use. On the other hand I would not be talking about this if it weren't for Solomonoff so maybe I just don't share your enthusiasm. If I have misunderstood something then all I can say is that I am still waiting for someone to explain it in a way that I can understand. Jim On Wed, Jul 7, 2010 at 1:58 PM, Abram Demski <abramdem...@gmail.com> wrote: > Jim, > > I am unable to find the actual objection to Solomonoff in what you wrote > (save for that it's "wrong as in really wrong"). > > It's true that a lot of programs won't produce any output. That just means > they won't alter the prediction. > > It's also true that a lot of programs will produce random-looking or > boring-looking output. This just means that Solomonoff will have some > expectation of those things. To use your example, given 000, the chances > that the next digit will be 0 will be fairly high thanks to boring programs > which just output lots of zeros. (Not sure why you mention the idea that it > might be .5? This sounds like "no induction" rather than "dim induction"...) > > --Abram > > On Wed, Jul 7, 2010 at 10:10 AM, Jim Bromer <jimbro...@gmail.com> wrote: > >> Suppose you have sets of "programs" that produce two strings. One set >> of outputs is 000000 and the other is 111111. Now suppose you used these >> sets of programs to chart the probabilities of the output of the strings. >> If the two strings were each output by the same number of programs then >> you'd have a .5 probability that either string would be output. That's ok. >> But, a more interesting question is, given that the first digits are 000, >> what are the chances that the next digit will be 1? Dim Induction will >> report .5, which of course is nonsense and a whole less useful than making a >> rough guess. >> >> But, of course, Solomonoff Induction purports to be able, if it was >> feasible, to compute the possibilities for all possible programs. Ok, but >> now, try thinking about this a little bit. If you have ever tried writing >> random program instructions what do you usually get? Well, I'll take a >> hazard and guess (a lot better than the bogus method of confusing shallow >> probability with "prediction" in my example) and say that you will get a lot >> of programs that crash. Well, most of my experiments with that have ended >> up with programs that go into an infinite loop or which crash. Now on a >> universal Turing machine, the results would probably look a little >> different. Some strings will output nothing and go into an infinite loop. >> Some programs will output something and then either stop outputting anything >> or start outputting an infinite loop of the same substring. Other programs >> will go on to infinity producing something that looks like random strings. >> But the idea that all possible programs would produce well distributed >> strings is complete hogwash. Since Solomonoff Induction does not define >> what kind of programs should be used, the assumption that the distribution >> would produce useful data is absurd. In particular, the use of the method >> to determine the probability based given an initial string (as in what >> follows given the first digits are 000) is wrong as in really wrong. The >> idea that this crude probability can be used as "prediction" is >> unsophisticated. >> >> Of course you could develop an infinite set of Solomonoff Induction values >> for each possible given initial sequence of digits. Hey when you're working >> with infeasible functions why not dream anything? >> >> I might be wrong of course. Maybe there is something you guys >> haven't been able to get across to me. Even if you can think for yourself >> you can still make mistakes. So if anyone has actually tried writing a >> program to output all possible programs (up to some feasible point) on a >> Turing Machine simulator, let me know how it went. >> >> Jim Bromer >> >> *agi* | Archives <https://www.listbox.com/member/archive/303/=now> >> <https://www.listbox.com/member/archive/rss/303/> | >> Modify<https://www.listbox.com/member/?&;>Your Subscription >> <http://www.listbox.com/> >> > > > > -- > Abram Demski > http://lo-tho.blogspot.com/ > http://groups.google.com/group/one-logic > *agi* | Archives <https://www.listbox.com/member/archive/303/=now> > <https://www.listbox.com/member/archive/rss/303/> | > Modify<https://www.listbox.com/member/?&;>Your Subscription > <http://www.listbox.com/> > ------------------------------------------- agi Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: https://www.listbox.com/member/?member_id=8660244&id_secret=8660244-6e7fb59c Powered by Listbox: http://www.listbox.com
