Thanks for the explanation. I want to learn more about statistical modelling and compression but I will need to take my time on it. But no, I don't apply Solomonoff Induction all the time, I never apply it. I am not being petty, it's just that you have taken a coincidence and interpreted it the way you want to.
On Thu, Jul 22, 2010 at 9:33 PM, Matt Mahoney <matmaho...@yahoo.com> wrote: > Jim Bromer wrote: > > Please give me a little more explanation why you say the fundamental > method is that the probability of a string x is proportional to the sum of > all programs M that output x weighted by 2^-|M|. Why is the M in a bracket? > > By |M| I mean the length of the program M in bits. Why 2^-|M|? Because each > bit means you can have twice as many programs, so they should count half as > much. > > Being uncomputable doesn't make it wrong. The fact that there is no general > procedure for finding the shortest program that outputs a string doesn't > mean that you can never find it, or that for many cases you can't > approximate it. > > You apply Solomonoff induction all the time. What is the next bit in these > sequences? > > 1. 0101010101010101010101010101010 > > 2. 1100100100001111110110101010001 > > In sequence 1 there is an obvious pattern with a short description. You can > find a short program that outputs 0 and 1 alternately forever, so you > predict the next bit will be 1. It might not be the shortest program, but it > is enough that "alternate 0 and 1 forever" is shorter than "alternate 0 and > 1 15 times followed by 00" that you can confidently predict the first > hypothesis is more likely. > > The second sequence is not so obvious. It looks like random bits. With > enough intelligence (or computation) you might discover that the sequence is > a binary representation of pi, and therefore the next bit is 0. But the fact > that you might not discover the shortest description does not invalidate the > principle. It just says that you can't always apply Solomonoff induction and > get the number you want. > > Perhaps http://en.wikipedia.org/wiki/Kolmogorov_complexity will make this > clear. > > > -- Matt Mahoney, matmaho...@yahoo.com > > > ------------------------------ > *From:* Jim Bromer <jimbro...@gmail.com> > *To:* agi <agi@v2.listbox.com> > *Sent:* Thu, July 22, 2010 5:06:12 PM > > *Subject:* Re: [agi] Comments On My Skepticism of Solomonoff Induction > > On Wed, Jul 21, 2010 at 8:47 PM, Matt Mahoney <matmaho...@yahoo.com>wrote: > The fundamental method is that the probability of a string x is > proportional to the sum of all programs M that output x weighted by 2^-|M|. > That probability is dominated by the shortest program, but it is equally > uncomputable either way. > Also, please point me to this mathematical community that you claim rejects > Solomonoff induction. Can you find even one paper that refutes it? > > You give a precise statement of the probability in general terms, but then > say that it is uncomputable. Then you ask if there is a paper that refutes > it. Well, why would any serious mathematician bother to refute it since you > yourself acknowledge that it is uncomputable and therefore unverifiable and > therefore not a mathematical theorem that can be proven true or false? It > isn't like you claimed that the mathematical statement is verifiable. It is > as if you are making a statement and then ducking any responsibility for it > by denying that it is even an evaluation. You honestly don't see the > irregularity? > > My point is that the general mathematical community doesn't accept > Solomonoff Induction, not that I have a paper that *"refutes it,"*whatever > that would mean. > > Please give me a little more explanation why you say the fundamental method > is that the probability of a string x is proportional to the sum of all > programs M that output x weighted by 2^-|M|. Why is the M in a bracket? > > > On Wed, Jul 21, 2010 at 8:47 PM, Matt Mahoney <matmaho...@yahoo.com>wrote: > >> Jim Bromer wrote: >> > The fundamental method of Solmonoff Induction is trans-infinite. >> >> The fundamental method is that the probability of a string x is >> proportional to the sum of all programs M that output x weighted by 2^-|M|. >> That probability is dominated by the shortest program, but it is equally >> uncomputable either way. How does this approximation invalidate Solomonoff >> induction? >> >> Also, please point me to this mathematical community that you claim >> rejects Solomonoff induction. Can you find even one paper that refutes it? >> >> >> -- Matt Mahoney, matmaho...@yahoo.com >> >> >> ------------------------------ >> *From:* Jim Bromer <jimbro...@gmail.com> >> *To:* agi <agi@v2.listbox.com> >> *Sent:* Wed, July 21, 2010 3:08:13 PM >> >> *Subject:* Re: [agi] Comments On My Skepticism of Solomonoff Induction >> >> I should have said, It would be unwise to claim that this method could >> stand as an "ideal" for some valid and feasible application of probability. >> Jim Bromer >> >> On Wed, Jul 21, 2010 at 2:47 PM, Jim Bromer <jimbro...@gmail.com> wrote: >> >>> The fundamental method of Solmonoff Induction is trans-infinite. Suppose >>> you iterate through all possible programs, combining different programs as >>> you go. Then you have an infinite number of possible programs which have a >>> trans-infinite number of combinations, because each tier of combinations can >>> then be recombined to produce a second, third, fourth,... tier of >>> recombinations. >>> >>> Anyone who claims that this method is the "ideal" for a method of applied >>> probability is unwise. >>> >>> Jim Bromer >>> >> >> *agi* | Archives <https://www.listbox.com/member/archive/303/=now> >> <https://www.listbox.com/member/archive/rss/303/> | >> Modify<https://www.listbox.com/member/?&;>Your Subscription >> <http://www.listbox.com/> >> *agi* | Archives <https://www.listbox.com/member/archive/303/=now> >> <https://www.listbox.com/member/archive/rss/303/> | >> Modify<https://www.listbox.com/member/?&;>Your Subscription >> <http://www.listbox.com/> >> > > *agi* | Archives <https://www.listbox.com/member/archive/303/=now> > <https://www.listbox.com/member/archive/rss/303/> | > Modify<https://www.listbox.com/member/?&;>Your Subscription > <http://www.listbox.com/> > *agi* | Archives <https://www.listbox.com/member/archive/303/=now> > <https://www.listbox.com/member/archive/rss/303/> | > Modify<https://www.listbox.com/member/?&;>Your Subscription > <http://www.listbox.com/> > ------------------------------------------- agi Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: https://www.listbox.com/member/?member_id=8660244&id_secret=8660244-6e7fb59c Powered by Listbox: http://www.listbox.com
