Bill Jefferys wrote: > > At 10:59 AM +0200 4/3/02, Juergen Schmidhuber wrote: > >The theory of inductive inference is Bayesian, of course. > >But Bayes' rule by itself does not yield Occam's razor. > > "By itself?" No one said it did. Of course assumptions must be made. > At minimum one always has to choose priors in Bayesian inference. > > Our paper shows that there is a Bayesian interpretation that yields > something very suggestive of Ockham's razor. It is appealing in that > if one has a "simple" versus a "complex" hypothesis, "simple" meaning > that the prior probability is concentrated and "complex" meaning that > it is vague and spread out, "simple" meaning that you don't have many > knobs to tweak, "complex" meaning the opposite, then the "simple" > hypothesis will be favored over the "complex" one unless the data lie > well away from where the "simple" hypothesis has placed its bets. > Bayesians distinguish this from Ockham's formulation by calling it > the "Bayesian Ockham's razor", recognizing that it is not what > William of Ockham wrote, "Entia non sunt multiplicanda sine > necessitate" (or one of his other genuine formulations). > > Please don't read more into our article than is there. > > "By itself." First you said that the AP "by itself" has no predictive > power. I missed the "by itself" so misunderstood you, but when I > understood what you were saying I agreed. Now you say that Bayes' > rule "by itself" does not yield Ockham's razor. Jim and I never said > that it did. I am hard pressed to see how anything nontrivial > relating to the real world can be gotten from any principle "by > itself," so I don't regard these comments as very profound, or very > useful. > > [Remainder of article snipped] > > Bill One has to choose priors. Exactly. To repeat the nontrivial point: The only choice you need to obtain Occam's razor is to restrict yourself to priors computable in the limit (this is not much of a restriction in the sense that other priors are beyond constructive description anyway). Then you'll automatically prefer few knobs to tweak, or, more generally, short descriptions of the observations: http://www.idsia.ch/~juergen/toesv2/ Juergen (will be out of town until April 16)

- Re: Optimal Prediction Russell Standish
- Re: Optimal Prediction Hal Finney
- Re: Optimal Prediction H J Ruhl
- Re: Optimal Prediction Bill Jefferys

- Re: Optimal Prediction Hal Finney
- Re: Optimal Prediction H J Ruhl
- Re: Optimal Prediction Hal Finney
- Re: Optimal Prediction Bill Jefferys
- Re: Optimal Prediction Juergen Schmidhuber
- Re: Optimal Prediction Bill Jefferys
- Re: Optimal Prediction Juergen Schmidhuber
- Re: Optimal Prediction Bill Jefferys

- Re: Optimal Prediction Hal Finney
- Re: Optimal Prediction Hal Finney