At 10:59 AM +0200 4/3/02, Juergen Schmidhuber wrote:
>The theory of inductive inference is Bayesian, of course.
>But Bayes' rule by itself does not yield Occam's razor.

"By itself?" No one said it did. Of course assumptions must be made. 
At minimum one always has to choose priors in Bayesian inference.

Our paper shows that there is a Bayesian interpretation that yields 
something very suggestive of Ockham's razor. It is appealing in that 
if one has a "simple" versus a "complex" hypothesis, "simple" meaning 
that the prior probability is concentrated and "complex" meaning that 
it is vague and spread out, "simple" meaning that you don't have many 
knobs to tweak, "complex" meaning the opposite, then the "simple" 
hypothesis will be favored over the "complex" one unless the data lie 
well away from where the "simple" hypothesis has placed its bets. 
Bayesians distinguish this from Ockham's formulation by calling it 
the "Bayesian Ockham's razor", recognizing that it is not what 
William of Ockham wrote, "Entia non sunt multiplicanda sine 
necessitate" (or one of his other genuine formulations).

Please don't read more into our article than is there.

"By itself." First you said that the AP "by itself" has no predictive 
power. I missed the "by itself" so misunderstood you, but when I 
understood what you were saying I agreed. Now you say that Bayes' 
rule "by itself" does not yield Ockham's razor. Jim and I never said 
that it did. I am hard pressed to see how anything nontrivial 
relating to the real world can be gotten from any principle "by 
itself," so I don't regard these comments as very profound, or very 

[Remainder of article snipped]


Reply via email to