On 01/03/2008, Jey Kottalam <[EMAIL PROTECTED]> wrote: > On Sat, Mar 1, 2008 at 3:10 AM, William Pearson <[EMAIL PROTECTED]> wrote: > > > > Keeping the same general shape of the system (trying to account for > > all the detail) means we are likely to overfit, due to trying to model > > systems that are are too complex for us to be able to model, whilst > > trying to model for the noise in our systems. > > > Could you explain this further? I followed what you're saying up to > this paragraph. How and why does the overfitting happen? >
Lets say you have a sine wave on an electrical cable with some noise, you are trying to predict its next value. The noise isn't actually unpredictable, its main components are from radiation from the ionosphere and sun spot activity (or magnetic field variations from the earths crust or whatever). The Turing machine to represent such noise precisely (that is model the sun/ionosphere in enough detail) is more complex than you inductor can represent. However as it is trying to find a program that *exactly* predicts the sequence, sin x will be discarded for some other program that achieves greater accuracy on the training set. Not that it would be bad over fitting, just that it would likely be sin x + a function for pseudo random noise. Real infinite resources Solomonoff induction avoids this by eventually being able to predict what we generally think of as noise. Hope this clears up what I mean. Will Pearson ------------------------------------------- agi Archives: http://www.listbox.com/member/archive/303/=now RSS Feed: http://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: http://www.listbox.com/member/?member_id=8660244&id_secret=95818715-a78a9b Powered by Listbox: http://www.listbox.com
