Thank you very much Thomas. So in fact the walk-forward measures the data-mining bias in some way? I will read what you say I should read, and I will look at chapter 20 in Howard's book...
But still, so far I get the impression that if I backtested let's say cross (ma,ma...) for 2000 to 2008 and I take this best result and it is 100% CAR, then eben if OOS is 50% CAR I guess there can still be place for data-ming bias (or curve-fitting) because the optimization was done with the best result. Anyway, if I understand correctly what you say, the OOS will ALWAYS be less than IS because the IS is optimized - that is, it will take the best-of-100 (or 200, etc.) result, and compare it with a more random result that would occure in real life. Am I wrong? Thanks, Louis 2008/4/16, Thomas Ludwig <[EMAIL PROTECTED]>: > > Louis, > > in the IS period your system is optimized, then the best values from the > optimization are used to perform a test over the IS and OOS periods. > > If the OOS results are worse than the IS results, this means that the > system doesn't generalize well enough. BTW: This topic is very well > explained in chapter 20 of Howard's book. I also suggest to look at > > http://www.amibroker.com/kb/2008/02/12/getting-started-with-automatic-walk-forward-optimization/. > > I must say, that walk forward testing has completely changed my way of > thinking. It's much easier to see now if a trading system is worth a > second look. > > Greetings, > > Thomas > > > > Hi, > > > > I've been experimenting with walking-forward, and I have some > > questions regarding how it works. > > > > I ran a complete random optimization or buying/selling using the > > variables I set (a MCS in fact), and systematically OOS results were > > worst than IS. I don't understand how it works, because whatever if > > the sampling is IS or OOS it is always the same variables that are in > > place. > > > > Anyone could explain how this work? > > > > Thanks, > > > > Louis > > > > > ------------------------------------ > > Please note that this group is for discussion between users only. > > To get support from AmiBroker please send an e-mail directly to > SUPPORT {at} amibroker.com > > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG: > http://www.amibroker.com/devlog/ > > For other support material please check also: > http://www.amibroker.com/support.html > Yahoo! Groups Links > > > >
