Your are absolutely right Eugene. That is why it is important that both in-sample and out-of-sample be representative, i.e. contain both English and Japanese. For limited dtatasets, it is hard to create two representative sets while maintaining large enough sample size to yield statistically significant results. Hence, "bootstrapping" procedures, of which walk-forward is one example.
Using your analogy, if the market shifts from English to Japanese abruptly, through a "bubble" burst or other dramatic "black swan" event, all parameter optimization done on "English" piece will of course be useless and you will have losses. If the market shifts gradually, as they usually do, walk-forward optimization will prevent the loss. > > It is like with people, if you really want someone to understand > something, you will teach him (presenting examples > is one approach for teaching). But at the end you want to know whether > he really understood (i.e., got the principles > and is able to use them to solve knew problems) or whether he just > memorized. The only way to find this out is to show him s.th. he has > not seen before... That's a good analogy, and it does demonstrate the benefits of the out- of-sample testing. However, it also illuminates the trap. Let's take your example and modify it a little. Let's say you are teaching a child to read in English. She made a good progress thus far, so you decide to test whether she *really* learned how to read. You present a test, which happens to be a Japanese poetry piece. Naturally, the child gets an "fail" on that test, and so do you as a teacher. The same thing may happen with your "in-sample, out of sample" approach. If your in-sample is too short, your system may learn the patterns while the market was "speaking" English, and apply those patterns when the market shifted to speaking Japanese. -- You received this message because you are subscribed to the Google Groups "JBookTrader" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/jbooktrader?hl=en. ________________________________ From: nonlinear5 <[email protected]> To: JBookTrader <[email protected]> Sent: Thu, December 9, 2010 11:52:09 PM Subject: [JBookTrader] Re: Dynamic Parameter Optimization -- You received this message because you are subscribed to the Google Groups "JBookTrader" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/jbooktrader?hl=en.
