Hi OAM -- The purpose of walk forward testing is to see what happens in the out-of-sample period after a trading has been chosen based on performance over an in-sample period. In walk forward processing, you (the system designer) do not have an opportunity to evaluate any of the alternative systems that are associated with parameter sets other than the one that is at the top of the ranking according to your objective function. The purpose of the objective function is to incorporate everything that is important and arrive at a single-valued score.
So, the question is -- what happens during selection of the best of the alternatives based on the in-sample period that tells you to change something (use a different system) in the out-of-sample period? If you have a quantifiable answer to that question, incorporate whatever decision making you would do into the original objective function and run the tests in the normal manner. Or am I missing something? Thanks, Howard www.blueowlpress.com On Wed, Jan 14, 2009 at 1:57 PM, ozzyapeman <[email protected]> wrote: > Hello, I am trying to branch my code whenever the Walk-Forward engine > is processing OOS data. Hoping someone can give me a tip on this. > > Normally, when one does a Walk-Forward test, the trading system being > tested should be identical for both in-sample (IS) and out-of-sample > (OOS) data. But there are some rare instances when one might want to > use a slightly modified version of the main trading system for the OOS > data. Without going into details on my trading system, assume for the > moment that the reason for doing such a thing is valid. On that basis, > what I essentially want to achieve in my AFL is something like: > > if ( Walk-Forward engine is processing IS data) > > { Use Trading System A } > > if ( Walk-Forward engine is processing OOS data) > > { Use Trading System B } > > However, using traces with ActionEx status, indicates that the engine > state for both of the above cases is always "actionExOptimizeBacktest". > > As such, there is no way (that I can think of) to effectuate the above > branching code. > > Is there a way around this? Some alternate way of testing when OOS > data is being processed vs IS data? Or is the only practical solution > to build a custom version of the Walk-Forward process using OLE with > the AA objects? > > Thank you for any input. > > >
