Hi Mike, And thanks for your comments. However, the main point I must emphasise is that the OOS1 and IS2 time periods are identical, which is why I find the results so disturbing. Regardless of the fitness function used, the new IS results should at least be able to equal those from the previous OOS, given that an exhaustive search was used. For the record, I used 6 month time periods for both OOS and IS runs, and RAR/MDD as the function to be optimised, based on Howard Bandy's suggestion. My desire is to develop a system that will potentially stay in trades for a considerable time, provided the stock behaves. I believe that extending the timeframes to get more samples would be unlikely to improve matters, because of changes in the market characteristics over a longer period. However, I might experiment with other fitness functions, even reverting to the simple Net Profit.
I hope that either TJ or Howard B might be tempted to comment on the basic issue here - i.e. why can't an exhaustive IS run at least equal the results from the previous OOS run, when the time periods used are identical? Brian --- In [email protected], "Mike" <sfclimb...@...> wrote: > > Hi, > > If your IS time frame is not the same as your OOS time frame (e.g 2 year IS > with rolling 6 month OOS) then the values used in the preceding OOS can end > up more profitable than the optimized values over the following IS since the > IS period is optimized over a different (e.g. longer) data set. > > P.S. > > I would not trust any walk forward that was only producing 1 or 2 trades IS. > While no amount of trades IS can completely predict what will happen OOS, all > of your performance calculations (i.e. fitness function) are based on IS > statistics, and 1 or 2 trades are simply too few data points to calculate any > kind of meaningful statistic. > > I would suggest extending your IS time frame to capture more trades such that > the statistics have a chance to level out. 30 is the number usually suggested > to get a normally distributed sample. Your next OOS period would then be less > likely to be based on an IS fluke. > > Mike > > --- In [email protected], "brianw468" <wild21@> wrote: > > > > Hi > > I recently tested a trading system but got some strange results from the WF > > test. The WF process used exhaustive optimization. > > An extract from the results (too large to reproduce in full) is:- > > Item Profit No. Trades > > IS1 533 1 > > OOS1 1126 2 > > > > IS2 474 1 > > OOS2 -537 2 > > > > IS3 43 1 > > OOS 784 2 > > > > In two cases the OOS results are better than the preceding IS results. > > Though a little surprising,this is quite plausible, because the time > > periods differ. > > However,the IS2 profit is much poorer than the OOS1 profit, and this is a > > worry since exhaustive optimization is used, and the time periods are the > > same. Can anyone offer an explanation? Would it lie in the way profit is > > handled when a trade is not completed by the end of the selected period? > > > > Thanks > > Brian > > >
