> Isn't it the out-of-sample results we are trying to estimate? Do we care > what the in-sample results look like? If the out-of-sample results are > terrible, why bother computing the ratio. If the out-of-sample results are > good, why bother computing the ratio -- how will that information be used to > improve the system? And if it is used to modify the system, then the > previously out-of-sample data has become in-sample. > > Do any of the forum member have examples they can contribute where computing > the ratio is helpful?
Consider a low WFE as a warning that the walk forward results are suspect and may be contaminated by data snooping. As I recall, Pardo was a little vague on the details of how to calculate. One way: Use Ami's walk forward and view the OOS graph that is scaled and concatenated. Take the ending net profit value and write it down. This assumes fixed factional position sizing rather than fixed $. Now do a simple optimization over the time period covered by the OOS period above. Take the ending net profit value from the highest scoring run and write it down. Take the ratio. It can be > 1, but is probably somewhat less. Less than .65 or so is a big red flag. How can a system have good WF OOS results yet have a poor WFE? Easy. Just use all of the data to design your system, then do the WF test. Or, do lots of WF tests while tweaking the rules. How many of us truly have never done this? So, that supper duper system that has 8 optimizable variables that the trader spent the last year designing and after much effort, now gets 300% CAR out of sample, but a .3 WFE? How do you think it will trade live? 300% car, like the OOS? I'd take the other side of that trade.
