Hi Richard -- It depends a little on what is being tested, but the questions to ask are: "Is this result probably because the system is good? What is the probability that these results came from random results? Are these results significantly better than some standard I want to use?" Those questions certainly make sense when the result being tested is whether expectancy is greater than 0, or profit per trade is greater than 0. With a function like CAR/MDD, the test is not so clear. Try running some tests using your system and your objective function. See what the critical level of the objective function seems to be -- that is, what level produces results you would trade. This part does not have to be out-of-sample, you are just trying to calibrate your metric. Once you have done that, then the t-test is ((mean-criticallevel)/stdev)*sqrt(N).
Thanks, Howard On Sun, Oct 11, 2009 at 10:31 PM, Richard <[email protected]> wrote: > > > Hi Howard, > > I follow this discussion with interest and I thank you for your invaluable > comments. I am still confused about the statement .."look for systems that > have truly out-of-sample t-test scores of 2.0". > > In AB terms, what is the objective function we need to compare it to > "t-test"? Are we looking for CAR/ADD to be greater than 2.0? > > Kind Regards > Richard > > > >
