Hi All,
I'm looking for input on the calculation of the t-statistic of expectancy for
the OOS periods of the walkforward simulation for the purposes of validation
and control as put forward by Dr. Howard Bandy in his ATAA presention of
October 2009. Many thanks to Howard for making available this excellent
information.
At first glance I thought this was a pretty straightforward matter ... until I
got down to the nuts and bolts of the calculation. So let me explain what
I've done and the questions I'm having:
Assumptions:
-t-statistic of expectancy (for the null hypothesis Ho: expectancy <= 0 ) =
((mean)/(standard deviation))*sqrt(N-1) (1)
- use data for each OOS period generated by Amibroker (no CBT generated data)
Calculation Possibilities:
1.0 calculate the mean and standard deviation using the expectancy values for
each OOS period
2.0 calculate the weighted mean and weighted standard deviation of expectancy
where the weight for each OOS period is
wt = (number of trades for the OOS period / total trades for all OOS
periods) and where the weighted standard deviation
is defined here:
http://www.itl.nist.gov/div898/software/dataplot/refman2/ch2/weightsd.pdf
3.0 N = number of OOS periods
4.0 N = total trades for all OOS periods
Observations and Questions:
A) What is the correct combination of inputs from the possibilities above or
from possibilities I have failed to recognize that yields the valid t-statistic
?
B) comparing 1.0 and 2.0 above yeilds slightly different means in my case (
where the standard deviation in the average number of trades per OOS period is
small). 2.0 above also yeilds the same result as calculating expectancy by
the alternative method:
Expectancy ($) = (TotalProfit - TotalLoss) / NumberOfTrades = NetProfit /
NumberOfTrades over all OOS periods so my suspicion is that 2.0 above is the
way to go
C) with respect to 3.0 and 4.0 above I'm at a loss. I can see using the
number of OOS periods as there are that many observations of expectancy however
I can also see using total trades for all OOS periods since the results of all
trades ultimately generate the expetancies over all OOS periods.
Thanks for your input.
Ray
