Ok, so the chi-squared tests for independence (real vs. expected) so, looking at your algorithm I'm not seeing expected. What exactly are you test significance against?
--- In [email protected], Thomas Ludwig <[EMAIL PROTECTED]> wrote: > > Hello, > > in "Quantitative Trading Systems" on p. 256, Howard describes a z- score > test in order to evaluate the statistical significance of a trading > system. While the formula is easy to write in AFL, I don't think that > it can be done as a custom metric since the system to be evaluated is > compared with a Random System. Any idea how to sensibly implement it in > Amibroker? > > I'm using another statistical test proposed by the late Arthur Merrill > some years ago in S&C. It's the "chi squared with one degree of > freedom, with the Yates correction". Here's how I implemented it in AB: > > //chi squared with one degree of freedom, with the Yates correction > wi=st.GetValue("WinnersQty"); > Lo=st.GetValue("LosersQty"); > Chi = (abs(wi-Lo)-1)^2/(wi+Lo); > bo.AddCustomMetric( "Chi-Squared modif.: >10.83: very > significant(1000:1), >6.64: significant (100:1) , >3.84: probably > significant (20:1), <3.84: significance doubtful", Chi ); > > While this metric doesn't tell you anything if your system is > profitable, it tells you if its signals are only pure coincidence > (simply put). It's remarkable that many systems that seem to be > promising according to the usual metrics, are below 3.84, i.e. > significance doubtful. You need either a rather high number of trades > or a very high percentage of winning trades to shift this metric > significantly higher. At least for (medium-term) EOD systems (that's > what I trade) this is not easy to achieve. > > What do you think about this metric? Are there other "better" > statistical metrics? If yes - would you mind sharing the AFL code? > > Best regards, > > Thomas >
