> There are methods (windowing+padding) to increase FFT resolution on >short data samples, > and there are also methods different than FFT for spectal analysis. > But that is rather broad subject ....
I have my own simple and direct methods of dealing with waves, so I haven't felt the need to go down those paths, but, in the interests of 'breadth of knowledge' I made a mental note to have a closer look at spectral analysis and ElliotWaves (based on feedback from the forum). Thanks for your comments - always appreciated. brian_z --- In [email protected], "Tomasz Janeczko" <[EMAIL PROTECTED]> wrote: > > > Some of the 'academic' stats methods are not quite as good a fit to > > trading as we first assume e.g. look at the recent discussion on > > Fourier Transform, which comes from Signal Processing (electronics?) > > and falls down rather spectacularly in trading because stockmarket > > data isn't stationary. > > For what is worth, FFT is most often applied to audio signals, that > are definitelly NOT stationary :-) so that argument is not really valid :-) > There are methods (windowing+padding) to increase FFT resolution on short data samples, > and there are also methods different than FFT for spectal analysis. > But that is rather broad subject .... > > Best regards, > Tomasz Janeczko > amibroker.com > ----- Original Message ----- > From: "brian_z111" <[EMAIL PROTECTED]> > To: <[email protected]> > Sent: Wednesday, March 12, 2008 12:41 AM > Subject: [amibroker] Re: Statistical tests as custom metrics > > > > Thomas, > > > > I have included Jeffs question with your opening query since they fit > > so closely. > > > >> looking at your algorithm I'm not seeing expected. What exactly > >>are > >> you test significance against? > > > > The expected value of an unbiased binomial event is 0.50 (50% win). > > > > Since the (equity) market has a slight upward bias we need to use an > > expected value, that includes the market bias, as the base line (or > > detrend as per Howards/Aronsons methods). > > > > (Personally I don't detrend as I like to condition myself to see the > > markets as they actually are, and mentally factor in the trend, so I > > use market bias benchmarking). > > > >> > in "Quantitative Trading Systems" on p. 256, Howard describes a z- > >> score > >> > test in order to evaluate the statistical significance of a > > trading > >> > system. While the formula is easy to write in AFL, I don't think > >> that > >> > it can be done as a custom metric since the system to be > > evaluated > >> is > >> > compared with a Random System. Any idea how to sensibly implement > >> it in > >> > Amibroker? > > > > At page 91,in his book, (Entries and Exits chapter) Howard gives some > > very good (random entry) examples of how we can get an estimation for > > the 'standardized binomial expectancy' of any market i.e. you can get > > the mean expected wins for the actual market you are testing your > > system in and use that in the Z score calculation - I think you would > > be better to use random entries with an exit after a set number of > > days == the average time your system trades are in the market. > > > > I am still learning AB myself so I am not sure if we can implement > > Howards Z equation directly in AB - you will probably have to do it > > outside somewhere - I haven't figured out how we can get the SD of a > > trade series (from a backtest) in AB - anyway we don't have built in > > stats tables (I suppose you could manually plug in the typical Z > > scores). > > > > I'm exporting to Excel and doing my evaluations there, but I don't > > get that fancy. > > > > > > I'm using another statistical test proposed by the late Arthur > >> Merrill > >> > some years ago in S&C. It's the "chi squared with one degree of > >> > freedom, with the Yates correction". Here's how I implemented it > > in > >> AB: > >> > > >> > //chi squared with one degree of freedom, with the Yates > > correction > >> > wi=st.GetValue("WinnersQty"); > >> > Lo=st.GetValue("LosersQty"); > >> > Chi = (abs(wi-Lo)-1)^2/(wi+Lo); > >> > bo.AddCustomMetric( "Chi-Squared modif.: >10.83: very > >> > significant(1000:1), >6.64: significant (100:1) , >3.84: probably > >> > significant (20:1), <3.84: significance doubtful", Chi ); > >> > > >> > What do you think about this metric? > > > > I think it is a very conservative measure. > > > > One of the problems we have, in evaluation, is that 'academic' > > statistics filtered into freelance trading via institutional > > investing - nothing against academics or institutional traders but > > their focus is somewhat different to freelance traders. > > > > Some of the 'academic' stats methods are not quite as good a fit to > > trading as we first assume e.g. look at the recent discussion on > > Fourier Transform, which comes from Signal Processing (electronics?) > > and falls down rather spectacularly in trading because stockmarket > > data isn't stationary. > > > > Most of the stats we are using assume stationarity and also assume > > that data will be normal/ random i.e. it will have a normal > > distribution and that the datapoints are independent of each other. > > > > Neither is absolutely true, so the stats we are using are > > approximations (of course the data we are using is only an > > approximation anyway) - hence the doubts about Merrill's Chi. > > > >> > While this metric doesn't tell you anything if your system is > >> > profitable, it tells you if its signals are only pure coincidence > >> > (simply put). It's remarkable that many systems that seem to be > >> > promising according to the usual metrics, are below 3.84, i.e. > >> > significance doubtful. You need either a rather high number of > >> trades > >> > or a very high percentage of winning trades to shift this metric > >> > significantly higher. At least for (medium-term) EOD systems > >> (that's > >> > what I trade) this is not easy to achieve. > >> > > > > > Yes, it is very hard to find good trading systems. > > > > This is what I have found - many tests that come up with nothing, > > especially in the first two years. > > > >>>Are there other "better" > >> > statistical metrics? If yes - would you mind sharing the AFL code? > >> > > > > > Try Howards Z method, using his random code, to find your expected > > win rate for your market and see how that works out. > > > > I have started some original (to me) work, based on binomial > > simulation of equity curves and the behaviour of random, 50/50, > > trading systems. > > > > It is only at the experimental, concept stage. > > I intend posting it to the UKB one day so that the mathematically > > trained people in the forum can critique it (it might be a load of > > old rubbish for all I know). > > > > Based on that work I am using PowerFactor, with sample error, to > > guestimate significance (I can quickly do that in my head). > > > > Note that in PowerFactor the binomial component is considered to be > > Gaussian, with independent variables, while the distribution of the > > trades (ave%won/ave%lost) is not. > > > > Because of that I only apply the significance test to the W/L > > binomial component (I claim that carrying out stats analysis on the > > compound system results is biased because of the non-normal nature of > > the distribution etc). > > > > For binomial events: > > > > variance == sample error (sort of) > > > > For 100 trades: > > > > sample error = +_10%; > > expected random result (benchmark) == 50 wins; > > > > a no win trade will have a range of: > > > > 45-50 wins (one standard dev) > > 40-60 wins (two standard devs) > > 35- 65 (three standard devs) etc > > > > So for 100 trades 60 wins doesn't happen all that often, if the coin > > is a fair coin (a random event). > > > > We are defintely going to take notice of 60/100 wins BUT it is > > not 'out of this world' and we do not have certainty - we only have > > the expectation that it is good - reality can, and does, dash > > expectations on occasion. > > > > Because of this, W/L results are never a sure thing. > > > > To ensure against this take control of the ave%W/ave%L ratio - that > > is something we can control via stops - if the W/L ratio turns out to > > be a 'BlackSwan' our good stops will save us from crashing and > > burning (keep us at low drawdowns). > > > > Comparing to Chi (for 100 trades with a 60% win record): > > > > Chi = (abs(wi-Lo)-1)^2/(wi+Lo); > > > > == ((60-40)-1)^2/(wi+Lo); > > == 19^2/100; > > == 361/100 > > == 3.6 > > == Not significant according to Chi but significant according to > > brian (always look on the bright side of life!). > > > > I agree that finding 60% winners, in any market or timeframe, is very > > difficult - that is the reality of trading. > > > > This problem is especially prevalent in mid - long term trading - say > > indicators with long lookbacks are used - then the number of signals > > available tends towards becoming a rare event and the trader then can > > only see a small part of the longterm (10000 plus) trades - the > > trader soon runs out of clean data and can't get high enough trade > > counts. > > > > That is why I like shorter term trading (intraday to 2-3 day cycles) > > where I can take advantage of statistical smoothing (I quickly > > approach my theoretical edge i.e. relative to the calendar days). > > > > > > As I said - please use 'my' theories at your own risk, at least until > > after I post on the topic, and the mathematicians in the forum have a > > chance to bash up my hypotheses. > > > > > > brian_z > > > > > > > > > > > > > > --- In [email protected], "jeffro861" <jeffro861@> wrote: > >> > >> Ok, so the chi-squared tests for independence (real vs. expected) > > so, > >> looking at your algorithm I'm not seeing expected. What exactly > > are > >> you test significance against? > >> > >> --- In [email protected], Thomas Ludwig <Thomas.Ludwig@> > >> wrote: > >> > > >> > Hello, > >> > > >> > in "Quantitative Trading Systems" on p. 256, Howard describes a z- > >> score > >> > test in order to evaluate the statistical significance of a > > trading > >> > system. While the formula is easy to write in AFL, I don't think > >> that > >> > it can be done as a custom metric since the system to be > > evaluated > >> is > >> > compared with a Random System. Any idea how to sensibly implement > >> it in > >> > Amibroker? > >> > > >> > I'm using another statistical test proposed by the late Arthur > >> Merrill > >> > some years ago in S&C. It's the "chi squared with one degree of > >> > freedom, with the Yates correction". Here's how I implemented it > > in > >> AB: > >> > > >> > //chi squared with one degree of freedom, with the Yates > > correction > >> > wi=st.GetValue("WinnersQty"); > >> > Lo=st.GetValue("LosersQty"); > >> > Chi = (abs(wi-Lo)-1)^2/(wi+Lo); > >> > bo.AddCustomMetric( "Chi-Squared modif.: >10.83: very > >> > significant(1000:1), >6.64: significant (100:1) , >3.84: probably > >> > significant (20:1), <3.84: significance doubtful", Chi ); > >> > > >> > While this metric doesn't tell you anything if your system is > >> > profitable, it tells you if its signals are only pure coincidence > >> > (simply put). It's remarkable that many systems that seem to be > >> > promising according to the usual metrics, are below 3.84, i.e. > >> > significance doubtful. You need either a rather high number of > >> trades > >> > or a very high percentage of winning trades to shift this metric > >> > significantly higher. At least for (medium-term) EOD systems > >> (that's > >> > what I trade) this is not easy to achieve. > >> > > >> > What do you think about this metric? Are there other "better" > >> > statistical metrics? If yes - would you mind sharing the AFL code? > >> > > >> > Best regards, > >> > > >> > Thomas > >> > > >> > > > > > > > > > > Please note that this group is for discussion between users only. > > > > To get support from AmiBroker please send an e-mail directly to > > SUPPORT {at} amibroker.com > > > > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG: > > http://www.amibroker.com/devlog/ > > > > For other support material please check also: > > http://www.amibroker.com/support.html > > > > Yahoo! Groups Links > > > > > > >
