hi, i've been researching the formula and found some sites:
http://tinyurl.com/yaspu7b http://tinyurl.com/ydymfcg now the question is what do you use for the "target" or cut off point? i will have to run some examples and compare with using stdev and see what happens to SQN. i've coded a SQN calc in the backtester, so it's probably easy to just replace the stdev calc with semi-dev calc once i'm confortable with the formula... nick On Thu, Oct 8, 2009 at 11:33 AM, Thomas Ludwig <[email protected]> wrote: > Nick, > > using semideviation seems to be a self-evident approach. Any idea how to > code > that in AB? > > Thanks > > Thomas > > On 08.10.2009, 18:01:36 NickW wrote: > > Hi, > > > > This is a great thread. Thanks for all the valuable information. > > > > Using SQN is something I've struggled with aswell where is punishes the > > outliers. When designing trend following systems, 5% of the trades are > the > > big winners that makes trend following work, so you really don't want to > > punish the system for having few great winners. I am going to look more > > into semideviation to see how that can solve that problem. > > > > Thanks > > Nick > > > > On Thu, Oct 8, 2009 at 7:50 AM, Howard B <[email protected]> wrote: > > > Hi Mike -- > > > > > > CAR is compound annual rate of return. Expectancy is the percentage > (or > > > dollar amount, but not for this discussion) gain on the average trade. > > > The final value of the trading account, Terminal Relative Wealth in > some > > > descriptions, is: (1 + expectancy) raised to the power of the number of > > > trades. When there are a few large trades included, this relationship > is > > > slightly different, but not enough to worry this discussion. CAR is: > (1 > > > + annual gain) raised to the power of the number of years. These are > > > both based on geometric means. Aren't they the same, or so close that > > > they act the same when used as an objective function? > > > > > > Van Tharp defines System Quality Number on page 28 of "Definitive Guide > > > to Position Sizing." SQN = (expectancy / standard deviation) times > > > squareroot of number of trades. SQN decreases when a median trade is > > > replaced by either a large win or a large loss. It is possible to > > > redefine the metric so that large wins are not penalized, such as by > > > using the semi-deviation instead of the standard deviation as the > > > denominator. But without making that change, outliers, both good and > > > bad, do affect SQN by reducing it. > > > > > > What happens with open (or potentially open) trades at the boundaries > of > > > walk forward periods is difficult to handle in both theory and > practice. > > > Tomasz' implementation is to close all open trades at the end of each > WF > > > period; and go into each WF period flat, not taking a new position > until > > > there is a new signal. This creates a potentially serious distortion > of > > > results when trades are typically held a long time (a large proportion > of > > > a WF period) or when there are a few large trades that comprise the > > > majority of a system's profit or loss. > > > > > > I think this leads to a conclusion that we both agree on -- high > quality > > > trading systems that can benefit from position sizing are based on high > > > frequency trading with very careful control over losses, and even > control > > > over gains. > > > > > > Thanks for listening, > > > Howard > > > > > > On Wed, Oct 7, 2009 at 2:36 PM, Mike <[email protected]> wrote: > > >> Howard, > > >> > > >> Assuming that SQN is the t-test for expectancy, then optimizing on the > > >> t-test of expectancy (i.e. SQN) is not the same as optimizing on CAR. > > >> > > >> The primary reason that CAR is a poor target for optimization is that > > >> outliers can significantly improve the calculation. The exact opposite > > >> is true for SQN. > > >> > > >> SQN rewards consistency and punishes outliers. Consistent winners with > a > > >> few large wins will improve CAR but hurt SQN, resulting in different > > >> parameter combinations being selected during an optimization. > > >> > > >> As for writing a custom method, AmiBroker's stats are calculated based > > >> on the assumption that all open trades are closed out at the backtest > > >> boundary date. Many open trades, or even just a few large open trades, > > >> can skew these values. > > >> > > >> For high frequency strategies or strategies using heavy position > sizing, > > >> creating a custom function is the only way to get reliable > measurements. > > >> > > >> Mike > > >> > > >> > > >> --- In [email protected] <amibroker%40yahoogroups.com>, > Howard B > > >> > > >> <howardba...@...> wrote: > > >> > Greetings all -- > > >> > > > >> > There has been a lot of activity on this thread. I'll not respond to > > >> > > >> each > > >> > > >> > point individually, but will make a couple of general comments. > > >> > > > >> > I know David Aronson, speak with him regularly, and collaborate with > > >> > him > > >> > > >> on > > >> > > >> > projects. I have a copy of his book, "Evidence-Based Technical > > >> > > >> Analysis." > > >> > > >> > His book is excellent and I highly recommend it. I think David and I > > >> > are > > >> > > >> in > > >> > > >> > pretty close agreement on most of the modeling, simulation, testing, > > >> > and validation issues. > > >> > > > >> > I have spoken with Robert Pardo and have exchanged several emails > and > > >> > > >> forum > > >> > > >> > postings with him. I think his earlier book was very good, > > >> > particularly > > >> > > >> at > > >> > > >> > the time it was published. And his more recent book is not quite up > to > > >> > those standards. There are several important areas he did not cover > > >> > and several areas where I see things considerably differently than > > >> > Robert. > > >> > > > >> > I have spoken with and exchanged emails with Van Tharp, and I have > > >> > > >> copies of > > >> > > >> > his books "Trade Your Way to Financial Freedom" and "Definitive > Guide > > >> > to Position Sizing." Both are excellent, and I recommend them both > > >> > highly. > > >> > > >> Be > > >> > > >> > sure to get the second edition of Trade Your Way to Financial > Freedom > > >> > -- > > >> > > >> it > > >> > > >> > has some important corrections and clarifications. > > >> > > > >> > Permit me a short rant on my soapbox. I really dislike it when > people > > >> > > >> claim > > >> > > >> > ownership of common terms. Tom DeMark, Robert Pardo, Van Tharp, and > > >> > > >> others > > >> > > >> > put Service Mark symbols on terms that they think are unique to > them, > > >> > > >> but > > >> > > >> > are not. I appreciate Tharp's enthusiasm over what he calls System > > >> > > >> Quality > > >> > > >> > Number, but I wish he would not put the Service Mark symbol next to > > >> > > >> every > > >> > > >> > occurrence of it. And trying to Service Mark the term Position > Sizing > > >> > is like a dietician service marking "calorie counting." Robert Pardo > > >> > claims "Walk Forward." I used exactly that term describing exactly > > >> > that process > > >> > > >> in > > >> > > >> > research papers I delivered at conferences in the late 1960s. The > mark > > >> > > >> has > > >> > > >> > been registered, not by Robert, but by a company I used to work for > > >> > and > > >> > > >> with > > >> > > >> > which Robert was not associated, over my strong objection. End of > > >> > rant. > > >> > > > >> > System quality number is equivalent to t-test. Systems with SQNs > above > > >> > 2 work well for exactly the same reasons that systems with t-test > > >> > scores > > >> > > >> above > > >> > > >> > 2 work well. In fact, it is possible to create a custom objective > > >> > > >> function > > >> > > >> > that Is the t-test and use it for optimization. Attendees at my > > >> > > >> workshops > > >> > > >> > in Melbourne later this month will see that demonstrated. Optimizing > > >> > for the t-test of expectancy is equivalent to optimizing for CAR, so > > >> > don't bother creating the custom function unless you have a better > > >> > candidate > > >> > > >> for > > >> > > >> > your objective function than CAR. > > >> > > > >> > Back to the topic at hand ----- > > >> > > > >> > There is No rule of thumb to determine how long the in-sample period > > >> > > >> should > > >> > > >> > be. The Only way to determine that is by testing the model and the > > >> > data together. And be prepared for that length to change over time. > > >> > Some writers suggest a relationship between the number of free > > >> > parameters and > > >> > > >> the > > >> > > >> > number of data points, or some proportional division of the > available > > >> > > >> data. > > >> > > >> > Those techniques do work on industrial time-series data which is > > >> > usually stationary, but they do not work on financial time-series > data > > >> > which is non-stationary and changes as trading systems become better > > >> > at > > >> > > >> extracting > > >> > > >> > inefficiencies from it. > > >> > > > >> > No matter how good the in-sample results look, no matter how high > the > > >> > > >> t-test > > >> > > >> > score is, no matter how many closed trades are represented -- > > >> > in-sample results have no value in estimating the future performance > > >> > of the > > >> > > >> system. > > >> > > >> > None. The only information you have that gives any indication of > > >> > future performance are the out-of-sample results from testing on > data > > >> > that was never used at all -- not even once -- during system > > >> > development. > > >> > > > >> > Tomorrow is out-of-sample. The only way to prepare for real-money > > >> > > >> trading > > >> > > >> > tomorrow is to be rigorous during the system testing and validation > > >> > process. Anything less will overestimate the probability of success. > > >> > > > >> > Thanks for listening, > > >> > Howard > > > > > ------------------------------------ > > **** IMPORTANT PLEASE READ **** > This group is for the discussion between users only. > This is *NOT* technical support channel. > > TO GET TECHNICAL SUPPORT send an e-mail directly to > SUPPORT {at} amibroker.com > > TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at > http://www.amibroker.com/feedback/ > (submissions sent via other channels won't be considered) > > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG: > http://www.amibroker.com/devlog/ > > Yahoo! Groups Links > > > >
