Hi Howard, I'm a bit confused by your remarks.
1. You have said that Definitive Guide To Positoin Sizing is an excellent book. 2. You have said that SQN is a t-test (of expectancy?). 3. You have said that a t-test of expectancy is functionally equivalent to CAR. 4. You have said that CAR is a poor fitness function. Given that the book repeatedly promotes usage of SQN as a comparator (i.e. a fitness function) with which to measure two systems, how do you reconcile point 1 with points 2-4? Just to pin it down without any indirection, are you saying that you consider SQN as poor a fitness function as CAR? Or, have I misunderstood something you've said? Thanks, Mike --- In [email protected], Howard B <howardba...@...> wrote: > > Hi Mike -- > > CAR is compound annual rate of return. Expectancy is the percentage (or > dollar amount, but not for this discussion) gain on the average trade. The > final value of the trading account, Terminal Relative Wealth in some > descriptions, is: (1 + expectancy) raised to the power of the number of > trades. When there are a few large trades included, this relationship is > slightly different, but not enough to worry this discussion. CAR is: (1 + > annual gain) raised to the power of the number of years. These are both > based on geometric means. Aren't they the same, or so close that they act > the same when used as an objective function? > > Van Tharp defines System Quality Number on page 28 of "Definitive Guide to > Position Sizing." SQN = (expectancy / standard deviation) times squareroot > of number of trades. SQN decreases when a median trade is replaced by > either a large win or a large loss. It is possible to redefine the metric > so that large wins are not penalized, such as by using the semi-deviation > instead of the standard deviation as the denominator. But without making > that change, outliers, both good and bad, do affect SQN by reducing it. > > What happens with open (or potentially open) trades at the boundaries of > walk forward periods is difficult to handle in both theory and practice. > Tomasz' implementation is to close all open trades at the end of each WF > period; and go into each WF period flat, not taking a new position until > there is a new signal. This creates a potentially serious distortion of > results when trades are typically held a long time (a large proportion of a > WF period) or when there are a few large trades that comprise the majority > of a system's profit or loss. > > I think this leads to a conclusion that we both agree on -- high quality > trading systems that can benefit from position sizing are based on high > frequency trading with very careful control over losses, and even control > over gains. > > Thanks for listening, > Howard > > > On Wed, Oct 7, 2009 at 2:36 PM, Mike <sfclimb...@...> wrote: > > > > > > > Howard, > > > > Assuming that SQN is the t-test for expectancy, then optimizing on the > > t-test of expectancy (i.e. SQN) is not the same as optimizing on CAR. > > > > The primary reason that CAR is a poor target for optimization is that > > outliers can significantly improve the calculation. The exact opposite is > > true for SQN. > > > > SQN rewards consistency and punishes outliers. Consistent winners with a > > few large wins will improve CAR but hurt SQN, resulting in different > > parameter combinations being selected during an optimization. > > > > As for writing a custom method, AmiBroker's stats are calculated based on > > the assumption that all open trades are closed out at the backtest boundary > > date. Many open trades, or even just a few large open trades, can skew these > > values. > > > > For high frequency strategies or strategies using heavy position sizing, > > creating a custom function is the only way to get reliable measurements. > > > > Mike > > > > > > --- In [email protected] <amibroker%40yahoogroups.com>, Howard B > > <howardbandy@> wrote: > > > > > > Greetings all -- > > > > > > There has been a lot of activity on this thread. I'll not respond to each > > > point individually, but will make a couple of general comments. > > > > > > I know David Aronson, speak with him regularly, and collaborate with him > > on > > > projects. I have a copy of his book, "Evidence-Based Technical Analysis." > > > His book is excellent and I highly recommend it. I think David and I are > > in > > > pretty close agreement on most of the modeling, simulation, testing, and > > > validation issues. > > > > > > I have spoken with Robert Pardo and have exchanged several emails and > > forum > > > postings with him. I think his earlier book was very good, particularly > > at > > > the time it was published. And his more recent book is not quite up to > > > those standards. There are several important areas he did not cover and > > > several areas where I see things considerably differently than Robert. > > > > > > I have spoken with and exchanged emails with Van Tharp, and I have copies > > of > > > his books "Trade Your Way to Financial Freedom" and "Definitive Guide to > > > Position Sizing." Both are excellent, and I recommend them both highly. > > Be > > > sure to get the second edition of Trade Your Way to Financial Freedom -- > > it > > > has some important corrections and clarifications. > > > > > > Permit me a short rant on my soapbox. I really dislike it when people > > claim > > > ownership of common terms. Tom DeMark, Robert Pardo, Van Tharp, and > > others > > > put Service Mark symbols on terms that they think are unique to them, but > > > are not. I appreciate Tharp's enthusiasm over what he calls System > > Quality > > > Number, but I wish he would not put the Service Mark symbol next to every > > > occurrence of it. And trying to Service Mark the term Position Sizing is > > > like a dietician service marking "calorie counting." Robert Pardo claims > > > "Walk Forward." I used exactly that term describing exactly that process > > in > > > research papers I delivered at conferences in the late 1960s. The mark > > has > > > been registered, not by Robert, but by a company I used to work for and > > with > > > which Robert was not associated, over my strong objection. End of rant. > > > > > > System quality number is equivalent to t-test. Systems with SQNs above 2 > > > work well for exactly the same reasons that systems with t-test scores > > above > > > 2 work well. In fact, it is possible to create a custom objective > > function > > > that Is the t-test and use it for optimization. Attendees at my workshops > > > in Melbourne later this month will see that demonstrated. Optimizing for > > > the t-test of expectancy is equivalent to optimizing for CAR, so don't > > > bother creating the custom function unless you have a better candidate > > for > > > your objective function than CAR. > > > > > > Back to the topic at hand ----- > > > > > > There is No rule of thumb to determine how long the in-sample period > > should > > > be. The Only way to determine that is by testing the model and the data > > > together. And be prepared for that length to change over time. Some > > > writers suggest a relationship between the number of free parameters and > > the > > > number of data points, or some proportional division of the available > > data. > > > Those techniques do work on industrial time-series data which is usually > > > stationary, but they do not work on financial time-series data which is > > > non-stationary and changes as trading systems become better at extracting > > > inefficiencies from it. > > > > > > No matter how good the in-sample results look, no matter how high the > > t-test > > > score is, no matter how many closed trades are represented -- in-sample > > > results have no value in estimating the future performance of the system. > > > None. The only information you have that gives any indication of future > > > performance are the out-of-sample results from testing on data that was > > > never used at all -- not even once -- during system development. > > > > > > Tomorrow is out-of-sample. The only way to prepare for real-money trading > > > tomorrow is to be rigorous during the system testing and validation > > > process. Anything less will overestimate the probability of success. > > > > > > Thanks for listening, > > > Howard > > > > > > > > > >
