So, in my first ever attempt at optimization I am presented with a conundrum.
If I opt MACrossver(C,X), and look down the list of top candidates (using the AB default objective function), I see that historically this system was both a 'trend following system' and a 'reversion to mean' system. If I could travel back in time, armed with this info, should I trade MA crosses as a 'trend follower', 'reversion to mean' or both? OR ...on the other hand am I failing to interpret the results correctly ... is there something about opt that I don't understand ... if I develop my opt skills will this help me solve this riddle? OR ... is optimization itself somehow not providing me with a clear understanding of how I should have followed the markets (for that historical period)? OR ... is it something to do with MACrossovers ... perhaps other systems are more amenable to optimization ... if so how can I filter systems in advance to save me wasting my time opting non-compliant systems? OR .... is the objective function the underlying cause of this dilemna? OR ... I have done something wrong .. failed to optimize correctly or used AB incorrectly? OR ... is it the data... is there something wrong with the Yahoo data I used? --- In [email protected], "brian_z111" <brian_z...@...> wrote: > > What I am headlining here is that looking back,at 9 years of data, the best > MA system was 'trendfollowing' then, only 1.5 years later the best MA system > was turned completely upside down into a 'reversion to mean' system ?????? > > Say, what? > > I don't have an explanation for any of this but it is too early anyway ... I > need to make a lot more observations before it is time to hypothesis. > > --- In [email protected], "brian_z111" <brian_z111@> wrote: > > > > This is the first time I have optimized or used any kind of synthetic data > > in AB ... so far I haven't used any sophisticated methods to produce > > synthtetic data either. > > > > I have only done a small amount of testing but I immediately found three > > anomalies that might be worth further investigation. > > I have aleady reported bcak on two: > > > > - why does an apparently worthless 'system' (plucked out of thin air unless > > my subconscious mind intervened) outperfrom on approx 6% of stocks when > > those stocks are assumed to be correlated to a fair extent ... chance? OR > > some property of the data that correlation does not measure ... what > > property of the data would favour that randomly selected 'system'? > > > > Note. if anything I expected the system to test the assumption that the MA > > is the trend and I expected the system to 'fail'. > > > > > > - why does the same system then outperform approx 50% of the time when > > tested over randomly generated price series ... is it a coincidence that > > the outperformance ratio, on random data, is close to the expected for > > randomness? and why didn't the bull 'system' outperform only on the random > > price series that outperformed? > > > > > > The third anomaly is: > > > > - I optimized the following on some Yahoo ^DJI data ... 10 years EOD ... > > 9951 quotes ... 2/01/1970 until June 4th 2009. > > Default objective (fitness) function = CAR/MDD. > > > > fast = Optimize( "MA Fast", 1, 1, 30, 1 ); > > slow = Optimize("MA Slow", 1, 1, 30, 1 ); > > > > Buy = Cross(MA(C,fast),MA(C,slow)); > > Sell = Cross(MA(C,slow),MA(C,fast)); > > > > - when I optimized on the total range I found that the top values, were > > inverted (as per Howard's examples in this forum and his books) but when I > > left out the 2008/09 extreme market conditions I found this did not hold. > > > > Why does sucn a relatively small change in the test range make such a > > radical difference in the outcomes? > > > > Here are some of the reported metrics from AB .. notice that they are > > similar in some cases and markedly dissimilar in others. > > > > I am not sure if that leads to a question but it certainly gets my > > attention considering that I am in the business of engineering reward/risk. > > > > Note that I am using ProfitFactor because it is typical in the industry but > > it has some question marks over whether is it the best CoreMetric to use (I > > am investgating PowerFactor and assymetricalPayoffRatio which might be more > > apt ... I hope to post more on these metrics later). > > > > Opt1: > > > > using all data > > > > top model = CAR/MDD == 0.25 AND periods == fast 10, slow 7; > > > > NETT PROFIT 1749% > > Exposure 44.9; > > CAR 7.68; > > RAR 17.07 > > MAXDD 31.57 > > RECOVERYFACTOR 2.64 > > PF 1.62 (WIN 68% * PR 0.75) > > #TRADES 588 > > > > > > Opt2: > > > > using data range from 2/01/1970 to 31/12/2007 (that's Dec for the benefit > > of timezoners). > > > > top model = CAR/MDD == 0.42 AND periods == fast 1, slow 6; > > > > NETT PROFIT 1921% > > Exposure 54.51; > > CAR 8.23; > > RAR 15.09 > > MAXDD 21.67 > > RECOVERYFACTOR 5.07 > > PF 1.35 (WIN 40% * PR 2.03) > > #TRADES 1049 > > > > > > I hope I reported the metrics correctly but anyone can replicate my tests > > and report otherwise. > > > > I think it also demonstrates that if PoF (PowerFactor) is a better > > CoreMetric than ProfitFactor it will need to be standardized on a > > returns/time basis (choose your time period = the basetimeframe you trade > > ... PoF is related to GeoMean per bar?) > > > > --- In [email protected], "brian_z111" <brian_z111@> wrote: > > > > > > Following recent discussions on benchmarking and using rule based systems > > > to engineer returns to meet 'clients' profiles i.e.Samantha's MA(C,10) > > > example, I did some follow up R&D with the intent of expanding the > > > examination a little further via a zboard post. > > > > > > I may, or may not, get around to that so in the meantime I decided I > > > would share a couple of things while they are still topical. > > > > > > I made up some quick and dirty randomly generated eq curves so that I > > > could optimise MA(C,10) on them (out of curiosity). > > > > > > Also, out of curiosity, I decided to see how the example signal/filter > > > code that I made up, as the study piece for Yofas topic on benchmarking, > > > would actually perform. > > > > > > Buy = Ref(ROC(MA(C,1),1),-1) < 0 AND ROC(MA(C,1),1) > 0 AND > > > ROC(MA(C,10),1) > 0; > > > Sell = Cross(MA(C,10),C);//no thought went into this exit and I haven't > > > tried any optimization of the entry or the exit > > > > > > By chance I noticed that it outperformed on one or two of the > > > constituents of the ^DJI (Yahoo data ... 2005 to 2009) and to the naked > > > eye the constituents all seem to be correlated to a fair extent over that > > > time range. > > > > > > Also, to the naked eye, it outperforms on randomly generated stock prices > > > around 50% of the time and the outperformnce doesn't appear to be > > > correlated to the underlying(I haven't attempted to find an explanation > > > for this). > > > > > > Here is the code I used to make up some randomly generated 'stocks'. > > > > > > As we would expect it produces, say, 100 price series with a concatenated > > > mean of around zero (W/L = 1 and PayoffRatio == 1) etc. > > > When plotted at the same time ... individual price series are dispersed > > > around the mean in a 'probability cone' ... in this case it is a > > > relatively tight cone because the method doesn't introduce a lot of > > > volatility to the series. > > > > > > /*P_RandomEquity*/ > > > > > > //Use as a Scan to create PseudoRandom Equity curves > > > //Current symbol, All quotations in AA, select basetimeframe in AA > > > Settings > > > //It will also create the curves if used as an indicator (add the > > > appropriate flag to ATC) > > > // but this is NOT recommended as it will recalculate them on every > > > refresh. > > > //Indicator mode is good for viewing recalculated curves (click in > > > whitespace) > > > //CommentOut the Scan code before using the indicator code. > > > //Don't use a very large N or it will freeze up indicator scrolling etc > > > > > > n = 100;//manually input desired number - used in Scan AND Indicator mode > > > > > > ///SCAN/////////////////////////////////////////////////////////////////// > > > > > > > > > Buy=Sell=0; > > > > > > for( i = 1; i < n; i++ ) > > > > > > { > > > > > > VarSet( "D"+i, 100 * exp( Cum(log(1 + (Random() - 0.5)/100)) ) ); > > > AddToComposite(VarGet( "D"+i ),"~Random" + i,"X",1|2|128); > > > //Plot( VarGet( "D"+i ), "D"+i, 1,1 ); > > > //PlotForeign("~Random" + i,"Random" + 1,1,1); > > > } > > > > > > /* > > > ////PLOT///////////////////////////////////////////////////// > > > > > > //use the same number setting as for the Scan > > > > > > > > > for( i = 1; i < n; i++ ) > > > > > > { > > > > > > PlotForeign("~Random" + i,"Random" + i,1,1); > > > > > > } > > > > > > > > > ////OPTIMIZE/////////////////////////////////////////////////////////// > > > > > > //use the filter to run on Group253 OR add ~Random + i PseudoTickers to a > > > Watchlist and define by AA filter > > > > > > > > > //fast = Optimize( "MA Fast", 1, 1, 10, 1 ); > > > //slow = Optimize("MA Slow", 4, 4, 20, 1 ); > > > > > > //PositionSize = -100/P; > > > //Buy = Cross(MA(C,fast),MA(C,slow)); > > > //Sell = Cross(MA(C,slow),MA(C,fast)); > > > > > > //Short = Sell; > > > //Cover = Buy; > > > > > > I also stumbled on this, which seems to have some relevance: > > > > > > http://www.scribd.com/doc/6737301/Trading-eBookCan-Technical-Analysis-Still-Beat-Random-Systems > > > > > > > > > It contains a link to a site that has a free download of some RNG > > > produced datasets. > > > > > > There hasn't been much discussion on using synthetic data in the forum > > > ... Patrick recommended it for testing? OR benchmarking? ... Fred is > > > against using it ("If we knew enough about the characteristics of the > > > data, in the first place, to be able to create synthetic data then we > > > would know enough to design trading systems to exploit the data's profile > > > anyway", OR something like that). > > > > > > I was titillated enough by my first excursion into benchmarking with > > > synthetic data to bring me back for some more. > > > > > >
