What I am headlining here is that looking back,at 9 years of data, the best MA system was 'trendfollowing' then, only 1.5 years later the best MA system was turned completely upside down into a 'reversion to mean' system ??????
Say, what? I don't have an explanation for any of this but it is too early anyway ... I need to make a lot more observations before it is time to hypothesis. --- In [email protected], "brian_z111" <brian_z...@...> wrote: > > This is the first time I have optimized or used any kind of synthetic data in > AB ... so far I haven't used any sophisticated methods to produce synthtetic > data either. > > I have only done a small amount of testing but I immediately found three > anomalies that might be worth further investigation. > I have aleady reported bcak on two: > > - why does an apparently worthless 'system' (plucked out of thin air unless > my subconscious mind intervened) outperfrom on approx 6% of stocks when those > stocks are assumed to be correlated to a fair extent ... chance? OR some > property of the data that correlation does not measure ... what property of > the data would favour that randomly selected 'system'? > > Note. if anything I expected the system to test the assumption that the MA is > the trend and I expected the system to 'fail'. > > > - why does the same system then outperform approx 50% of the time when tested > over randomly generated price series ... is it a coincidence that the > outperformance ratio, on random data, is close to the expected for > randomness? and why didn't the bull 'system' outperform only on the random > price series that outperformed? > > > The third anomaly is: > > - I optimized the following on some Yahoo ^DJI data ... 10 years EOD ... 9951 > quotes ... 2/01/1970 until June 4th 2009. > Default objective (fitness) function = CAR/MDD. > > fast = Optimize( "MA Fast", 1, 1, 30, 1 ); > slow = Optimize("MA Slow", 1, 1, 30, 1 ); > > Buy = Cross(MA(C,fast),MA(C,slow)); > Sell = Cross(MA(C,slow),MA(C,fast)); > > - when I optimized on the total range I found that the top values, were > inverted (as per Howard's examples in this forum and his books) but when I > left out the 2008/09 extreme market conditions I found this did not hold. > > Why does sucn a relatively small change in the test range make such a radical > difference in the outcomes? > > Here are some of the reported metrics from AB .. notice that they are similar > in some cases and markedly dissimilar in others. > > I am not sure if that leads to a question but it certainly gets my attention > considering that I am in the business of engineering reward/risk. > > Note that I am using ProfitFactor because it is typical in the industry but > it has some question marks over whether is it the best CoreMetric to use (I > am investgating PowerFactor and assymetricalPayoffRatio which might be more > apt ... I hope to post more on these metrics later). > > Opt1: > > using all data > > top model = CAR/MDD == 0.25 AND periods == fast 10, slow 7; > > NETT PROFIT 1749% > Exposure 44.9; > CAR 7.68; > RAR 17.07 > MAXDD 31.57 > RECOVERYFACTOR 2.64 > PF 1.62 (WIN 68% * PR 0.75) > #TRADES 588 > > > Opt2: > > using data range from 2/01/1970 to 31/12/2007 (that's Dec for the benefit of > timezoners). > > top model = CAR/MDD == 0.42 AND periods == fast 1, slow 6; > > NETT PROFIT 1921% > Exposure 54.51; > CAR 8.23; > RAR 15.09 > MAXDD 21.67 > RECOVERYFACTOR 5.07 > PF 1.35 (WIN 40% * PR 2.03) > #TRADES 1049 > > > I hope I reported the metrics correctly but anyone can replicate my tests and > report otherwise. > > I think it also demonstrates that if PoF (PowerFactor) is a better CoreMetric > than ProfitFactor it will need to be standardized on a returns/time basis > (choose your time period = the basetimeframe you trade ... PoF is related to > GeoMean per bar?) > > --- In [email protected], "brian_z111" <brian_z111@> wrote: > > > > Following recent discussions on benchmarking and using rule based systems > > to engineer returns to meet 'clients' profiles i.e.Samantha's MA(C,10) > > example, I did some follow up R&D with the intent of expanding the > > examination a little further via a zboard post. > > > > I may, or may not, get around to that so in the meantime I decided I would > > share a couple of things while they are still topical. > > > > I made up some quick and dirty randomly generated eq curves so that I could > > optimise MA(C,10) on them (out of curiosity). > > > > Also, out of curiosity, I decided to see how the example signal/filter code > > that I made up, as the study piece for Yofas topic on benchmarking, would > > actually perform. > > > > Buy = Ref(ROC(MA(C,1),1),-1) < 0 AND ROC(MA(C,1),1) > 0 AND ROC(MA(C,10),1) > > > 0; > > Sell = Cross(MA(C,10),C);//no thought went into this exit and I haven't > > tried any optimization of the entry or the exit > > > > By chance I noticed that it outperformed on one or two of the constituents > > of the ^DJI (Yahoo data ... 2005 to 2009) and to the naked eye the > > constituents all seem to be correlated to a fair extent over that time > > range. > > > > Also, to the naked eye, it outperforms on randomly generated stock prices > > around 50% of the time and the outperformnce doesn't appear to be > > correlated to the underlying(I haven't attempted to find an explanation for > > this). > > > > Here is the code I used to make up some randomly generated 'stocks'. > > > > As we would expect it produces, say, 100 price series with a concatenated > > mean of around zero (W/L = 1 and PayoffRatio == 1) etc. > > When plotted at the same time ... individual price series are dispersed > > around the mean in a 'probability cone' ... in this case it is a relatively > > tight cone because the method doesn't introduce a lot of volatility to the > > series. > > > > /*P_RandomEquity*/ > > > > //Use as a Scan to create PseudoRandom Equity curves > > //Current symbol, All quotations in AA, select basetimeframe in AA Settings > > //It will also create the curves if used as an indicator (add the > > appropriate flag to ATC) > > // but this is NOT recommended as it will recalculate them on every refresh. > > //Indicator mode is good for viewing recalculated curves (click in > > whitespace) > > //CommentOut the Scan code before using the indicator code. > > //Don't use a very large N or it will freeze up indicator scrolling etc > > > > n = 100;//manually input desired number - used in Scan AND Indicator mode > > > > ///SCAN/////////////////////////////////////////////////////////////////// > > > > > > Buy=Sell=0; > > > > for( i = 1; i < n; i++ ) > > > > { > > > > VarSet( "D"+i, 100 * exp( Cum(log(1 + (Random() - 0.5)/100)) ) ); > > AddToComposite(VarGet( "D"+i ),"~Random" + i,"X",1|2|128); > > //Plot( VarGet( "D"+i ), "D"+i, 1,1 ); > > //PlotForeign("~Random" + i,"Random" + 1,1,1); > > } > > > > /* > > ////PLOT///////////////////////////////////////////////////// > > > > //use the same number setting as for the Scan > > > > > > for( i = 1; i < n; i++ ) > > > > { > > > > PlotForeign("~Random" + i,"Random" + i,1,1); > > > > } > > > > > > ////OPTIMIZE/////////////////////////////////////////////////////////// > > > > //use the filter to run on Group253 OR add ~Random + i PseudoTickers to a > > Watchlist and define by AA filter > > > > > > //fast = Optimize( "MA Fast", 1, 1, 10, 1 ); > > //slow = Optimize("MA Slow", 4, 4, 20, 1 ); > > > > //PositionSize = -100/P; > > //Buy = Cross(MA(C,fast),MA(C,slow)); > > //Sell = Cross(MA(C,slow),MA(C,fast)); > > > > //Short = Sell; > > //Cover = Buy; > > > > I also stumbled on this, which seems to have some relevance: > > > > http://www.scribd.com/doc/6737301/Trading-eBookCan-Technical-Analysis-Still-Beat-Random-Systems > > > > > > It contains a link to a site that has a free download of some RNG produced > > datasets. > > > > There hasn't been much discussion on using synthetic data in the forum ... > > Patrick recommended it for testing? OR benchmarking? ... Fred is against > > using it ("If we knew enough about the characteristics of the data, in the > > first place, to be able to create synthetic data then we would know enough > > to design trading systems to exploit the data's profile anyway", OR > > something like that). > > > > I was titillated enough by my first excursion into benchmarking with > > synthetic data to bring me back for some more. > > >
