OR ... is opt correctly flagging something about market behaviour or system trading that I don't understand?
OR ... all of the above, none of the above, a combination of some of the above? Also, the MACrossover changed from TrendFollowing to MeanReversion when I added 1.5 years to a 30 year lookback and not a 10 year lookback ... not a 9 year lookback as per my previous post. --- In [email protected], "brian_z111" <brian_z...@...> wrote: > > So, in my first ever attempt at optimization I am presented with a conundrum. > > If I opt MACrossver(C,X), and look down the list of top candidates (using the > AB default objective function), I see that historically this system was both > a 'trend following system' and a 'reversion to mean' system. > > If I could travel back in time, armed with this info, should I trade MA > crosses as a 'trend follower', 'reversion to mean' or both? > > OR > > ...on the other hand am I failing to interpret the results correctly ... is > there something about opt that I don't understand ... if I develop my opt > skills will this help me solve this riddle? > > OR > > ... is optimization itself somehow not providing me with a clear > understanding of how I should have followed the markets (for that historical > period)? > > OR > > ... is it something to do with MACrossovers ... perhaps other systems are > more amenable to optimization ... if so how can I filter systems in advance > to save me wasting my time opting non-compliant systems? > > OR > > .... is the objective function the underlying cause of this dilemna? > > OR > > ... I have done something wrong .. failed to optimize correctly or used AB > incorrectly? > > OR > > > ... is it the data... is there something wrong with the Yahoo data I used? > > --- In [email protected], "brian_z111" <brian_z111@> wrote: > > > > What I am headlining here is that looking back,at 9 years of data, the best > > MA system was 'trendfollowing' then, only 1.5 years later the best MA > > system was turned completely upside down into a 'reversion to mean' system > > ?????? > > > > Say, what? > > > > I don't have an explanation for any of this but it is too early anyway ... > > I need to make a lot more observations before it is time to hypothesis. > > > > --- In [email protected], "brian_z111" <brian_z111@> wrote: > > > > > > This is the first time I have optimized or used any kind of synthetic > > > data in AB ... so far I haven't used any sophisticated methods to produce > > > synthtetic data either. > > > > > > I have only done a small amount of testing but I immediately found three > > > anomalies that might be worth further investigation. > > > I have aleady reported bcak on two: > > > > > > - why does an apparently worthless 'system' (plucked out of thin air > > > unless my subconscious mind intervened) outperfrom on approx 6% of stocks > > > when those stocks are assumed to be correlated to a fair extent ... > > > chance? OR some property of the data that correlation does not measure > > > ... what property of the data would favour that randomly selected > > > 'system'? > > > > > > Note. if anything I expected the system to test the assumption that the > > > MA is the trend and I expected the system to 'fail'. > > > > > > > > > - why does the same system then outperform approx 50% of the time when > > > tested over randomly generated price series ... is it a coincidence that > > > the outperformance ratio, on random data, is close to the expected for > > > randomness? and why didn't the bull 'system' outperform only on the > > > random price series that outperformed? > > > > > > > > > The third anomaly is: > > > > > > - I optimized the following on some Yahoo ^DJI data ... 10 years EOD ... > > > 9951 quotes ... 2/01/1970 until June 4th 2009. > > > Default objective (fitness) function = CAR/MDD. > > > > > > fast = Optimize( "MA Fast", 1, 1, 30, 1 ); > > > slow = Optimize("MA Slow", 1, 1, 30, 1 ); > > > > > > Buy = Cross(MA(C,fast),MA(C,slow)); > > > Sell = Cross(MA(C,slow),MA(C,fast)); > > > > > > - when I optimized on the total range I found that the top values, were > > > inverted (as per Howard's examples in this forum and his books) but when > > > I left out the 2008/09 extreme market conditions I found this did not > > > hold. > > > > > > Why does sucn a relatively small change in the test range make such a > > > radical difference in the outcomes? > > > > > > Here are some of the reported metrics from AB .. notice that they are > > > similar in some cases and markedly dissimilar in others. > > > > > > I am not sure if that leads to a question but it certainly gets my > > > attention considering that I am in the business of engineering > > > reward/risk. > > > > > > Note that I am using ProfitFactor because it is typical in the industry > > > but it has some question marks over whether is it the best CoreMetric to > > > use (I am investgating PowerFactor and assymetricalPayoffRatio which > > > might be more apt ... I hope to post more on these metrics later). > > > > > > Opt1: > > > > > > using all data > > > > > > top model = CAR/MDD == 0.25 AND periods == fast 10, slow 7; > > > > > > NETT PROFIT 1749% > > > Exposure 44.9; > > > CAR 7.68; > > > RAR 17.07 > > > MAXDD 31.57 > > > RECOVERYFACTOR 2.64 > > > PF 1.62 (WIN 68% * PR 0.75) > > > #TRADES 588 > > > > > > > > > Opt2: > > > > > > using data range from 2/01/1970 to 31/12/2007 (that's Dec for the > > > benefit of timezoners). > > > > > > top model = CAR/MDD == 0.42 AND periods == fast 1, slow 6; > > > > > > NETT PROFIT 1921% > > > Exposure 54.51; > > > CAR 8.23; > > > RAR 15.09 > > > MAXDD 21.67 > > > RECOVERYFACTOR 5.07 > > > PF 1.35 (WIN 40% * PR 2.03) > > > #TRADES 1049 > > > > > > > > > I hope I reported the metrics correctly but anyone can replicate my tests > > > and report otherwise. > > > > > > I think it also demonstrates that if PoF (PowerFactor) is a better > > > CoreMetric than ProfitFactor it will need to be standardized on a > > > returns/time basis (choose your time period = the basetimeframe you trade > > > ... PoF is related to GeoMean per bar?) > > > > > > --- In [email protected], "brian_z111" <brian_z111@> wrote: > > > > > > > > Following recent discussions on benchmarking and using rule based > > > > systems to engineer returns to meet 'clients' profiles i.e.Samantha's > > > > MA(C,10) example, I did some follow up R&D with the intent of expanding > > > > the examination a little further via a zboard post. > > > > > > > > I may, or may not, get around to that so in the meantime I decided I > > > > would share a couple of things while they are still topical. > > > > > > > > I made up some quick and dirty randomly generated eq curves so that I > > > > could optimise MA(C,10) on them (out of curiosity). > > > > > > > > Also, out of curiosity, I decided to see how the example signal/filter > > > > code that I made up, as the study piece for Yofas topic on > > > > benchmarking, would actually perform. > > > > > > > > Buy = Ref(ROC(MA(C,1),1),-1) < 0 AND ROC(MA(C,1),1) > 0 AND > > > > ROC(MA(C,10),1) > 0; > > > > Sell = Cross(MA(C,10),C);//no thought went into this exit and I haven't > > > > tried any optimization of the entry or the exit > > > > > > > > By chance I noticed that it outperformed on one or two of the > > > > constituents of the ^DJI (Yahoo data ... 2005 to 2009) and to the naked > > > > eye the constituents all seem to be correlated to a fair extent over > > > > that time range. > > > > > > > > Also, to the naked eye, it outperforms on randomly generated stock > > > > prices around 50% of the time and the outperformnce doesn't appear to > > > > be correlated to the underlying(I haven't attempted to find an > > > > explanation for this). > > > > > > > > Here is the code I used to make up some randomly generated 'stocks'. > > > > > > > > As we would expect it produces, say, 100 price series with a > > > > concatenated mean of around zero (W/L = 1 and PayoffRatio == 1) etc. > > > > When plotted at the same time ... individual price series are dispersed > > > > around the mean in a 'probability cone' ... in this case it is a > > > > relatively tight cone because the method doesn't introduce a lot of > > > > volatility to the series. > > > > > > > > /*P_RandomEquity*/ > > > > > > > > //Use as a Scan to create PseudoRandom Equity curves > > > > //Current symbol, All quotations in AA, select basetimeframe in AA > > > > Settings > > > > //It will also create the curves if used as an indicator (add the > > > > appropriate flag to ATC) > > > > // but this is NOT recommended as it will recalculate them on every > > > > refresh. > > > > //Indicator mode is good for viewing recalculated curves (click in > > > > whitespace) > > > > //CommentOut the Scan code before using the indicator code. > > > > //Don't use a very large N or it will freeze up indicator scrolling etc > > > > > > > > n = 100;//manually input desired number - used in Scan AND Indicator > > > > mode > > > > > > > > ///SCAN/////////////////////////////////////////////////////////////////// > > > > > > > > > > > > Buy=Sell=0; > > > > > > > > for( i = 1; i < n; i++ ) > > > > > > > > { > > > > > > > > VarSet( "D"+i, 100 * exp( Cum(log(1 + (Random() - 0.5)/100)) ) ); > > > > AddToComposite(VarGet( "D"+i ),"~Random" + i,"X",1|2|128); > > > > //Plot( VarGet( "D"+i ), "D"+i, 1,1 ); > > > > //PlotForeign("~Random" + i,"Random" + 1,1,1); > > > > } > > > > > > > > /* > > > > ////PLOT///////////////////////////////////////////////////// > > > > > > > > //use the same number setting as for the Scan > > > > > > > > > > > > for( i = 1; i < n; i++ ) > > > > > > > > { > > > > > > > > PlotForeign("~Random" + i,"Random" + i,1,1); > > > > > > > > } > > > > > > > > > > > > ////OPTIMIZE/////////////////////////////////////////////////////////// > > > > > > > > //use the filter to run on Group253 OR add ~Random + i PseudoTickers to > > > > a Watchlist and define by AA filter > > > > > > > > > > > > //fast = Optimize( "MA Fast", 1, 1, 10, 1 ); > > > > //slow = Optimize("MA Slow", 4, 4, 20, 1 ); > > > > > > > > //PositionSize = -100/P; > > > > //Buy = Cross(MA(C,fast),MA(C,slow)); > > > > //Sell = Cross(MA(C,slow),MA(C,fast)); > > > > > > > > //Short = Sell; > > > > //Cover = Buy; > > > > > > > > I also stumbled on this, which seems to have some relevance: > > > > > > > > http://www.scribd.com/doc/6737301/Trading-eBookCan-Technical-Analysis-Still-Beat-Random-Systems > > > > > > > > > > > > It contains a link to a site that has a free download of some RNG > > > > produced datasets. > > > > > > > > There hasn't been much discussion on using synthetic data in the forum > > > > ... Patrick recommended it for testing? OR benchmarking? ... Fred is > > > > against using it ("If we knew enough about the characteristics of the > > > > data, in the first place, to be able to create synthetic data then we > > > > would know enough to design trading systems to exploit the data's > > > > profile anyway", OR something like that). > > > > > > > > I was titillated enough by my first excursion into benchmarking with > > > > synthetic data to bring me back for some more. > > > > > > > > > >
