I will second Dingo's comments about sample size. However, the other factors to consider (aong many) are what I call sample "segments." For example if you have a size bias built into your screening process there will be times when your system will produce above/below average returns relative to longer historical averages. An example of that is a system who results have been focused in the small cap space. We have had 6+ straight years of small cap out-performance - a period that coincides with your backtest period ( the last period was 1976-1982.) You have now perhaps a transitory period where both volatility and rotation is affecting your current results.
Just something to ponder.... Regards, Duke Jones, CMT > -------Original Message------- > From: dingo <[EMAIL PROTECTED]> > Subject: RE: [amibroker] Re: Backtest vs Forwardtest > Sent: 30 Aug '06 19:37 > > In most circles 5 years is not enough. Plus profits are not all you should > be looking at imho. How about drawdowns? I get nausea just typing that > word. > > If you have access to past emails on this list then you might want to search > for "robust". There have been several nice exchanges on this term - one > that your have just tripped over. > > d > > > -----Original Message----- > > From: [email protected] > > [mailto:[EMAIL PROTECTED] On Behalf Of intermilan04 > > Sent: Wednesday, August 30, 2006 2:37 PM > > To: [email protected] > > Subject: [amibroker] Re: Backtest vs Forwardtest > > > > Hi dingo, > > > > Thank you for your prompt reply. > > > > 75% is really good, if I can snatch it. Often times I see > > that number when I backtest, but it drops once I start > > following my system... > > > > The past results are indeed too good to be true but my system > > is not looking at future quotes. I guess it could be the > > curve-fitting problem though, if 5 years of data isn't enough. > > > > I had tried with longer data range but what happened was this > > system racked up 1900% and 3700% in 1999-2000 and 2000-2001, > > which skewed my average CAR... > > > > Are there ways to avoid the curve-fitting issue other than > > backtesting against longer data range? > > > > Regards, > > > > intermilan04 > > > > --- In [email protected], "dingo" <[EMAIL PROTECTED]> wrote: > > > > > > And 75% isn't good enough for you? > > > > > > And don't you think the past results just might be too good > > to be true? > > > > > > And you checked your formula to see if it is looking into > > the future? > > > > > > And if it isn't then it just could be so optimized that it > > is "curve > > > fitting". > > > > > > And without your code its hard to pinpoint what's going on.... > > > > > > d > > > > > > > -----Original Message----- > > > > From: [email protected] > > > > [mailto:[EMAIL PROTECTED] On Behalf Of intermilan04 > > > > Sent: Wednesday, August 30, 2006 2:11 PM > > > > To: [email protected] > > > > Subject: [amibroker] Backtest vs Forwardtest > > > > > > > > Hi all, > > > > > > > > I'm having a puzzling situation where my backtest results are > > > > fantastic yet my forwardtest result is nowhere near it. > > > > > > > > My system is optimized between 2001/1/1 and 2006/1/1. > > > > Results YTD is "forwardtest" since it is beyond the scope of > > > > optimized data range. > > > > > > > > Here are some numbers of backtests: > > > > Year-by-year-results (CAR) > > > > 2001/1/1-2002/1/1: 393.70% > > > > 2002/1/1-2003/1/1: 232.64% > > > > 2003/1/1-2004/1/1: 721.79% > > > > 2004/1/1-2005/1/1: 400.82% > > > > 2005/1/1-2006/1/1: 490.72% > > > > > > > > and at last--forwardtest > > > > 2006/1/1-2006/8/29: 74.64% > > > > > > > > I am at a loss to explain this. It's very sad that I > > work hard to > > > > come up with a system that has worked, only to see it not working > > > > nearly as good as it should be. > > > > > > > > Any analysis/suggestions to fix the problem above is greatly > > > > appreciated. > > > > > > > > Sincerely, > > > > > > > > intermilan04 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please note that this group is for discussion between users only. > > > > > > > > To get support from AmiBroker please send an e-mail directly to > > > > SUPPORT {at} amibroker.com > > > > > > > > For other support material please check also: > > > > http://www.amibroker.com/support.html > > > > > > > > > > > > Yahoo! Groups Links > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > No virus found in this incoming message. > > > > Checked by AVG Free Edition. > > > > Version: 7.1.405 / Virus Database: 268.11.7/433 - Release > > > > Date: 8/30/2006 > > > > > > > > > > > > > > > > > > > > > > > > > > > Please note that this group is for discussion between users only. > > > > To get support from AmiBroker please send an e-mail directly > > to SUPPORT {at} amibroker.com > > > > For other support material please check also: > > http://www.amibroker.com/support.html > > > > > > Yahoo! Groups Links > > > > > > > > > > > > > > -- > > No virus found in this incoming message. > > Checked by AVG Free Edition. > > Version: 7.1.405 / Virus Database: 268.11.7/433 - Release > > Date: 8/30/2006 > > > > > > > > Please note that this group is for discussion between users only. > > To get support from AmiBroker please send an e-mail directly to > SUPPORT {at} amibroker.com > > For other support material please check also: > http://www.amibroker.com/support.html > > > Yahoo! Groups Links > > > > > > > > > Please note that this group is for discussion between users only. To get support from AmiBroker please send an e-mail directly to SUPPORT {at} amibroker.com For other support material please check also: http://www.amibroker.com/support.html Yahoo! Groups Links <*> To visit your group on the web, go to: http://groups.yahoo.com/group/amibroker/ <*> To unsubscribe from this group, send an email to: [EMAIL PROTECTED] <*> Your use of Yahoo! Groups is subject to: http://docs.yahoo.com/info/terms/
