Re: [amibroker] Re: Backtest vs Forwardtest

Wed, 30 Aug 2006 14:20:35 -0700

You may want to try optimizing on the last year and backtest out of sample on previous years.

herman


intermilan04 wrote: 
Hi Sebastian,

"you can't really know that your system has "broken down" until you
get the final results on January 1, 2007.:)"

Very true.  I do not think my system has "broken down," it's just that
it is underperforming at a dreadful level compared to how it had been.

As for data range, I also stated it in reply to dingo but for some
reason this system just rips apart 1999-2000 and 2000-2001 (1900% and
3700% CARs respectively) that I thought it was better to omit the two
years and test after the dot-com bust. (yeah, 2000-2001 is technically
including the bust but the system performs surprisingly well)

I will once again optimize with 10 years of data and see how it goes.

Thank you for your reply,

intermilan04

--- In [email protected], "sebastiandanconia"
<[EMAIL PROTECTED]> wrote:
  
Without knowing any more about your method (and I'm not asking), maybe
there's nothing wrong and your system's performance is simply an
accurate reflection of market action during those times?

>From Jan.-Jan. in all those years there was a substantial movement
(either up or down) in the overall stock market.  Since January of
this year, though, the markets have scarcely gone anywhere by
    
comparison.
  
So, two things:  First, the obvious one, you can't really know that
your system has "broken down" until you get the final results on
January 1, 2007.:)

Second, if you believe that your system is based on actual market
behaviors (not just randomly optimized) maybe it's working properly
and this is just one of those years when nothing much happens.  Which
is why I believe that 8-10 years is pretty much the minimum necessary
for an honest track record/backtest, rant, rant.:)  This may be a
totally valid "performance lull" period for a system that is stellar
most of the time.


Luck,

Sebastian

--- In [email protected], "intermilan04" <intermilan04@> wrote:
    
Hi all,

I'm having a puzzling situation where my backtest results are
fantastic yet my forwardtest result is nowhere near it.

My system is optimized between 2001/1/1 and 2006/1/1.  Results YTD is
"forwardtest" since it is beyond the scope of optimized data range.

Here are some numbers of backtests:
Year-by-year-results (CAR)
2001/1/1-2002/1/1: 393.70%
2002/1/1-2003/1/1: 232.64%
2003/1/1-2004/1/1: 721.79%
2004/1/1-2005/1/1: 400.82%
2005/1/1-2006/1/1: 490.72%

and at last--forwardtest
2006/1/1-2006/8/29: 74.64%

I am at a loss to explain this.  It's very sad that I work hard to
come up with a system that has worked, only to see it not working
nearly as good as it should be.

Any analysis/suggestions to fix the problem above is greatly
      
appreciated.
    
Sincerely,

intermilan04

      







Please note that this group is for discussion between users only.

To get support from AmiBroker please send an e-mail directly to 
SUPPORT {at} amibroker.com

For other support material please check also:
http://www.amibroker.com/support.html

 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/amibroker/

<*> To unsubscribe from this group, send an email to:
    [EMAIL PROTECTED]

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
__._,_.___

Please note that this group is for discussion between users only.

To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com

For other support material please check also:
http://www.amibroker.com/support.html




SPONSORED LINKS
Software support Small business finance Business finance online
Business finance training Business finance course

YAHOO! GROUPS LINKS



__,_._,___

Reply via email to