Hi Fred, In-sample/out-of-sample testing can be performed on Monte Carlo Simulation, on its own. If you get a significantly different MCS bell curve on the out-of-sample data, I would imagine this would indicate the system does not function well on the OOS data. I usually test MCS with OOS and also known bear and bull cycle data, as well. This covers all bases.
I have never been able to quickly identify a system that works well in MCS using a straight backtest first. However, every time I find a system that works well in MCS, it always works well in the straight backtest. By "works well" I mean a system that is based in realistic real- world statistics. That is, testing that mirrors the true randomness of everyday reality. MCS was originally developed to predict where nuclear fallout would land, from nuclear tests in the Southwest desert of the US. The predictions for this MUST be very accurate. MCS was specifically developed for this kind of highly predictive accuracy. Weather patterns are just as random as the stock markets. OOS testing is specifically used for preventing curve fitting in backtesting for system optimization. It's a whole different enchilada. You can use OOS with MCS, however. I hope this makes sense. Thanks for reading, rhelfer --- In [email protected], "Fred" <[EMAIL PROTECTED]> wrote: > > I don't think your ideas or practices nullify anything ... At worst > they supplement it ... > > While I'm not a huge fan of MCS, the methodology does provide > additional information over and above what a straight backtest > provides ... > > It's my contention however that this additional information while > beneficial is no substitute for out of sample testing regardless of > how it is performed. > Fred
