In does make sense except for the comment about randomness in markets and the weather neither of which I agree with ...
--- In [EMAIL PROTECTED], "rhelfer123" <[EMAIL PROTECTED]> wrote: > > Hi Fred, > > In-sample/out-of-sample testing can be performed on Monte Carlo > Simulation, on its own. If you get a significantly different MCS > bell curve on the out-of-sample data, I would imagine this would > indicate the system does not function well on the OOS data. I > usually test MCS with OOS and also known bear and bull cycle data, > as well. This covers all bases. > > I have never been able to quickly identify a system that works well > in MCS using a straight backtest first. However, every time I find a > system that works well in MCS, it always works well in the straight > backtest. > > By "works well" I mean a system that is based in realistic real- > world statistics. That is, testing that mirrors the true randomness > of everyday reality. > > MCS was originally developed to predict where nuclear fallout would > land, from nuclear tests in the Southwest desert of the US. The > predictions for this MUST be very accurate. MCS was specifically > developed for this kind of highly predictive accuracy. Weather > patterns are just as random as the stock markets. > > OOS testing is specifically used for preventing curve fitting in > backtesting for system optimization. It's a whole different > enchilada. You can use OOS with MCS, however. > > I hope this makes sense. > > Thanks for reading, > > rhelfer > > --- In [EMAIL PROTECTED], "Fred" <ftonetti@> wrote: > > > > I don't think your ideas or practices nullify anything ... At > worst > > they supplement it ... > > > > While I'm not a huge fan of MCS, the methodology does provide > > additional information over and above what a straight backtest > > provides ... > > > > It's my contention however that this additional information while > > beneficial is no substitute for out of sample testing regardless > of > > how it is performed. > > > Fred >
