> This may be true for your brain, but not for a mechanical trading >system ...
True, mechanical system analysis takes the subjective factor out of the equation, provided the practitioner doesn't inadvertently let subjective analysis creep in. Mechanical system analyses are performed by people with brains though. The value of any analysis, however, rests on the premise that the future will resemble the past with enough similarity to make historical data relevant ... if the future is changing faster than in the past where are we going to get the data to test? Of course a prediction that the future will not be similar to the past isn't worth the paper it is written on ... it doesn't hurt to be cautious though, especially when you see some warning signs, does it? >In more complex optimizations .... this a little more than a little >bit more >difficult ..... where it counts i.e. during the >optimization process itself. Have you published a recommended method, or commentary, for these difficult analyses anywhere? Has anyone else published anything on sensitivity analysis worth reading? Thanks for the feedback ... sensitivity analysis is the aspect of opt that is leaping out at me. --- In [email protected], ftone...@... wrote: > > Brian, > > In regards to your statement of ... > > ... "As an aside I am sceptical that there is evne such a thing as OOS when > we use historical data ... for one thing we are all too widely read to be > naive about any trading idea so in that sense we have all walked over any > historical data, we care to get our hands on, thousands of times .. in that > sense live trading is the only 'real' OutOfSample data" ... > > This may be true for your brain, but not for a mechanical trading system ... > > As far as optimization is concerned the way to work the problem is backwards > ... > > Assuming that the goal is to have a robust set of variable values i.e. one > that is most likely to do well OOS then this translates into having those > parameter values not be highly sensitive. In a two variable system this is > fairly easy to visualize with AB's 3d plots picking whatever performance > metric one wants for the the z-axis. In more complex optimizations that > involve more than 2 variables and/or enough combinations where exhaustive > search is not feasible then this a little more than a little bit more > difficult even after the fact let alone where it counts i.e. during the > optimization process itself. >
