Brian, In regards to your statement of ...
... "As an aside I am sceptical that there is evne such a thing as OOS when we use historical data ... for one thing we are all too widely read to be naive about any trading idea so in that sense we have all walked over any historical data, we care to get our hands on, thousands of times .. in that sense live trading is the only 'real' OutOfSample data" ... This may be true for your brain, but not for a mechanical trading system ... As far as optimization is concerned the way to work the problem is backwards ... Assuming that the goal is to have a robust set of variable values i.e. one that is most likely to do well OOS then this translates into having those parameter values not be highly sensitive. In a two variable system this is fairly easy to visualize with AB's 3d plots picking whatever performance metric one wants for the the z-axis. In more complex optimizations that involve more than 2 variables and/or enough combinations where exhaustive search is not feasible then this a little more than a little bit more difficult even after the fact let alone where it counts i.e. during the optimization process itself.
