>Whether or not you believe sensitivity analysis has any value during >the IS >optimization phase, if the topic is of interest to you I >suggest you read >that section of the IO doc which can be found in >the files section here.
Pure speculation only but yes I believe it is THE value point of opting. Yes I am interested in the theory ... I can see where it fits but confirming the details will be instructive. I don't know if I am ready to come over to the dark side just yet though. Thanks. --- In [email protected], ftone...@... wrote: > > Brian, > > In response to ... > > ... "Have you published a recommended method, or commentary, for these > difficult analyses anywhere? > > Has anyone else published anything on sensitivity analysis worth reading? > > Thanks for the feedback ... sensitivity analysis is the aspect of opt that is > leaping out at me." ... > > Yes ... It's called IO ... ;) ... Whether or not you believe sensitivity > analysis has any value during the IS optimization phase, if the topic is of > interest to you I suggest you read that section of the IO doc which can be > found in the files section here. > > In particular: > > - Pages 18 - 20 for the graphical presentation of sensitivity of variable > values arrived at during optimization which can be utilized regardless of the > number of variables > > - Pages 36 - 41 for theory behind, methodology of and results from > sensitivity testing during IS optimization > > - Pages 34 - 35 for directives that can be used to affect how thoroughly > sensitivity testing during IS optimization can be performed > > I had posted some abbreviated content about this in the UKB but I don't know > if thats still there as I don't seem to be able to reach the UKB url today. > In any case the pages in the doc I referenced above have more diagrams and > pictures than words so it shouldn't take long to get through what's there > with the middle reference being the real guts. > > ----- Original Message ----- > From: brian_z111 > Date: Sunday, June 21, 2009 1:59 am > Subject: [amibroker] Re: Benchmarking > To: [email protected] > > > > This may be true for your brain, but not for a mechanical > > trading >system ... > > > > > > True, mechanical system analysis takes the subjective factor out > > of the equation, provided the practitioner doesn't inadvertently > > let subjective analysis creep in. > > > > Mechanical system analyses are performed by people with brains though. > > > > The value of any analysis, however, rests on the premise that > > the future will resemble the past with enough similarity to make > > historical data relevant ... if the future is changing faster > > than in the past where are we going to get the data to test? > > > > Of course a prediction that the future will not be similar to > > the past isn't worth the paper it is written on ... it doesn't > > hurt to be cautious though, especially when you see some warning > > signs, does it? > > > > >In more complex optimizations .... this a little more than a > > little >bit more difficult ..... where it counts i.e. during the > > >optimization process itself. > > > > Have you published a recommended method, or commentary, for > > these difficult analyses anywhere? > > > > Has anyone else published anything on sensitivity analysis worth > > reading? > > Thanks for the feedback ... sensitivity analysis is the aspect > > of opt that is leaping out at me. > > > > --- In [email protected], ftonetti@ wrote: > > > > > > Brian, > > > > > > In regards to your statement of ... > > > > > > ... "As an aside I am sceptical that there is evne such a > > thing as OOS when we use historical data ... for one thing we > > are all too widely read to be naive about any trading idea so in > > that sense we have all walked over any historical data, we care > > to get our hands on, thousands of times .. in that sense live > > trading is the only 'real' OutOfSample data" ... > > > > > > This may be true for your brain, but not for a mechanical > > trading system ... > > > > > > As far as optimization is concerned the way to work the > > problem is backwards ... > > > > > > Assuming that the goal is to have a robust set of variable > > values i.e. one that is most likely to do well OOS then this > > translates into having those parameter values not be highly > > sensitive. In a two variable system this is fairly easy to > > visualize with AB's 3d plots picking whatever performance metric > > one wants for the the z-axis. In more complex optimizations > > that involve more than 2 variables and/or enough combinations > > where exhaustive search is not feasible then this a little more > > than a little bit more difficult even after the fact let alone > > where it counts i.e. during the optimization process itself. > > > > > > > > > >
