Re comparing random testing to a system, to confirm the edge: Generally speaking I agree with Mike ... that is the obvious answer, but I have gained a great deal by continual labttesting and whatif specualation (actual and virtual).
Testing the tools, the myths, the philosophy, authors ideas, strategies, systems, data, crazy ideas, holy cows, indicators ... in short everything. The results from all of this weren't immediate, or direct ... over time it revealed my trading philosophy, temperament and strategies etc which are directly reflected in my systems. Re random entries: - the larger the sample (N) the more we can rely on the result - so, massive random testing is more instructive than a small sample - if we conduct a massive number of random entries (say on daily bars) we will eventually end up entering every bar an approx equal number of times (with a random entry every ball in the bucket has an equal chance of getting drawn, assuming sampling with replacement).... so why not just go straight to buying every day (on close, on open or something else?) ... it saves the extra work of including the random entry code and uses up less of your data in the IS..... the only difference between your benchmark and your real signals will be N .... the real sample set will have a smaller N and a larger sample error. Re optimising random entries: - very interesting - at first thoughts it looks like, at the least, you are testing the validity of optimising - including your filters may complicate the research ... perhaps you should try isolating each component (filters, trade management, optimisation). - it would be interesting to see what type of opt results you get from a random entry - even more interesting to see how many 'good' optimised random entries perform in OOS walk -forward - if any perform above expectations, for a random system, the explanation would be even more interesting Speculating on the outcomes of your research: - without knowing what your objectives are - leaving out filters and trade management - optimising random entries is likely to produce a set of parameters that approximate the perfect buy and sell signals for the data you are analysing (see Howard's QTS book page81 for some insight into perfect signals) ... notice that, in Howards screenshot example, they approximate smoothed cycles. - if you wanted to hit every perfect buy/sell then you would need to add more lines of code and of course the number of trades in your system will increase until the ratio of trades/number of perfect trades == 1 If any 'systems' produced by opting random signals survives the OOS sample test it can only be because they are not significant, compared to random chance OR they have indentified some persisent and recurring cycle (seasonal patterns, moon phase, Fibonacci or a new one that you can discover .... fascetious there ... just having some fun with it). An interesting variation, on your research, might be to randomly generate data and see what optimising it can do. Note that randomly generated data doesn't match up to real market data ... the markets tend to have a lot more extreme results than can explained by a normal dist (I think Mandelbrot did some work with fractal maths and generation of more realistic distributions if you want to stress test opt in a realistic environment). (Some people claim that fractals is a pseudoscience ... according to others it has something to say about wave patterns, or cycles, in market data ... I don't know about that but I think the computer generated fractal art would look great on a Tshirt). http://www.scientificamerican.com/article.cfm?id=multifractals-explain-wall-street Most of the time the text books don't answer the hard trading questions, or answer them in enough detail, so you can only can an advantage by pushing the envelope on optimisation theory. Howard's chapter on benchmarking is only scratching the surface and I haven't found a lot of critical analysis of optimisation anywhere (only a few high tech efforts like Whites reality check for datamining etc). --- In [email protected], "Yofa" <jtoth...@...> wrote: > > Hi All, > > I'm trying to improve my optimization method. So I divided my trading system > into parts: entry logic, trade management logic (trailing, profit target, > volatility exit, etc ), filters, etc. > > I created a random entry system, that uses the same trade management logic > as my trading system. > With random entries I optimized the parameters of the trade management logic. > I also try to improve filters the same way. > > My questions: > Is there anyone who uses similar technic for optimization? > Is there anyone how uses similar approach to validate the trading system > and its parameters? > Is it reasonable optimization method? > > Any opinion or experiance appreciated. > > Regards, > > Y >
