Near Random ? You're kidding right ? ... I'd thought you had beeen around longer then that ...
On another level the number of trades has no relevance in calculating K-Ratio and as calculated in AB is useless for anything but systems that trade constant dollar amounts ... By definition that would imply no degree of success over a long period of time. ----- Original Message ----- From: brian_z11 Date: Sunday, May 10, 2009 8:58 pm Subject: [amibroker] Re: Expectancy - and related--specifically K-rato To: [email protected] > My understanding of Bob Pardo's book is that he feels > that the length of the > out-of-sample period can be determined by a calculation based on > the length > of the in-sample period. If that were true, then the ratio > would be a > little easier to compute. But, in my experience, there is no > relationshipbetween the length of the out-of-sample period and > the length of the > in-sample period. And gathering the data and performing the > calculation of > the ratio for some objective functions would be difficult. > > I don't believe that it would be worth the effort but setting > tbe BT the task of collecting the same number of trades in the > OOS test as were collected in the IS test is the way to > standardise and simplify calculating this ratio. > > N == the number of closed trades in a test run; > Where IS(N) == OOS(N) the ratio IS(N bars)/OOS(N bars) will > always be defined as a probability distribution. > > No constant relationsip, between IS and OOS length (in bars) can > exist because of the (near) random nature of the markets. > > Believe that such a relationship does exist, and hence can be > calculated/exploited, is an artefact of the belief that the > markets are non random i.e. have a structure that we can > identify, model and therefore use to make predictions about the > future of the markets. > > Any attempt to improve OOS performance by 'tuning' the length of > the sample periods is a futile excercise. > > The exception to this, as you point out in your book, is that > the markets do change (they are not quite random .... human > behaviour leaves a faint trail of non randomness) and so current > data may be more relevant than non- current data e.g. increased > use of computers, and access to information, by traders may have > changed the markets in the last decade or two. > > Whether these changes are strong enough, and occur over short > enough time periods or identifiable time periods, to justify the > view that we can/must account for this, in our design processes, > is arguable i.e. it is questionable that we can "synchronise the > market and our trading system by shortening (varying) the time > periods used in our walk forward testing", as you suggested in > your QTS book (P261). > > Granted that we are attempting to synchronise our systems to > market behaviour but this is done within our system rules, not > by altering the N bars tested. > > The fundamental premise of trading is that the system will > perform, in the future, irrespective of market conditions or the > time period involved. > > The OOS walk through is a test to find out if the patterns we > identified in the IS data still exist in the future and that the > code we used to synchronise to that pattern(s) is effective. > > The only time that will be time dependent is when the patterns > are time based e.g. the number of times the price of oil goes up > on the first day of the month is statistically significant. > > Very few exploitable inefficiencies in the markets are time based. > > > On another point: > > Theoretically the OOS results should be better/worse than the IS > tests on a 50/50 basis. > > Continual skewing of that ratio to the downside is an indicator > of curve fitting at the IS stage? > > > > > > --- In [email protected], Howard B wrote: > > > > Greetings all -- > > > > In-sample results and out-of-sample results can be, and > usually are, very > > different in their characteristics. > > > > My experience is that the ratio (OOS/(IS+OOS)), where these > are the > > in-sample and out-of-sample results, is difficult to compute > and often even > > difficult to define. I have not found it to be of value in > estimating the > > future performance of the system. > > > > My understanding of Bob Pardo's book is that he feels that the > length of the > > out-of-sample period can be determined by a calculation based > on the length > > of the in-sample period. If that were true, then the ratio > would be a > > little easier to compute. But, in my experience, there is no > relationship> between the length of the out-of-sample period and > the length of the > > in-sample period. And gathering the data and performing the > calculation of > > the ratio for some objective functions would be difficult. > > > > Isn't it the out-of-sample results we are trying to estimate? > Do we care > > what the in-sample results look like? If the out-of-sample > results are > > terrible, why bother computing the ratio. If the out-of- > sample results are > > good, why bother computing the ratio -- how will that > information be used to > > improve the system? And if it is used to modify the system, > then the > > previously out-of-sample data has become in-sample. > > > > Do any of the forum member have examples they can contribute > where computing > > the ratio is helpful? > > > > Thanks, > > Howard > > > > >
