Part1 of Project Based Training No1.
The objective of the project is
to introduce new traders to the main
concepts of system design/testing and
demonstrate their application
in AmiBroker.
At the same time it is
hoped that the ideas presented will provoke
discussion and provide trading
stimulation.
All of the stages in the design process will not be
demonstrated as
most have already been covered elsewhere in the AmiBroker
support
material.
A basic understanding of the application of some
statistical methods
to the trading environment is a pre-requisite.
The
opening topics address this need.
To those who find the subject matter
new *the project* will be a
workbook .
To those who have experience in
the subject it will be an
opportunity to workshop.
I would like to
acknowledge my indebtedness to the academic
community .
I often refer
to the material so generously interpreted for the
layperson and made
available at websites by academic specialists,
particularly those
associated with
Universities.
*******************************************************************
Margin
of Error.
Back-testing of historical data provides traders with a
sample,
typical of the trade they are testing. From that sample they make
inferences about the larger group, or population, of all past trades
and future trades, of the same type, that were not included in their
test window.
Despite the fact that the people who teach them to
back-test also
teach them that the past can not predict the future, some
continue
to act as if it can.
If the past can't predict the future.
How can anyone trade with
confidence?
The answer is that while the
future can't be predicted, the
likelihood of some mathematically defined
outcomes can be predicted
with a degree of confidence.
Statistics is
the mathematical discipline that manages that very
well.
The caveat
is that to apply statistical methods to trading samples,
the assumption is
made that they are the result of a random process.
Where the trading system
chosen is biased to non-random behaviour it
will be prone to failure if
the market acts contrary to that bias.
For that reason system traders
are faced with a choice between
attempting to define market behaviour e.g.
a trend, and pick a
system to suit that, or search for a universal signal
that is
consistent irrespective of any assumed market bias.
If
statistics can predict the likelihood of future trading outcomes,
how
accurate will it be?
*Standard error* or *margin of error* offers
traders a solution but
they are not subjects that are often
discussed.
In his book ,*Design, Testing, and Optimisation of Trading
Systems*
(John Wiley & Sons, 1992), Robert Pardo raises the issue of
the
accuracy of trading *predictions* based on the size of the sample
used:
* The sample size must be large enough to allow the trading
system
to generate a statistically significant sample of trades.
A
sample of one trade is certainly insignificant, whereas a sample
of 50
trades or more is generally adequate.*
He uses Standard Error as a
measure of significance:
StdError = = 1/SquareRoot(sample
size),
1/SqRt(50) = = 14.1%.
There is little by way of further
explanation provided.
Applying the formula to a greater number of
samples:
Where N = = the number of trades in the sample
StdError
factor = = 1/SqRt(N)
StdError% = 1/SqRt(N) * 100
If N = = 2500 the
StdError% = = 1/SqRt(2500) * 100 = = +/- 2%
If N = = 10000 the StdError% =
= 1/SqRt(10000) * 100 = = +/- 1%
A trade sample of 10000 to provide
statistical accuracy of 1% is not
easily achievable for traders, although
a lot easier than accurately
surveying the eye colour of Polar
Bears.
Pardos equation is in fact, a rounding of the StdError equation
for
a 95% level of confidence:
Margin of error at 99% confidence =
= 1.29/SqRt(N)
Margin of error at 95% confidence = =
0.98/SqRt(N)
Margin of error at 90% confidence = =
0.82/SqRt(N)
Later in the project I will use a basic random number
generator,
within Xcel, to provide a visual aid that traders can use to
understand the *sample* concept and decide for themselves what
constitutes an adequate sample.
Wikipedia provides some additional
clarity on the subject:
http://en.wikipedia.org/wiki/Margin_of_error
*The
margin of error expresses the amount of the random variation
underlying a
survey's results. This can be thought of as a measure
of the variation one
would see in reported percentages if the same
poll were taken multiple
times. The larger the margin of error, the
less confidence one has that
the poll's reported percentages are
close to the "true" percentages, that
is the percentages in the
whole population.*
*An interesting
mathematical fact is that the margin of error
depends only on the sample
size and not on the population size,
provided that the population is
significantly larger than the sample
size, and provided a simple random
sample is used. Thus for
instance
.the running example with 1,013 random
samples
would
yield essentially the same margin of error (4% with a 99%
level of
confidence) regardless of whether the population
.consisted of
100,000 or 100,000,000.*
In short the tail of the trading
system sample is swinging the
trading system
cat.
BrianB2
The material contained in this topic is for
educational and
discussion use only.
It is not intended as financial
advice and should not be construed
as such.
The author is not an
accredited academic or financial advisor.
