Dear Forum
I'm a relatively new user of Best Charts haven't use it for real yet, but am
trying to get a handle on it theoretically first. BC seems to be an
interesting and potentially powerful tool, but unfortunately it will take a bit
of testing to get it set-up before I can use it with confidence....
I completed my first test run and I thought I would publish some of the results
to get some discussion going on here and also feedback from some of the more
advanced forum members.
Some specifics of the test:
Test Period: 19 April '10 to 17 May `10
Test Stocks: Top 20 Australian stocks by market capitalisation (I used these
as they are large, well traded stocks good volumes, and hopefully the market
is well enough informed such that true technical analysis principles can
operate)
Test Indicator: BCI (I only did the BCI indicator as based on previous
optimisation testing, it seemed to give the best overall gains)
Parameters: BCI was `optimised' each day (according to the series description
below). I realise that the last parameter of the BCI is user defined and not
automatically optimised I just left this as the default.
Buy/Sell Triggers: Instruction from the optimisation output each day simply
go long when `Bullish' and go short when `Bearish' (as per the intent of the
test series below)
Prices Used: Buy/sell at the opening price on the day following the signal
generation.
Stop Losses: No stop loss triggers were used investment was held until the
signal changed.
Brokerage Costs: No brokerage costs were taken into account
Test Series: #1 Test Performance on Long Investment Only
Optimisation Period: 229 days
Optimise on Long Only
Invest Long Only
Test Series: #2a Test Performance on Long and Short Investment
Optimisation Period: 229 days
Optimise on Long & Short
Invest Long Only
Test Series: #2b Test Performance on Long and Short Investment
Optimisation Period: 229 days
Optimise on Long & Short
Invest Long & Short
Test Series: #3a to 3h Test Performance on Optimisation Period
Optimisation Period: 20, 60, 90, 120, 150, 180, 250, 320 days respectively
Optimise on Long & Short
Invest Long & Short
Issues that may have affected the results/market:
The overall ASX market during the test period commenced at the start of a
downtrend, 3 weeks duration. Recovery in the last week.
The Greek sovereign debt crisis was rearing its head during the test period and
got decidedly worse during the latter half of the test period.
The Australian government announced the imposition of a higher rate of taxation
on mining companies during the last week of the test period this had a severe
negative effect on resource stocks and resource service industry type stocks.
Test results:
1. Interestingly, the percentage of winning trades for each test series (so #
of winning trades for all 20 stocks in each test series compared to the total #
of trades for all 20 stocks in each test series) came in at around 50% - the
law of averages...... series #1 was low @ 38%, but the rest ranged from high
40's to mid 50's. This would suggest to me that using BCI in the way that I
did is no better than flipping a coin 50% of the time you will get heads and
50% of the time you will get tails. This is a bit disappointing as I would
view the ability to pick a winning trade as one of the most important metrics
you need to be on winning trades for something like 70% of the time to have
some sort of confidence that you are on the right track.
2. In terms of the most number of winning trades, tests 3b (60 day), 3c (90
day) and 3e (150 day) were the best performers with winning rates of 57%, 54%
and 59% respectively. Encouraging, but not stellar.
3. 2/20 stocks did extremely well in terms of the # of test series with
winning trades <70% - RIO (6/11 test series) and WOW (8/11 test series) one
resource stock and a consumer staple stock. During the test period, RIO was in
a major down trend and WOW was in a range-bound pattern. As a comparison other
stocks in the test population of 20 came in at 2-3 (at best) with series trades
>70% winners. I don't know if there is anything to be read from this
observation......
4. % winning LONG trades (41-67%) was decidedly greater than the % winning
SHORT trades (29-36%) for all test series. This is interesting, but not sure
what I can conclude from it. It seems to suggest that BCI's shorting accuracy
is less than that of long trigger accuracy. If you were to extend this concept
and say, only invest long (but optimise on long and short), you would come up
with a better outcome, but you would still only get in the high 60%'s for # of
winning trades close but not quite at the comfort level. WOW performed very
well in terms of % of long winning trades 7/11 tests achieved >70% winners.
5. It is difficult to present statistically correct and meaningful data for
this item, but in general, the % gain/trade for winning shorts was
significantly greater compared to the gain/trade for winning long trades. %
Losses/trade for losing longs and losing shorts were about equal.
6. Performance against benchmarks 4 benchmarks were used - #1 was the `buy
and hold' movement of the share price for each stock during the test period, #2
was a theoretical 7% pa return, #3 was the movement of the ASX 200 index (top
200 ASX stocks) over the test period, and #4 preservation of invested capital.
6a. Stock buy and hold - All test series performed considerably better
than the buy and hold scenario for each stock the BCI series achieved gains >
the benchmark for between 70-90% of the 20 stock test set.
6b. Nominal 7%pa interest Mixed results. Series 1 and 2a were decidedly
worse than the benchmark (10-15% of stocks outperformed the benchmark) this
is logical as these were series without shorting (so less time in the market
compared to a constant 7% earning rate). Other series ranged from 40-70%, with
best performers being 3b (70%), 3c (65%) and 3e (65%). The remainder averaged
out at the 50% mark......the law of averages again ??
6c. ASX200 buy and hold similar to 6a not surprising given make-up of
ASX200 index.
6d. Preservation of capital initial investment capital was preserved in
most series (generally 60-70% of the 20 stocks preserved capital, for each of
the test series), with the exception of series 1 and 2a (only 20% of the stocks
preserved capital). The inability to short in these series is the obvious
difference to the other series, however this doesn't explain the poor
performance if you couldn't short, then you just keep the money uninvented
so no risk. The only reason I can think of here is that the falling market
did not correlate well with the 229 day test period data and the long only
optimisation condition (for test 1) and the mismatched optimisation (long and
short) vs investment practice (long only) (for test 2a) resulted in BCI not
predicting well.
7. Annualised % returns across the 20 stock spread if you invested equal
initial amounts across the 20 stocks for each of the series, the following
annualised returns are realised:
Series 1 -38% pa
Series 2a -41% pa
Series 2b -1% pa
Series 3a 22% pa
Series 3b 47% pa
Series 3c 44% pa
Series 3d -26% pa
Series 3e 20% pa
Series 3f 24% pa
Series 3g -13% pa
Series 3h 3% pa
Benchmark 1 -87% pa
Benchmark 2 7% pa
Benchmark 3 -94% pa
Benchmark 4 0% pa
So, in summary, the BCI testing showed OK results against a `buy and hold'
strategy in a falling market, but the specific process used did not deliver
consistently high probability results (refer point 1). Shorter optimisation
periods (60 and 90 day) seem to work the best in this instance. Long trading
was more successful than shorting (don't know why this is counter market
trend) and shorting wins were bigger than long wins. No results came close the
optimised theoretical backtest results of ++100%pa (not sure if this is
realistic though).
I will conduct more testing going forward.
Any comments or suggestions. Analysis spreadsheet is available upon request
(it is a bit messy, but you will get the idea).
Paul
