Thanks for response Mike,
I don't think I was very clear as I read my statement again. I am trying to
modify the K Ratio function that AB has hard wired into code. And it takes such
things as standard deviation. standard error and linear regression functions to
compute K Ratio. And to get those functions it needs the number of bars in each
OOS and IS run. I've been doing it manually with param functions and should
have removed them before posting. Sorry.
Let me start again.
The objective of this AFL is to create a custom metric to be used by an
optimization run on another set of parameters. This custom metric is getting
the values for linear regression slope, standard deviation and standard error.
The problem I have is getting formula to do this into the custom metric AFL.
Specifically
New Metric = st.GetValue("StandardError"); // Standard Error is available,
but not Linear regression slope, standard deviation
Hope this clears up what I'm trying to do.
Thanks
Dave
----- Original Message -----
From: Mike
To: [email protected]
Sent: Friday, March 21, 2008 6:34 PM
Subject: [amibroker] Re: Custom Metric for Walk Forward Optimization
Hi,
One of the goals of walk forward optimization is to remove the
discretionary decisions (i.e. the Params) and see how your system
would perform on out of sample (OOS) data after having been optimized
over in sample (IS) data.
At each OOS phase, the sytem uses only one set of values; the optimal
values of the previous IS optimization. Think of it as optimization
followed by backtest followed by optimization followed by backtest...
As such, you probably want to convert your Param statements to
Optimize statements, then set your custom metric as the Optimization
Target in the Walk Forward settings tab (
http://www.amibroker.com/kb/category/afl/systems/ ) and select walk
forward from the Optimize button of the AA window.
Note, your customization metric will be calculated for each
optimization combination and used as the measure by which each
optimization is judged when determining which single set of values to
apply to the next OOS backtest.
Specifically; At each iteration, your script will be optimized over
all combinations against the IS period. Then the optimal values,
based on the best rank as per your optimization target, will be
applied to the OOS period to give the OOS results.
Then the dates advance forward such that the OOS data becomes part of
the IS data and the process is repeated, giving OOS results with
possibly different optimized values at each stage.
However, the following line of your code makes no sense to me, and
should probably just be removed, since you've already calculated your
new value and all you need to do is set it as a custom metric:
> NewKRatio = st.getvalue("NewKRatio");
Mike
--- In [email protected], "David Fitch" <[EMAIL PROTECTED]>
wrote:
>
> I am trying to make a custom metric for walk forward optimization
testing. This custom metric includes parameters that need input from
the walk forward periods. For example, if I were making a custom
metric K Ratio, I would need the number of bars in the IS and OOS
periods to figure Standard Error, Standard Deviation, and Linear
Regression slope. How do I do this? And is there a better way?
>
> Here's a custom metric AFL with parameter boxes that would be
replaced with whatever is needed to make this work when running walk
forward optimization.
>
> LRPeriods=Param("LinRegPeriods",10,1,500,1);
> LR=LinRegSlope(e,LRPeriods);
>
> SEPeriods=Param("SEPeriods",10,1,500,1);
>
> SE=StdErr(e,SEPeriods);
>
> SDPeriods=Param("SDPeriods",10,1,500,1);
>
> SD=StDev(e,SDPeriods);
>
> NewKRatio= LR / ((SE/SD)* sqrt(SDPeriods));
>
> SetCustomBacktestProc("");
>
> if (Status("Action") == actionPortfolio)
>
> {
>
> bo = GetBacktesterObject();
>
> bo.backtest();
>
> st = bo.getperformancestats(0);
>
> NewKRatio = st.getvalue("NewKRatio");
>
> bo.addcustommetric("NewKRatio", NewKRatio);
>
> }
>
> Thanks
>
> Dave
>
