Mich, 
See answer below:
Paul.
________________________________

From: [email protected] [mailto:[EMAIL PROTECTED] On
Behalf Of Tom Tom
Sent: Saturday, 2 December 2006 7:34 AM
To: [email protected]
Subject: Re: [amibroker-ts] FW: Sample Data Range, Sampling Rate, and Window
Effect on Au



Nice function !
Thanks Paul.

I don't have any C compilor here (i will try to make translate it in VB 
maybe).

>>>> You can download one here for free -
http://www.bloodshed.net/devcpp.html
>>>>

I try to anderstand... you want to compute noise varaince for each order and

for each length...
and from this find the better fit by minimising error variance.
I agree this is really a good technics i think to find optimum length.
And like you and Fred, yes i find (gaphicaly, even on simple sinus 
theorical) that include one peak more of a cycle (just one data before the 
peak, so the cross with 0 from the derivation is include) significantly 
improve frequency matching (just one data difference ! but this data include

the reversal of the derivation sign).

>>>> If I understand this correctly, you use linregslope  or  
>>>> something like that to find a peak in the price data and 
>>>> set your length that way??? I don’t undetstand the bit about
>>>> "just one data before the peak....." do you mean "one extra bar
>>>> of price data before the peak/trough?"

But this technic can be very long to compute.
You say you code a way so lenght is recursuive and take back previous 
calculation in account.
I read fastly the code (i don't konw well C, but there is just FOR loop and 
arythmetic so i hope i have well anderstand your code ; ) ), and fin that 
you keep same Nom and Denom value during length variation...
Keep "end" to "maxlen", but just modifiing "start" by shifting it from 
"stepwidth" at each iteration ?
Problem is end stay always the same and so there are overlapping...

>>>>  It doesn’t overlap. At the end of each loop the "end" variable takes
>>>>  on the value of "start" before the next recursion and p, num, denom
>>>>  keep adding to the value of the last recursion.

example : 10 step, 100 stepwidth
n=0 -> start = 1000+i, end = maxlen
n=1 -> start = 900+i, end = maxlen
....
n=step-1 -> start = 100+i, end = maxlen
n=step -> start = 0+i, end = maxlen

>>>>> n = 0 -> start = 1000 + I, end = maxlen;
>>>>> n = 1 -> start =900  + I, end = 1000 + I
>>>>> n = step - 1 -> start = 100 + I, end = 200 + I
>>>>> n = step -> start= I, end = 100 + I;
>>>>> of course, I = 0 when p is considered.


And Nom and Denom are not rest to 0 between each loop (to conserve previous 
data) :
for n = 0 all is ok you compute Den/Nom between [1000+i;maxlen]
for n = 1 ... hum you compute Den/Nom between [900+i;maxlen] + 
[1000+i;maxlen], so [1000+i;maxlen] is counted 2 times,
....etc...

>>>> n = 0 -> denom, num and p computes the sum between 1ooo+I and maxlen-1;
>>>> n = 1 -> denom, num and p computes the sum between 900+I and 1000+i-1
and adds to the existing values
>>>> n = 2 -> computes the sum between 800+I and 900+i-1 and adds to the
existing values,
>>>> which is equivalent to the sum of 800+I to maxlen-1.

So "end" should change so no overlapping act ?
Problem maybe, if end is modified is that new update of f/b are not taken in

account (f/b should be different depending the lentgt and the order)...
Seems difficult to make recursion on the length...
Maybe you can explain me better the code if i am wrong (C is not my "dada" i

don't know the word for this expression in english... "C is not my horse" 
for direct translation héhé)
if we need to make computation start back from 0 for each length, will be 
heavy yes.
For AR length it can be shortcut during recursion.

>>>> I hope this sorts out the confusion
>>>> there isnt a big speed penality for doing multiple lengths

In my code i test all AR order between 1 and MaxOrder, but it is recursive 
so you can stop it when you thing erreur is enough small (but maybe there is

a higher order where error is more small... but we would never know it 
because we stop recursion earlier...). This can be a compromise.


>>>> I computes coeff to all the max orders and maxlen and then do a simple
comparsion
>>>> to find the optimium value based the selection criteria, below is a
snippets of
>>>> code to illustrate its use

memcof(inputseries, maxlen, steps, stepwidth, degree, ar, xms);

order = 1;      
double tmp;
len = minlen;
error = ErrorCalc(xms[0][1], minlen, 1);
for(n = 0; n < steps; n++)
{
        int length = minlen + n * stepwidth;
        for(i = 1; i < degree + 1; i++)
        {
                if((tmp = ErrorCalc(xms[n][i], length, i)) < error )
                {
                        error = tmp;
                        order = i;
                        len = length;
                }
        }
}
n = (len - minlen)/stepwidth;
coefficients[0] = xms[n][order];

>>>>> end of code

Cheers,
Mich.

----- Original Message -----
From: Paul Ho
To: [email protected] <mailto:amibroker-ts%40yahoogroups.com> 
Sent: Friday, December 01, 2006 4:04 PM
Subject: RE: [amibroker-ts] FW: Sample Data Range, Sampling Rate, and Window

Effect on Au

I have finally decided on automating length selection through minimising the
forward prediction errors, just like auto selections of orders in linear
prediction coeffs. The results oftens comes out at reasonably close to
select a length to include a point just before a move takes place as Fred
has suggested previously. There are few computing penalties as each
iterations for one length forms the bases for the next length. Most of the
work is done through memcof which is mostly rewritten.

Feel free to use it for your own purposes.

As to my point about using d(s(f)/df - yes, I Know both mich and myself are
using deviation or differencing to find the peak. But I was referring to
doing it mathmatically by solving the equations d(s(f)/df = 0 to obtain an
equation for f which can be calculated directly to reduce the processing
time. Then we might be able to talk about backtesting and so on. It is fast
enough for charts right now but that’s about all. Dll certain has speed
advantage and I agree its about 3 - 5 times, but also allows flexibility in
DSP work such as setting dimensions of vectors and using double precision.
Combining both AFL and dll is a powerful development environment. Scripts
have their place too, especially in dealing with com objects, but I am
hopeless in debugging scripts.

//
****************************************************************************
***********************************
// * This inline function calculates AR coeffs and delivers them in a 3D
array (len, degree, iterations) and
// * noise variances in a 2D array (len, degree). It is expected that the
optimium values lie within.
// * The orginal source code from Numeric Recipes is heavily modified.
// * Input : a vector of data[0...maxlen -1]; maxlen - max length of the
vector being considered.
// * Input : steps - No of steps between the minimum length and max length
stepwidth - No of bars in each step.
// * Input : degree - maximum order begin considered.
// * Output : through pointers to two vectors ar and noisevariance.
// *
//
****************************************************************************
***********************************
inline void memcof(float *data, int maxlen, int steps, int stepwidth, int
degree, double ***ar, double **noisevariance)
{
int i,j, n, start = 0;
double p=0.0;

double *ef= new double[maxlen];
double *eb= new double[maxlen];
double *bshift = new double[maxlen];
double *refl = new double[degree + 1];

memset(refl, 0, (degree+1) * sizeof(double));
memset(ef, 0, maxlen* sizeof(double));
memset(eb, 0, maxlen* sizeof(double));
memset(bshift, 0, maxlen* sizeof(double));

int end = maxlen;
//
// When iterating through the steps, the first loop starts with
// processing data of min length
// in the 2nd loop it moves back one step and process upto the
beginning of
// first loop. 3rd loop moves back another step and so on.
//
for(n = 0; n < steps+1; n++)
{
start = (steps - n) * stepwidth;
for (j = start;j < end ;j++)
{
p += SQR(data[j]);
ef[j] = data[j];
eb[j] = data[j];
}
noisevariance[n][0] = p/(maxlen - start);
end = start;
}

for (i = 1;i< degree + 1;i++)
{
double num=0.0,denom=0.0;
end = maxlen;
for(n = 0; n <= steps; n++)
{
start = (steps - n) * stepwidth + i; // advance
start by i
for (j = start;j < end ;j++)
{
bshift[j] = eb[j-1];
num += ef[j]*bshift[j];
denom += SQR(ef[j])+SQR(bshift[j]);
}
refl[i]=2.0*num/denom;
noisevariance[n][i] = noisevariance[n][i -1] *
(1.0-SQR(refl[i]));

for (j= 1;j < i ;j++)
{
ar[n][i][j] = ar[n][i -1][j] - ar[n][i-1][i
- j] * refl[i];
}
ar[n][i][i] = refl[i];
for (j=start;j < end ;j++)
{
eb[j] = bshift[j] - refl[i] *ef[j];
ef[j] -= refl[i] * bshift[j];
}
end = start;
}
}

delete []eb;
delete []ef;
delete []refl;
delete []bshift;
return;
}

________________________________

From: [email protected] <mailto:amibroker-ts%40yahoogroups.com>
[mailto:[email protected] <mailto:amibroker-ts%40yahoogroups.com>
] On
Behalf Of Tom Tom
Sent: Monday, 27 November 2006 5:48 PM
To: [email protected] <mailto:amibroker-ts%40yahoogroups.com> 
Subject: RE: [amibroker-ts] FW: Sample Data Range, Sampling Rate, and Window
Effect on Au

Hello !

1- Yes, I agree MEM dominant cycle (elhers) and MEM classic don't give same
result for main cycle.
But elhers for dominant cycle use Homondym (something like that... i don't
remember me well : ) ) discriminator. So it seems to be different. It
doens't use modeled data... but extract directly dominant cycle.
Dominant cycle can be used for every bar, but yes MEM with AR modeling is
more computer intensive, you are right ! I am working on code it in DLL,
seems 2 or 3 times faster than AFL, not so much. For back testing it should
take long time. But for computing it only for one bar (i.e: trading on bar
close for exmple) it is ok i think (maybe 1/10 sec to compute forecast...).

And problem for data length = n * yesterday’s cycle, is that yesterday
cycle depend from data length yesterday too... so it is recursive and the
problem is always here.

Maybe compute the length = one dominant cycle from elher ? (but dominante
cycle need peridos parameter too ?).

>>>

I think practical way is maybe to know how long we keep each trade line in
our account. For exemple for intraday trading if we keep position around 50
min in average, so if chart is 5 min bar, we should at least use 10 datas.
Maybe no more so that way we only are interesting in cycle which have impact

on our trading time horizon of 50 min.

-> For d(S(f))/df it is already in the AFL i posted to extract dominant
cycles.
It keep only the cycles which have more power than the noise (i take the
noise variance as refence to compute the spectrum in decibels). After i use
RegLinSlope to find when it is zero so it is a maximum like you suggest it.
This way there is often between 1 and 6 cycles.
Fred, you said were saying you find around same number of dominant cycle too

in one of your previous mail if i remember well...
This way is A LOT MORE POWERFULL than just elhers dominant cycle. Just try 3

theorical added sinus... dominant cycle is lost... but by taking the 1 to 6
six dominant cycle from AR spectrum you find back the 3 main frequency
without any pbl.

2- Good links thanks.
I have to read a little about it...

3- For downsampling, i didn't tryed for now. But i want to experiment non
linear (teking volatility and volume like i explained it before). I will
post about it after i done it, but for now many work.

Cheers,
Mich.

----- Original Message -----
From: Paul Ho
To: [email protected] <mailto:amibroker-ts%40yahoogroups.com>
ts%40yahoogroups.com>
Sent: Wednesday, November 22, 2006 1:25 PM
Subject: RE: [amibroker-ts] FW: Sample Data Range, Sampling Rate, and Window

Effect on Au

Mich,

I guess my last post really discuss two separate but somehow related issues

1. How to programmatically determine the data length used for MEM

a. John Elhers discussed in his book Mesa and Trading Market cycles
p.88 what he used to determine data length. In the beginning he maximised
the SNR to establish to optimum data length. Later on he set “todays” data
length to “yesterday” cycle length, assuming that it requires one cycle’s
worth of data to make a good measurement. Mich, you have said in your
previous post that perhaps ½ cycle will suffice according to your
experience. We of course can set today data length = n * yesterday’s cycle
period; There is also a practical problem in calculating dominant cycles. It
is computing intensive for every bar. I wonder if it can be calculated more
directly, for example using d(S(f))/df = 0. any views.

b. When compared the dominant cycle plot using MEM and that using
Elher’s formula , the result are quite different. MEM doesn’t always
produced a dominant cycle for every bar.

2. How to deal with non-stationary signals

a) My knowledge about Hilbert is fairly limited (left University more
than a quarter of a century ago) is mainly from reading Elhers’ Rocket
Science for traders. (pg 65) it is only about 10 pages – I am willing to
photocopy it send it to if its relevant. Another site I’ve reading is
http://wwwhitech.technion.ac.il/~feldman/intro.html
<http://wwwhitech.technion.ac.il/~feldman/intro.html> 
<http://wwwhitech.technion.ac.il/~feldman/intro.html
<http://wwwhitech.technion.ac.il/~feldman/intro.html> > and
http://lucid.igpp.ucla.edu/lessons/ess265/2005/hilbert_huang_transformation.
<http://lucid.igpp.ucla.edu/lessons/ess265/2005/hilbert_huang_transformation
.> 
<http://lucid.igpp.ucla.edu/lessons/ess265/2005/hilbert_huang_transformation
<http://lucid.igpp.ucla.edu/lessons/ess265/2005/hilbert_huang_transformation
> 
>
pdf which provides some sort of introduction. EMD may be interesting.

3. The effect of downsampling on Spectral density

a) It seems that down sampling often provides interesting result and
clearer spectral peaks which are otherwise masked. Have you tried it

3. I’ll have a look at your suggestions later on. Have you used any of
it so far.

Cheers

Paul.

__________________________________________________________
Live.com: le nouveau moteur de recherche par Messenger!
http://www.windowslive.fr/livecom/ <http://www.windowslive.fr/livecom/>
<http://www.windowslive.fr/livecom/ <http://www.windowslive.fr/livecom/> >

Send instant messages to your online friends http://au.messenger.yahoo.com
<http://au.messenger.yahoo.com> 

__________________________________________________________
Live.com : le moteur de recherche aux réponses bien carrossées! 
http://www.windowslive.fr/livecom/ <http://www.windowslive.fr/livecom/> 



 

Send instant messages to your online friends http://au.messenger.yahoo.com 

Reply via email to