Dear Donald,
Thank you so much for your reply! I was worried no one would be able
to answer until Monday night...
[EMAIL PROTECTED] (Donald Burrill) writes:
> I am not an expert on time series, and perhaps other colleagues will
> help you with that part. But to respond to part of your post:
>
> On 15 Jun 2003, helen-louise wrote:
>
> > I am using the aggregate variance method to analyse a time
> > series. This method has the following steps:
...
> > a) Calculate the mean of the entire data set, <C>
> > b) Divide the data set into N bins of width T
> > c) Calculate the mean of each bin, <Ci> for i=1,2,..., N
> > d) Sum over all bins {(<Ci> - <C>)^2}
> > e) Take the square root, divide by N
>
> Are you sure about this? I would have thought you'd divide by N (or
> possibly by N-1), then take the square root; and as you conjectured
> later (below), that would indeed be a standard deviation rather than
> a variance.
Oh dear. I just double-checked the paper I got the method from, and
they have definitely included the /N in the square root. It is
possible that this is a typo, but it's a depressingly major mistake if
it is.
> > f) Repeat for next N
> >
> > You end up with a list of T values and corresponding "aggregated
> > variance" values. Plot log T against the log of this variance
> > gives a line with gradient m or gamma, which is always 0 or
> > negative. This can be used as an estimator for other constants
> > such as the Hurst exponent, eg. gamma is 2H - 2, and it is
> > regarded as an indicator of self-similarity or fractality in the
> > data.
>
> Since log (variance) = log (standard deviation, squared)
> = 2 log (standard deviation), the only problem here is that the
> gradient will be off by a factor of 2 (except that I believe, as you
> conjectured, that you have not correctly calculated the standard
> deviation, which is too small by a factor of (square root of N), I
> suspect).
Right - that agrees with my thoughts (so I haven't forgotten all my
maths!). The factor of 2 does however make a tremendous difference to
my results. The theory I'm working with says (roughly) that gamma =
-1/3 implies that the data is self-similar, and gamma = -0.5 implies
it is random. I'm not sure what difference the square root of N factor
will make off the top of my head.
> > My problem is this: I have a data set with a lot of gaps or holes
> > in the data.
>
> < snip, 5 possible ways of dealing with missing data >
>
> > Which of these techniques is correct?
>
> Probably all of them are correct, or at least acceptable; at least
> in some contexts and under some conditions. If there is a
> definitive "correct" method for your situation, perhaps someone else
> can assist you on this point. I would be inclined to ask, "What
> difference do any of them make?" If you get answers that are so
> similar as to be indistinguishable when you try applying all five
> methods, clearly the choice of method is substantively and/or
> methodologically immaterial, and you can pick the one that's easiest
> to carry out (or to justify, or is otherwise convenient). If you
> get noticeably different results, that would be interesting (and
> possibly worth a paper?), and the effort of trying to figure out why
> those differences occur might be well repaid.
Thanks for the advice! That makes me less worried about _that_ part
of the method.
> > I am also now having a minor crisis about step (e) above, if I
> > really am supposed to take the square root method from did so,
>
> That sentence fragment doesn't make a whole lot of syntactic
> sense...
Sorry - bad cut & pasting - it was supposed to say "if I really am
supposed to take the square root - the original research paper I got
the method from did so, but I can't seem to find this confirmed by any
formulae online". I am already disgruntled with that paper for not
giving the name of the technique (we spent about 18 months using it
thinking it was an entirely novel method devised by that group), and
these two mistakes in the formula are extremely annoying. This is
what happens when physicists use statistics without a statistician
to check their work - I only wish I had realised earlier that there
are newsgroups for "useful" subjects.
helen-louise
[EMAIL PROTECTED]
.
.
=================================================================
Instructions for joining and leaving this list, remarks about the
problem of INAPPROPRIATE MESSAGES, and archives are available at:
. http://jse.stat.ncsu.edu/ .
=================================================================
