I'm taking the liberty of posting this reply to the edstat list, as
well as to the two colleagues who have responded (and to whom I'd sent
private comments):
On Wed, 8 Jan 2003, Scott Smail wrote:
> Thanks for the reply.
>
> Sorry, I should have used variance instead of standard deviation.
>
> What I'm trying to do is to keep statistics on a computer using very
> little memory. Right now I calculate the mean recursively using the
> new value, the last mean, and the number of samples used to
> calculate the value.
>
> I have a formula that I thought would do the same for variance be
> I'm not convinced that it's right.
>
> I've been using this:
> ("last" refers to the value of the last recursive calculation)
> ("new" refers to a recursively calculated value where the new
> value has already been calculated in)
> ("Val" is the new input)
>
> newVariance = lastStandardDeviation + ( Val - newMean) * (Val - lastMean)
>
> I am not convinced this formula works right though.
Well, for starters you need "last Variance", not "lastSD".
Multiply this by "last(N-1)" to get "lastSS" where "SS" is "Sum of
Squared deviations from the mean". Then you need to add to this the
squared deviation of "Val" from the "lastMean", and make a correction
for the difference between "lastMean" and "newMean". Sorry, I'm
thinking too slowly today: I'm not sure whether your
(Val - newMean)*(Val - lastMean) has that effect or not.
(I'm also not at all sure that Jon Cryer's Minitab macro makes that
correction: it seemed to me that it did not, but perhaps Jon could
enlighten me on that point.)
But in any case you would then need to divide by "new(N-1)" to get
"newVariance".
> Do you have anything better?
I'm sure there is something; I'm not sure, without working out the
algebra from first principles, precisely what it is.
> Precision is not too important. Values of new inputs are typically between
> 0 and 10000 and two decimal places of precision is OK.
>
> Thanks
>
> -Scott
Good luck! -- DFB.
-----------------------------------------------------------------------
Donald F. Burrill [EMAIL PROTECTED]
56 Sebbins Pond Drive, Bedford, NH 03110 (603) 626-0816
[was: 184 Nashua Road, Bedford, NH 03110 (603) 471-7128]
.
.
=================================================================
Instructions for joining and leaving this list, remarks about the
problem of INAPPROPRIATE MESSAGES, and archives are available at:
. http://jse.stat.ncsu.edu/ .
=================================================================
