Raul Miller-4 wrote: > > On Thu, Sep 15, 2011 at 12:18 PM, Viktor Cerovski > <[email protected]> wrote: >> You(r subtracted) mean (is) this: >> >> stddev _1e12+1e12+1e6?@$0 >> 0.288461 > > If I am allowed to know the mean of 1e12+1e6?@$0 -- if I am allowed to > treat the data as non-random -- then, yes. > > But that was not really my point. The point, I thought, was to come > up with an implementation of standard deviation that is stable, > numerically, when the mean is much larger than the deviation. > Well, that's not the point of the example. Let's see:
require'stats' x =. 1e6 ?@$ 0 stddev 1e12 + x 0.573404 NB. problem stddev@:(- mean) 1e12 + x 0.288938 NB. remedy So far so good. But it's *not* because of the large mean. Look: stddev 1e12 + x - 0.5 0.288938 Same x translated by tiny (compared to 1e12) 0.5, still large mean and the problem is gone. So, what you suggest is the correct remedy, but for the wrong reasons. > And, of course, Welford's approach does achieve that. But Welford's > algorithm assumes a scalar computing architecture. Meanwhile, it > seems to me that subtracting the computed mean achieves approximately > the same thing that Welford's approach achieves, but modularized to > eliminate the "scalar architecture" aspect. > > That said, if there are generic cases where Welford's approach behaves > better than subtracting the computed mean from the data set, I would > be interested in hearing about them. (But I doubt they exist, except > in hand crafted special cases with no general utility, because > Welford's algorithm is doing essentially the same thing, with a > running average.) > > -- > Raul > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm > > So before getting into all sorts of general questions & quests, the first thing to consider here is whether stddev@:(- mean) is more accurate than stddev in a sort of generic, standard kind of calculations. I think it is not. -- View this message in context: http://old.nabble.com/Welford%27s-method-for-standard-deviations-tp32458412s24193p32479345.html Sent from the J Programming mailing list archive at Nabble.com. ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
