Donald Burrill wrote:
> [snip]It is of course possible to circumvent this problem altogether, by > using algorithms that implement something like the definitional formulas > rather than the old-fashioned "computational" formulas; In my innocence, I would think that the definitional form would entail more rounding error, not less. > but the people > who programmed the statistical routines in Excel, from all the reports > I've seen, apparently were blissfully unaware of the existence of the > potential problem. Maybe they believe (a) it won't happen to them - people who use Excel for stats, or (b) there is little they can do about it. I have tried out one NIST sample data file on Excel, to see it get very confused in the 3rd & 2nd significant digit. that file, however, involved some very small differences in very large numbers. the data which started this discussion, I believe, had values on the order of -1.5 as a sum of square. Without the original data we can't see exactly the problem source, true? So I inferred that the numbers were not minute differences of large numbers, but an equation error. Certainly, one should double check that (naggingly simple & frustrating) alternative early on. > Anyone interested in this phenomenon (effects of digital precision) in > more detail may wish to consult my 1969 Ph.D. thesis (Cornell), an > abbreviated version of which was published in JSCS (the Journal of > Statistical Computation and Simulation?) about 1973 +/- 2: > "Computer-generated errors in statistical analyses: Variances and > covariances". Are these errors generated still valid with bigger machines? Cheers, Jay -- Jay Warner Principal Scientist Warner Consulting, Inc. 4444 North Green Bay Road Racine, WI 53404-1216 USA Ph: (262) 634-9100 FAX: (262) 681-1133 email: [EMAIL PROTECTED] web: http://www.a2q.com The A2Q Method (tm) -- What do you want to improve today? . . ================================================================= Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at: . http://jse.stat.ncsu.edu/ . =================================================================
