On Sun, 22 Dec 2002, Jay Warner wrote: > check your calculation for the sums of squares. They _cannot_ be > negative. > > the computational form appears to allow a negative value, but if you > transform it mathematically into the definitional form (or the > reverse) you will see that: the definition form cannot be negative, > therefore the computational form cannot.
This is true in algebra. It may not be true of the results of computations carried out on digital approximations to the values represented in the algebra. In particular, if the data require more precision (technically, more digits carried along in the arithmetic operations) than the computer supplies, the result of an arithmetic operation may not be accurate even to 1 digit's precision: indeed, one may be unable to tell the _sign_ of the result in a subtraction. As my earlier posts to Ronny indicated, I strongly suspect that's what the problem is here. (In some early work of mine, cited below, I found it entirely possible to get "results" from the computer that were not just imprecise but were wrong by several orders of magnitude as well as being of the wrong sign.) It is of course possible to circumvent this problem altogether, by using algorithms that implement something like the definitional formulas rather than the old-fashioned "computational" formulas; but the people who programmed the statistical routines in Excel, from all the reports I've seen, apparently were blissfully unaware of the existence of the potential problem. Anyone interested in this phenomenon (effects of digital precision) in more detail may wish to consult my 1969 Ph.D. thesis (Cornell), an abbreviated version of which was published in JSCS (the Journal of Statistical Computation and Simulation?) about 1973 +/- 2: "Computer-generated errors in statistical analyses: Variances and covariances". -- Don. ----------------------------------------------------------------------- Donald F. Burrill [EMAIL PROTECTED] 56 Sebbins Pond Drive, Bedford, NH 03110 (603) 626-0816 [was: 184 Nashua Road, Bedford, NH 03110 (603) 471-7128] . . ================================================================= Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at: . http://jse.stat.ncsu.edu/ . =================================================================
