Christian Hoffmann <[EMAIL PROTECTED]> writes: > > 0.1 would be stored as (1 + 0.6)*2^(-4) and 0.2 would be stored as (1 > > + 0.6)*2^(-3), whereas 0.3 would be stored as (1 + 0.2)*2^(-2). You > > should expect 56 decimal (binary?) place accuracy on 0.1, 55 place > > accuracy on 0.2, and 54 place accuracy on 0.3. It's not surprising > > weird things happen! > > I don *not* think so: all mantissas here have *52 binary* places!
(53, since the leading 1 is not stored...) Actually, in the x86 FPUs numbers get extended to 64 bits during evaluation, so after alignment for addition, the smaller numbers may have bits beyond the truncation point of the larger ones. These bits disappear when the result is stored, but rounding may be affected giving those unit-in-the-last-place differences. -- O__ ---- Peter Dalgaard Blegdamsvej 3 c/ /'_ --- Dept. of Biostatistics 2200 Cph. N (*) \(*) -- University of Copenhagen Denmark Ph: (+45) 35327918 ~~~~~~~~~~ - ([EMAIL PROTECTED]) FAX: (+45) 35327907 ______________________________________________ [EMAIL PROTECTED] mailing list https://www.stat.math.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
