No references -- but everything is an estimate. Unlike decimal/BCD (IEEE 854-2008, GDAS, etc) you cannot map binary precision directly to decimal precision. This is especially true since your last bits of precision will suffer as you perform the conversion to decimal. I was always told (and understood) that fast'n'cheap floating point (16-bit) was just as accurate as a slide rule.
All formats have some "extra bits" of precision which, like the slide rule, are useful for estimating the rounding of the least significant digit. For this to be true you have to have some extra bits in reserve. Generally speaking though even counting on 14 accurate decimal digits from double precision floating point is pushing the limits of precision. I'd use Quad precision for computation if I really needed accuracy to 14-16 significant digits. That doesn't mean that you can't get 15 or 16 significant digits from double precision, merely that you have to really understand the numerical methods being used to actually be certain that precision has been maintained. Then again, some people were still building things using two digit years in 1990. --- () ascii ribbon campaign against html e-mail /\ www.asciiribbon.org > -----Original Message----- > From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users- > boun...@sqlite.org] On Behalf Of Black, Michael (IS) > Sent: Sunday, 17 June, 2012 06:38 > To: General Discussion of SQLite Database > Subject: Re: [sqlite] VERY weird rounding error > > Do you have a reference for this? I found 3: > > > > Wikipedia says 16 > > http://en.wikipedia.org/wiki/IEEE_754-2008 > > BYU says 15 > > http://www.math.byu.edu/~schow/work/IEEEFloatingPoint.htm > > Oracle says 15-17 > > http://docs.oracle.com/cd/E19957-01/806-3568/ncg_math.html > > > > But I've never heard of "expected precision" and google doesn't come up with > term either -- it either has precision or it doesn't as far as I've ever > heard. And i've never heard of 14 digits before (which wouidn't surprise me > though). > > > > > > Michael D. Black > > Senior Scientist > > Advanced Analytics Directorate > > Advanced GEOINT Solutions Operating Unit > > Northrop Grumman Information Systems > > ________________________________ > From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on > behalf of Keith Medcalf [kmedc...@dessus.com] > Sent: Saturday, June 16, 2012 8:11 PM > To: General Discussion of SQLite Database > Subject: EXT :Re: [sqlite] VERY weird rounding error > > > > Ideally, you write your software so that differences in the 15th decimal > > place don't matter to you. If you want two platforms to give identical > > results you use integer arithmetic. It really depends on what kind of > > application you're writing (scientific, financial) or what your numbers > > actually represent. > > This is technically incorrect. Floating Point accuracy is in Decimal Digits, > not Decimal Places (unless, of course, the decimal formatted output is in > "Engineering Notation" in which case Decimal Digits == Decimal Places -- > Engineering Notation means that the number to the left of the decimal point > is 0). > > The "theoretical" accuracy (and the "reasonably expected" accuracy) is: > > Half (2 Bytes) ~ 3.3 2 > Single (4 Bytes) ~ 7.2 6 > Double (8 bytes) ~15.9 14 > Double extended (10 Byte) ~19.2 18 > Quad ( 16 Bytes) ~34.0 32 > > The value 9990.1 has 5 significant digits. > The output value 99990.1000000001 has 15 significant digits, of which the > expected 14 are accurate. > > Etienne, thanks for the thanks. > > --- > () ascii ribbon campaign against html e-mail > /\ www.asciiribbon.org<http://www.asciiribbon.org/> > > > > -----Original Message----- > > From: sqlite-users-boun...@sqlite.org [<thismessage:/>mailto:sqlite-users- > > boun...@sqlite.org] On Behalf Of Simon Slavin > > Sent: Saturday, 16 June, 2012 17:33 > > To: General Discussion of SQLite Database > > Subject: Re: [sqlite] VERY weird rounding error > > > > > > On 17 Jun 2012, at 12:06am, Etienne <ejlist-sql...@yahoo.fr> wrote: > > > > > Is there a trick (in the sys. libraries mentioned above, or through the > > win32 C API) for changing the way the FPU handles (long) doubles? > > > > You're at tricky low level detail now. Rounding and truncation modes used > by > > chips which implement IEEE754 are often carefully set by compiler writers > to > > get the results they want. If you mess with the settings they've chosen, > you > > can get unexpected results in other apps which were previously working > > correctly. > > > > > > It's discovering and caring about this sort of thing that separates out the > > professional programmer from the someone who shouldn't be paid for > > programming. > > > > I do appreciate your thanks, by the way. > > > > Simon. > > _______________________________________________ > > sqlite-users mailing list > > sqlite-users@sqlite.org > > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > > > > _______________________________________________ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > _______________________________________________ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
