> On Dec 17, 2018, at 1:12 PM, Keith Medcalf <kmedc...@dessus.com> wrote: > > > The "nearest" representation of 211496.26 is 211496.26000000000931323. The > two representable IEEE-754 double precision floating point numbers bounding > 211496.26 are: > > 211496.26000000000931323 > 211496.2599999999802094 > > The difference between 211496.25999999992 (which is itself a truncated > representation -- the actual stored value should presumably be > 211496.25999999992200173) and 211496.26000000000931323 is > 8.73114913702011e-11, or 3 ULP. > > Applying half-even rounding at the second "decimal place" to > 211496.25999999992 (which when multiplied by 100 is 21149625.9999999925494194 > which half-even rounds to 21149626.0 which when divided by 100 is > 211496.26000000000931323 or exactly the "nearest representable IEEE-754 > double precision floating point number" to 211496.26
Which brings up an important point…. When saying "double-precision floating point is accurate to within 15 decimal digits” it does NOT mean floating point can perfectly represent any number up to 15 digits long, it means that the delta between intent and representation is 15 digits smaller than the largest represented digit. In this case, the delta is 10 digits under the radix point, plus another 5 or 6 digits from the other side of the radix point. In short, the problem _is_ storage… or at least representation. And that’s before calculations muck everything up further. -j >> On 17 Dec 2018, at 5:16pm, James K. Lowden <jklow...@schemamania.org> >> wrote: >> >>> IEEE >>> double-precision floating point is accurate to within 15 decimal >>> digits. >> >> First, the problem is not storage it's calculation. >> _______________________________________________ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
