The "nearest" representation of 211496.26 is 211496.26000000000931323. The two representable IEEE-754 double precision floating point numbers bounding 211496.26 are:
211496.26000000000931323 211496.2599999999802094 The difference between 211496.25999999992 (which is itself a truncated representation -- the actual stored value should presumably be 211496.25999999992200173) and 211496.26000000000931323 is 8.73114913702011e-11, or 3 ULP. Applying half-even rounding at the second "decimal place" to 211496.25999999992 (which when multiplied by 100 is 21149625.9999999925494194 which half-even rounds to 21149626.0 which when divided by 100 is 211496.26000000000931323 or exactly the "nearest representable IEEE-754 double precision floating point number" to 211496.26 --- The fact that there's a Highway to Hell but only a Stairway to Heaven says a lot about anticipated traffic volume. >-----Original Message----- >From: sqlite-users [mailto:sqlite-users- >boun...@mailinglists.sqlite.org] On Behalf Of Simon Slavin >Sent: Monday, 17 December, 2018 10:36 >To: SQLite mailing list >Subject: Re: [sqlite] Question about floating point > >On 17 Dec 2018, at 5:16pm, James K. Lowden <jklow...@schemamania.org> >wrote: > >> IEEE >> double-precision floating point is accurate to within 15 decimal >> digits. > >First, the problem is not storage it's calculation. > >Second, the thread was started because a floating point calculation >in SQLite, exactly as it is run today, led to the following value: > >211496.25999999992 > >which is typical of such problems. Please don't consider that >there's no problem until you have solved that problem. > >Simon. >_______________________________________________ >sqlite-users mailing list >sqlite-users@mailinglists.sqlite.org >http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users _______________________________________________ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
