On Fri, 12 Aug 2016 09:46:38 -0700, [email protected] wrote: > > (9.999e-5 * 2e0**66).Int * 5**8 - 9999 * 2**58 > 287369 > > (9.99899999999999995e-5 * 2e0**66).Int * 5**8 - 9999 * 2**58 > -103256 > > The above computations show, scaled up, the difference between a Num value > and the exact value 9.999e-5. As you can see, the Num that arose from > the "9.99899999999999995e-5" literal was slightly closer than the Num > that arose from the "9.999e-5" literal. The closer of these two is in > fact the closest representable IEEE double precision value to 9.999e-5. > Thus this literal "9.999e-5" has not yielded the closest available Num > to its nominal value; this is a bug. glibc's strtod() handles this case > fine, yielding the closer value. > > Note that the only rounding occurring in this test case is on the > decimal->float conversion. The multiplication by a power of two, > conversion of integer-valued float to Int, and all the Int arithmetic, > are all exact. > > -zefram
as of b5aa3c5, these both output 287369 now. -- Will "Coke" Coleda
