I was like to make a small correction as I posted the 2 numbers in correctly.
My test values where 102 * 0.0254 which yields 2.5908, while i get a value of 2.5907999999999998 Sorry for the incorrect information during the first post, as i was still busy confirming this problem with multiple people on different devices. On Sep 4, 2:43 am, Indicator Veritatis <[email protected]> wrote: > As Kostya already commented, this is rounding error. The error is well > within the acceptable range for IEEE-754 floating point > multiplication, since the inaccuracy is still only 2 parts in 10^13. > > Still, it is a little embarrassing, considering that Python also > implements IEEE-754, and gets it exactly right: 100.0 * 0.0254 yields > 2.54. I wonder which rounding modes (there are 5 different choices in > IEEE-754) Python and Android (resp.) chose to implement. > > In fact, a few casual Google searches, such as "floating point > rounding Android", yield nothing about what choices they made when > implementing IEEE-754. It doesn't even yield who is responsible for > this: the OEM or the authors of Android itself. > > BTW: I didn't check too thoroughly, but it looks to me like despite > appearances, 0.254 is not exactly representable in binary -- it has > lots of trailing zeros followed by yet more digits. > > On Sep 3, 2:59 pm, Churky <[email protected]> wrote: > > > > > When Performing a division or multiplication. The value does not equal > > to the arithmetic. > > example i have: > > > double value = 100 * .0254; > > > in real life equals = 2.5908 > > > While android calculates it to: 2.5907999999999998 which is wrong. > > > I have verified this in the emulator using multiple machines, and > > multiple version of android. -- You received this message because you are subscribed to the Google Groups "Android Developers" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/android-developers?hl=en
