Churky, you have complete missed my point about IEEE-754. By no means
is it"a way to tell people that I can get a whole number from an
arithmetic" -- whatever that was really supposed to mean.
IEEE-754 is a standard for floating point computations. It describes
the required accuracy for a wide var
Admittedly, my example of precision calculation is overly simplified.
In a real case, one would use % precision instead of absolute
precision.
--
You received this message because you are subscribed to the Google
Groups "Android Developers" group.
To post to this group, send email to android-deve
Arithmetic error? Bug?
Well, sort of, but no.
Sort of, because at first sight it looks wrong. At least from a purely
mathematical perspective and because we kind of think computers are
purely methematical.
But no, because this is not even a computer issue. Take for example
the problem of dividin
Yes, that's how it works in the wonderful world of computers.
Floating point math is inexact, because numbers are represented in binary
base, and those fractional numbers that look nice and round in decimal often
don't have nice round representations in binary.
So rounding is inherent, at least i
I was like to make a small correction as I posted the 2 numbers in
correctly.
My test values where 102 * 0.0254 which yields 2.5908, while i get a
value of 2.59079998
Sorry for the incorrect information during the first post, as i was
still busy confirming this problem with multiple peopl
So according to Koysta, The IEEE-754, is a way to tell people that I
can get a whole number from an arithmetic, but Instead I am going to
get a trailing slightly off value? So that means IEEE-754 is the
answer to the US space shuttle Challenger exploding? And it is ok and
acceptable? And I am talki
As Kostya already commented, this is rounding error. The error is well
within the acceptable range for IEEE-754 floating point
multiplication, since the inaccuracy is still only 2 parts in 10^13.
Still, it is a little embarrassing, considering that Python also
implements IEEE-754, and gets it exac
7 matches
Mail list logo
