Heinrich Bohne created NUMBERS-120:
--------------------------------------
Summary: Major loss of precision in BigFraction.doubleValue() and
BigFraction.floatValue()
Key: NUMBERS-120
URL: https://issues.apache.org/jira/browse/NUMBERS-120
Project: Commons Numbers
Issue Type: Bug
Components: fraction
Affects Versions: 1.0
Reporter: Heinrich Bohne
The method {{BigFraction.doubleValue()}} calculates the double value of
fractions with numerators or denominators that, when converted to a {{double}},
round up to {{Double.POSITIVE_INFINITY}}, by right-shifting both the numerator
and denominator synchronously until both numbers fit into 1023 bits. Apart from
the fact that the maximum number of bits an integer representable as a finite
{{double}} can have is 1024 (an unbiased exponent of 1023, which is the largest
possible unbiased exponent of a {{double}} number, means 1.xxxx ⋅ 2^1023^,
which amounts to 1024 bits), this way of converting the fraction to a
{{double}} is incredibly wasteful with precision if the numerator and
denominator have a different bit length, because the smaller of the two numbers
will be truncated beyond what is necessary to represent it as a finite
{{double}}. Here is an extreme example:
The smallest integer that rounds up to {{Double.POSITIVE_INFINITY}} when
converted to a {{double}} is 2^1024^ - 2^970^. This is because
{{Double.MAX_VALUE}} as an integer is a 1024-bit number with the most
significant 53 bits set to 1 and all other 971 bits set to 0. If the 970 least
significant bits are changed in any way, the number will still round down to
{{Double.MAX_VALUE}} as long as the 971st bit remains 0, but as soon as the
971st bit is set to 1, the number will round up to {{Double.POSITIVE_INFINITY}}.
The smallest possible denominator greater than 1 where a single right-shift
will cause a loss of precision is 3. 2^1024^ - 2^970^ is divisible by 3, so
in order to create an irreducible fraction, let's add 1 to it:
(2^1024^ - 2^970^ + 1) / 3 ≈ 5.992310449541053 ⋅ 10^307^ (which can be
verified with {{BigDecimal}}, or, more easily, with [this online
tool|https://www.wolframalpha.com/input/?i=(2%5E1024+-+2%5E970+%2B+1)+%2F+3].
However, the current implementation of BigFraction.doubleValue() returns
8.98846567431158 ⋅ 10^307^, which differs from the correct result by a
relative error of 50%! The same problem applies to the method
{{BigFraction.floatValue()}}.
This can be prevented by truncating the numerator and denominator separately,
so that for each of the two numbers, the maximum possible precision is retained.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
