Alex Herbert created NUMBERS-204:
------------------------------------
Summary: Sum class may incorrectly sum products if their round-off
is sub-normal
Key: NUMBERS-204
URL: https://issues.apache.org/jira/browse/NUMBERS-204
Project: Commons Numbers
Issue Type: Bug
Components: core
Affects Versions: 1.1
Reporter: Alex Herbert
Assignee: Alex Herbert
The Sum class can be used to compute a sum of products with extended precision
by computing the round-off component of each product and using that in the sum.
When the round-off is sub-normal this computation may be incorrect as
intermediate sub-normal numbers may lose significant bits.
This test currently fails due to a 1 ULP rounding error in computing the
round-off. This is combined with the exact product to create an incorrect sum.
{code:java}
@ParameterizedTest
@CsvSource({
// Round-off == 0.0
"-2.73551683292218E-154, -1.0861547555023299E-154",
// Round-off = 4.9E-324
"1.4134286753429383E-154, -4.1395762189346144E-154",
})
void testSumOfProduct_tiny(double x, double y) {
final Sum s = Sum.create();
s.addProduct(x, y);
Assertions.assertEquals(x * y, s.getAsDouble());
}
{code}
The productLow method currently uses downscaling if the arguments are too large
for the computation. Applying upscaling to the arguments so the product and its
round-off are both normal numbers corrects this rounding error.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
