[
https://issues.apache.org/jira/browse/LUCENE-7312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312577#comment-15312577
]
Michael McCandless commented on LUCENE-7312:
--------------------------------------------
OK indeed this is a double precision issue ... I boiled it down to this small
test:
{noformat}
public void testOneValue() throws Exception {
double DECODE = 4.661822942981865E-10;
double x = -0.31622436580292346;
int xEnc = (int) Math.floor(x / DECODE); // -678327705
double xDec = xEnc * DECODE;
// because we floor'd on encoding to xEnc, this should be true:
assertTrue("x=" + x + " xDec=" + xDec, xDec <= x);
}
{noformat}
which fails with this:
{noformat}
java.lang.AssertionError: x=-0.31622436580292346 xDec=-0.3162243658029234
{noformat}
The reason is that the value {{x / DECODE}} is very, very close to an
integer value, just a hair below it, such that when quantized, it
jumps just a bit above that int value, causing the floor to return the
"wrong" value.
You can also see it with Python's rational number module ({{fractions}}) too:
{noformat}
>>> DECODE = fractions.Fraction(4.661822942981865E-10)
>>> x = fractions.Fraction(-0.31622436580292346)
>>> math.floor(x / DECODE)
-678327706
>>> math.floor(float(x / DECODE))
-678327705
{noformat}
I.e. the true floor in this case is -678327706, but if you first
quantize to 64 bits and take the floor, you get one higher.
I think we should go back to the "safe" doubles solution we used to
have, where {{DECODE}} is the next higher double that doesn't use any of
its lower 32 bits.
I'll also port over a nice 2D test Rob pointed me to, which should be
more efficient for finding quantization issues.
> Geo3dPoint test failure
> -----------------------
>
> Key: LUCENE-7312
> URL: https://issues.apache.org/jira/browse/LUCENE-7312
> Project: Lucene - Core
> Issue Type: Bug
> Components: modules/spatial3d
> Affects Versions: master (7.0)
> Reporter: Karl Wright
> Assignee: Karl Wright
>
> Here's the failure:
> {code}
> ant test -Dtestcase=TestGeo3DPoint -Dtests.method=testEncodeDecodeRoundsDown
> -Dtests.seed=7046405B94C1716E -Dtests.multiplier=3 -Dtests.slow=true
> -Dtests.locale=da-DK -Dtests.timezone=America/Detroit -Dtests.asserts=true
> -Dtests.file.encoding=UTF-8
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
