[
https://issues.apache.org/jira/browse/LUCENE-7179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15226352#comment-15226352
]
Nicholas Knize commented on LUCENE-7179:
----------------------------------------
This is not an issue for existing LatLonPoint or GeoPointField because:
1. LatLonPoint uses {{0x1L << 32 / 360.0}} and handles overflow for {{90.0}}
{{180.0}} by calling {{Math.nextDown}}.
2. GeoPointField uses 31 bits and tolerance 1e-6. So overflow is not a problem.
When switching to a morton encoding which maps {{-90.0, -180.0}} to {{0L}} and
{{90.0, 180.0}} to {{0xFFFFF...}} round off error becomes a problem when points
are quantized more than once. I will add the LUCENE-7165 patch along with a
test to show this.
> GeoPoint and LatLonPoint test data should quantize once
> -------------------------------------------------------
>
> Key: LUCENE-7179
> URL: https://issues.apache.org/jira/browse/LUCENE-7179
> Project: Lucene - Core
> Issue Type: Bug
> Reporter: Nicholas Knize
>
> {{LatLonPoint}} and {{GeoPointField}} tests pre quantizes test data to ensure
> consistency with indexed (encoded) data. The pre quantized data then becomes
> indexed, undergoing another quantization. To guarantee numerical stability
> this should be changed such that the test data is quantized after indexing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
