[ https://issues.apache.org/jira/browse/LUCENE-4342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Smiley updated LUCENE-4342: --------------------------------- Attachment: LUCENE-4342_fix_distance_precision_lookup_for_prefix_trees,_and_modify_the_default_algorit.patch Sure enough, there was an off by one bug in the grid level lookup. Thanks so much for the test case; I enhanced it quite a bit further. Because GeoHashUtils is in Spatial4j (quite debatable), some of the bug is still there where I need to get a fix in for Spatial4j0.3 but in the mean time I simply inlined the relevant lookup code with it fixed. I also modified the algorithm in which a specific error distance is derived from a shape and the percent. It's now half the diagonal distance of opposite corners of its bounding box. > Issues with prefix tree's Distance Error Percentage > ---------------------------------------------------- > > Key: LUCENE-4342 > URL: https://issues.apache.org/jira/browse/LUCENE-4342 > Project: Lucene - Core > Issue Type: Bug > Components: modules/spatial > Affects Versions: 4.0-ALPHA, 4.0-BETA > Reporter: Itamar Syn-Hershko > Assignee: David Smiley > Attachments: > LUCENE-4342_fix_distance_precision_lookup_for_prefix_trees,_and_modify_the_default_algorit.patch, > unnamed.patch > > > See attached patch for a failing test > Basically, it's a simple point and radius scenario that works great as long > as args.setDistPrecision(0.0); is called. Once the default precision is used > (2.5%), it doesn't work as expected. > The distance between the 2 points in the patch is 35.75 KM. Taking into > account the 2.5% error the effective radius without false negatives/positives > should be around 34.8 KM. This test fails with a radius of 33 KM. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org