[ https://issues.apache.org/jira/browse/LUCENE-8673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16762424#comment-16762424 ]
ASF subversion and git services commented on LUCENE-8673: --------------------------------------------------------- Commit 5d1d6448b91abdebbc68b54b062340c5baa88f4c in lucene-solr's branch refs/heads/master from Ignacio Vera [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=5d1d644 ] LUCENE-8673: Use radix partitioning when merging dimensional points instead of sorting all dimensions before hand. > Use radix partitioning when merging dimensional points > ------------------------------------------------------ > > Key: LUCENE-8673 > URL: https://issues.apache.org/jira/browse/LUCENE-8673 > Project: Lucene - Core > Issue Type: Improvement > Reporter: Ignacio Vera > Priority: Major > Attachments: Geo3D.png, Geo3D.png, Geo3D.png, LatLonPoint.png, > LatLonPoint.png, LatLonPoint.png, LatLonShape.png, LatLonShape.png, > LatLonShape.png > > Time Spent: 5h 20m > Remaining Estimate: 0h > > Following the advise of [~jpountz] in LUCENE-8623I have investigated using > radix selection when merging segments instead of sorting the data at the > beginning. The results are pretty promising when running Lucene geo > benchmarks: > > ||Approach||Index time (sec): Dev||Index Time (sec): Base||Index Time: > Diff||Force merge time (sec): Dev||Force Merge time (sec): Base||Force Merge > Time: Diff||Index size (GB): Dev||Index size (GB): Base||Index Size: > Diff||Reader heap (MB): Dev||Reader heap (MB): Base||Reader heap: Diff > |points|241.5s|235.0s| 3%|157.2s|157.9s|-0%|0.55|0.55| 0%|1.57|1.57| 0%| > |shapes|416.1s|650.1s|-36%|306.1s|603.2s|-49%|1.29|1.29| 0%|1.61|1.61| 0%| > |geo3d|261.0s|360.1s|-28%|170.2s|279.9s|-39%|0.75|0.75| 0%|1.58|1.58| 0%| > > edited: table formatting to be a jira table > > In 2D the index throughput is more or less equal but for higher dimensions > the impact is quite big. In all cases the merging process requires much less > disk space, I am attaching plots showing the different behaviour and I am > opening a pull request. > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org