[
https://issues.apache.org/jira/browse/LUCENE-7371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adrien Grand updated LUCENE-7371:
---------------------------------
Attachment: LUCENE-7371.patch
Good point, here is an updated patch. I am getting the following indexing
times, which suggests that it does not hurt:
{noformat}
278.064423505 sec to index part 0 // master
270.492947789 sec to index part 0 // patch
{noformat}
The index size is also unchanged, which was expected since the previous
heuristic should be equivalent with dense data.
bq. Also, I noticed TestBackwardsCompatibility seems not to test points! I'll
go fix that ...
Ouch, good catch. Thanks!
> BKDReader could compress values better
> --------------------------------------
>
> Key: LUCENE-7371
> URL: https://issues.apache.org/jira/browse/LUCENE-7371
> Project: Lucene - Core
> Issue Type: Bug
> Reporter: Adrien Grand
> Assignee: Adrien Grand
> Priority: Minor
> Attachments: LUCENE-7371.patch, LUCENE-7371.patch, LUCENE-7371.patch
>
>
> For compressing values, BKDReader only relies on shared prefixes in a block.
> We could probably easily do better. For instance there are only 256 possible
> values for the first byte of the dimension that the values are sorted by, yet
> we use a block size of 1024. So by using something simple like run-length
> compression we could save 6 bits per value on average.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
