wwwa1ter opened a new pull request, #16690:
URL: https://github.com/apache/lucene/pull/16690
Closes #16045
## Description
The Lucene90 compressing stored fields merge currently falls back to a
doc-by-doc copy for the entire segment as soon as it has a single
deletion, even though stored fields are compressed in chunks and chunks
without deletions could still be bulk-copied without decompression.
This PR adds a hybrid merge strategy: contiguous ranges of live docs are
collected, and
- whole compressed chunks whose docs are all live are copied in bulk
without decompression, as before;
- only chunks that contain deleted docs (or a partially-consumed range
head) are merged doc-by-doc via copyOneDoc.
The hybrid strategy is enabled by default and can be disabled with the
`org.apache.lucene.codecs.lucene90.compressing.Lucene90CompressingStoredFieldsWriter.enableHybridMerge`
system property, mirroring `enableBulkMerge`.
## Tests
Unit tests in TestCompressingStoredFieldsFormat cover:
- merges with various deletion patterns (no deletions, scattered
deletions, deletions at chunk boundaries, fully-deleted chunks),
- equivalence with the doc-by-doc merge path (same docs and order),
- the enableHybridMerge system property escape hatch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
