[
https://issues.apache.org/jira/browse/LUCENE-10004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir resolved LUCENE-10004.
----------------------------------
Resolution: Not A Problem
We absolutely need to flush pending docs before we start copying bytes. This is
where I hate bulk merge optimizations.
If we can't keep it simple and clean, maybe we need to remove this bulk merge
and associated stored fields compression.
> Delete unnecessary flush in
> Lucene90CompressingStoredFieldsWriter.copyChunks() to reduce dirty chunks
> -----------------------------------------------------------------------------------------------------
>
> Key: LUCENE-10004
> URL: https://issues.apache.org/jira/browse/LUCENE-10004
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/codecs
> Affects Versions: 8.8.2
> Reporter: kkewwei
> Priority: Major
>
> In CompressingStoredFieldsWriter.merge(): if the segment meet the following
> conditions:
> {code:java}
> else if (matchingFieldsReader.getCompressionMode() == compressionMode &&
> matchingFieldsReader.getChunkSize() == chunkSize &&
> matchingFieldsReader.getPackedIntsVersion()
> ==PackedInts.VERSION_CURRENT &&
> liveDocs == null &&
> !tooDirty(matchingFieldsReader)) {
> ......
> // flush any pending chunks
> if (numBufferedDocs > 0) {
> flush();
> numDirtyChunks++; // incomplete: we had to force this flush
> }
> ......
> }
> {code}
> We will copy the the all chunk to the new fdt, before copying the chunk, we
> will flush the buffer docs if numBufferedDocs >0, but the flush is
> unnecessary.
> The bufferedDocs in memory have nothing to do with copyChunk. We just need to
> ensure that it will be flush at the end of merge(In finish()).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
