amalatlas commented on issue #16213:
URL: https://github.com/apache/lucene/issues/16213#issuecomment-4667086176
When digging deeper into the code, I realised `fieldsStream =
d.openInput(fieldsStreamFN, context.withHints(FileTypeHint.DATA,
DataAccessHint.RANDOM));` is a no-op in the Merge path and that `NORMAL` is
actually being used when opening the file. The profiling indicates
`Lucene90CompressingStoredFieldsWriter.merge` is the culprit, so I believe this
could be originating from OpenSearch codebase than Lucene itself.
My current hypothesis is that OpenSearch is setting the `madv` to
`SEQUENTIAL` which is probably still not correct, as the merge path would read
to the end for checksum calculation and then moves back to read the stored
fields in the merge path. When it moves back kernel may have already discarded
the read pages due to aggressive `SEQUENTIAL` advise.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
