jpountz commented on a change in pull request #758: LUCENE-8833: Add a #load()
method to IndexInput to allow preloading file content into physical memory
URL: https://github.com/apache/lucene-solr/pull/758#discussion_r299462597
##########
File path:
lucene/core/src/java/org/apache/lucene/store/ByteBufferIndexInput.java
##########
@@ -491,4 +492,14 @@ protected ByteBufferIndexInput buildSlice(String
sliceDescription, long ofs, lon
return super.buildSlice(sliceDescription, this.offset + ofs, length);
}
}
+
+ @Override
+ public boolean load() {
+ for (ByteBuffer buffer : buffers) {
+ if (buffer instanceof MappedByteBuffer) {
+ ((MappedByteBuffer) buffer).load();
Review comment:
This would load more than necessary in the case of a slice. If you look at
ByteBufferIndexInput#buildSlice, the offset of the first ByteBuffer might not
be updated, so calling load() this way might load up to 1GB of data in main
memory that we don't need. This might be especially problematic in the case of
CFS files where all index inputs are slices of a single large file. We should
maybe override this is MultiBufferImpl to ignore the first `offset` bytes of
the first slice?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
