uschindler commented on PR #16684: URL: https://github.com/apache/lucene/pull/16684#issuecomment-5831885275
To me this reads: It should be fine to have multiple mappings with different madvise. If a page is in cache already it won't affect anything (it just reads from cache). But if one of the call paths cause a page fault, the "handling of that page fault" is done using the madvise of the mapping. So indeed the merging code will preload pages due to sequential, but if a search thread causes a page fault, it will only load the requested page. As the last section says, if only have an impact gloablly if you use `MADV_PAGEOUT` to remove pages from cache (which is Linux specific and not POSIX). So I agree, opening the files two times is fine. In fact we could also change the whole thing to maybe use the same open file, but remap a second time to another virtual address. But as MMapDirectory closes the file directly after mapping, this won't work anyways. So basically I think your approach should work. I think your approach with opening a seperate FD and mapping for merging is fine, but we should still look at how it behaves in production. For NIOFSDirectory we need to make sure that we don't run out of file handles! Maybe also leave a comment in #13920. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
