pauldoo opened a new pull request, #16787: URL: https://github.com/apache/lucene/pull/16787
Sharing this as an early RFC to allow collaboration. It is not proposed for merge: I have not yet found a compelling benchmark showing its value. This series comes in 3 commits: **Commit 1**: Adds `IndexInput.prefetchRange(..)` for asynchronous reads into an application-side buffer. Unlike the existing `prefetch`, which is a hint, this is intended to start explicit async IO, and the returned `IndexInput` is the way the bytes are read. Because the result is an `IndexInput`, callers can consume it however suits them (sequential reads, bulk reads, seek, slice), rather than having bytes copied into an array of a particular type. The default implementation checks the range and returns `null`, meaning the caller reads as it normally would. **Commit 2**: Adds `IOUringDirectory` to the sandbox module, an implementation of the above using Linux io_uring. It builds on the work in #16666 by @goankur, in particular the ring leasing. `prefetchRange` returns an `IndexInput` that owns the memory segment the io_uring read lands in; the first read call may block until that read completes. It is opt-in (`-Dlucene.store.ioUring=true`) and needs Linux, liburing (`liburing-ffi.so.2`) and `--enable-native-access`; the tests skip where io_uring is unavailable. There is an option to submit reads with [`RWF_DONTCACHE`](https://man7.org/linux/man-pages/man2/readv.2.html), which needs Linux 6.14+ and a filesystem that supports it. **Commit 3**: Example usage in `OffHeapFloatVectorValues`, showing how `prefetchRange` can be used by a caller. It is only an example and is not proposed for merge. --- The difference from #16666 is that the async read interface is exposed on `IndexInput` rather than at the vector reader level, which may allow broader use. Since `prefetchRange` returns an `IndexInput` representing the buffer, callers are not constrained in how they consume the data. This work ties with the mmap path in my benchmarks (25M documents, 1024-d Cohere v3 with binary quantization, memory-capped with a cold page cache, 4 threads). I believe there is not enough IO during reranking for the mechanism to matter, and the prefetch hint added in #16705 already overlaps the reads. Known gaps: - Returned inputs must be closed. A thread that drops them without closing pins its ring until the thread ends. A possible fix is to return rings when they are idle, rather than when all inputs have been closed. - The example integration in commit 3 assumes vectors are read in the order they were prefetched, and throws otherwise. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
