uschindler edited a comment on pull request #518: URL: https://github.com/apache/lucene/pull/518#issuecomment-1084758218
Thanks @rmuir for the clirifaction. To add, because also @mcimadamore asked: We use shared segments, because we only allocate and map the segment once. It is then used by multiple threads. The IndexReader only opens every new file only once and then mmap the files. Several search threads may access the mapped files concurrently (very small files are kept on heap until the next commit, this is what NRT caching directory does). Every search thread may use a clone of the IndexInput because the IndexInput has per-thread state information (like a read position), but the underlying memory segments are reused and the Indexreader only closes the "main" IndexInput. Possible clones will get invalid. On changes of index and after final commit, new files are writen to disk and fsynced (including the directory metadata). IndexWriter gets reopened and mmaps new files seen and releases old and no longer used ones by a close. Any thread that still accesses already closed files will get AlreadyClosedException (previously those may have sigsegfaulted due to forceful unmapping of MappedByteBuffer). With MMapDirectory using MemorySegments this is detected by IllegalStateException and transformed to AlreadyClosed and seen by search threads. So all is sane. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
