"Yonik Seeley" <[EMAIL PROTECTED]> wrote: > On 11/4/07, Michael McCandless <[EMAIL PROTECTED]> wrote: > > The problem is, on a hard shutdown (kill -9 or JVM/machine crashes), > > apparently future operations may have completed while some past > > operations have not. For example, the new segments_N file was > > successfully written while say the _X.fdx file of the just-flushed > > segment was not successfully written, even though Lucene had written & > > closed _X.fdx before segments_N. > > That should be impossible except for a machine crash. Kill -9 or a > JVM crash should have no effect on data already written.
OK, right. JVM crashing or getting killed should preserve order-of-completion on the IO operations: those IO operations that were handed off to the OS will eventually complete successfully. But OS crashing, machine crashing or power-cord gets pulled can result in out-of-order completion of IO operations, which is what can corrupt the index. > But a sync option would be both simple and useful for people trying to > take live snapshots of an index, or to protect against machine > crashes. This isn't an absolute 100% guarantee either (so don't test > for it) - the drives often lie to the OS about data being flushed. > It's the best we can do at our level though. > http://www.google.com/search?q=fsync+drive+lies Right, the best the OS can do is get all writes out to the drives, but if the drives then cache the writes (in non-stable storage) then we are still at risk. Mike --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
