I think he saying that there is a race condition involving writing out the buffered changes from ram, doing deletes, doing commits and writing out new segment files when Lucene runs out of disk. This race condition (or possibly a swallowed I/O Exception) may cause bogus segment files, and the index becomes unuseable even though the existing files make up a clean index.
On Thu, Jul 8, 2010 at 5:29 PM, Lance Norskog <[email protected]> wrote: > Forwarded to lucene-dev as a possible bug. > > On Wed, Jul 7, 2010 at 7:12 PM, Li Li <[email protected]> wrote: >> I use SegmentInfos to read the segment_N file and found the error is >> that it try to load deletedDocs but the .del file's size is 0(because >> of disk error) . So I use SegmentInfos to set delGen=-1 to ignore >> deleted Docs. >> But I think there is some bug. The logic of write my be -- it first >> writes the .del file then write the segment_N file. But it only write >> to buffer and don't flush to disk immediately. So when disk full. it >> may happen that segment_N file is flushed but del file faild. >> >> 2010/7/8 Lance Norskog <[email protected]>: >>> If autocommit does not to an automatic rollback, that is a serious bug. >>> >>> There should be a way to detect that an automatic rollback has >>> happened, but I don't know what it is. Maybe something in the Solr >>> MBeans? >>> >>> On Wed, Jul 7, 2010 at 5:41 AM, osocurious2 <[email protected]> >>> wrote: >>>> >>>> I haven't used this myself, but Solr supports a >>>> http://wiki.apache.org/solr/UpdateXmlMessages#A.22rollback.22 rollback >>>> function. It is supposed to rollback to the state at the previous commit. >>>> So >>>> you may want to turn off auto-commit on the index you are updating if you >>>> want to control what that last commit level is. >>>> >>>> However, in your case if the index gets corrupted due to a disk full >>>> situation, I don't know what rollback would do, if anything, to help. You >>>> may need to play with the scenario to see what would happen. >>>> >>>> If you are using the DataImportHandler it may handle the rollback for >>>> you...again, however, it may not deal with disk full situations gracefully >>>> either. >>>> -- >>>> View this message in context: >>>> http://lucene.472066.n3.nabble.com/index-format-error-because-disk-full-tp948249p948968.html >>>> Sent from the Solr - User mailing list archive at Nabble.com. >>>> >>> >>> >>> >>> -- >>> Lance Norskog >>> [email protected] >>> >> > > > > -- > Lance Norskog > [email protected] > -- Lance Norskog [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
