I believe I've seen a similar condition a few times. A segments file referring zero-length segment files after a disk full event.
On Fri, Jul 9, 2010 at 13:37, Michael McCandless <[email protected]> wrote: > I responded on the original thread. > > Disk full should never cause index corruption except on very old > versions of Lucene... > > Mike > > On Thu, Jul 8, 2010 at 10:40 PM, Lance Norskog <[email protected]> wrote: >> I think he saying that there is a race condition involving writing out >> the buffered changes from ram, doing deletes, doing commits and >> writing out new segment files when Lucene runs out of disk. This race >> condition (or possibly a swallowed I/O Exception) may cause bogus >> segment files, and the index becomes unuseable even though the >> existing files make up a clean index. >> >> On Thu, Jul 8, 2010 at 5:29 PM, Lance Norskog <[email protected]> wrote: >>> Forwarded to lucene-dev as a possible bug. >>> >>> On Wed, Jul 7, 2010 at 7:12 PM, Li Li <[email protected]> wrote: >>>> I use SegmentInfos to read the segment_N file and found the error is >>>> that it try to load deletedDocs but the .del file's size is 0(because >>>> of disk error) . So I use SegmentInfos to set delGen=-1 to ignore >>>> deleted Docs. >>>> But I think there is some bug. The logic of write my be -- it first >>>> writes the .del file then write the segment_N file. But it only write >>>> to buffer and don't flush to disk immediately. So when disk full. it >>>> may happen that segment_N file is flushed but del file faild. >>>> >>>> 2010/7/8 Lance Norskog <[email protected]>: >>>>> If autocommit does not to an automatic rollback, that is a serious bug. >>>>> >>>>> There should be a way to detect that an automatic rollback has >>>>> happened, but I don't know what it is. Maybe something in the Solr >>>>> MBeans? >>>>> >>>>> On Wed, Jul 7, 2010 at 5:41 AM, osocurious2 <[email protected]> >>>>> wrote: >>>>>> >>>>>> I haven't used this myself, but Solr supports a >>>>>> http://wiki.apache.org/solr/UpdateXmlMessages#A.22rollback.22 rollback >>>>>> function. It is supposed to rollback to the state at the previous >>>>>> commit. So >>>>>> you may want to turn off auto-commit on the index you are updating if you >>>>>> want to control what that last commit level is. >>>>>> >>>>>> However, in your case if the index gets corrupted due to a disk full >>>>>> situation, I don't know what rollback would do, if anything, to help. You >>>>>> may need to play with the scenario to see what would happen. >>>>>> >>>>>> If you are using the DataImportHandler it may handle the rollback for >>>>>> you...again, however, it may not deal with disk full situations >>>>>> gracefully >>>>>> either. >>>>>> -- >>>>>> View this message in context: >>>>>> http://lucene.472066.n3.nabble.com/index-format-error-because-disk-full-tp948249p948968.html >>>>>> Sent from the Solr - User mailing list archive at Nabble.com. >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Lance Norskog >>>>> [email protected] >>>>> >>>> >>> >>> >>> >>> -- >>> Lance Norskog >>> [email protected] >>> >> >> >> >> -- >> Lance Norskog >> [email protected] >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > -- Kirill Zakharenko/Кирилл Захаренко ([email protected]) Phone: +7 (495) 683-567-4 ICQ: 104465785 --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
