I responded on the original thread. Disk full should never cause index corruption except on very old versions of Lucene...
Mike On Thu, Jul 8, 2010 at 10:40 PM, Lance Norskog <[email protected]> wrote: > I think he saying that there is a race condition involving writing out > the buffered changes from ram, doing deletes, doing commits and > writing out new segment files when Lucene runs out of disk. This race > condition (or possibly a swallowed I/O Exception) may cause bogus > segment files, and the index becomes unuseable even though the > existing files make up a clean index. > > On Thu, Jul 8, 2010 at 5:29 PM, Lance Norskog <[email protected]> wrote: >> Forwarded to lucene-dev as a possible bug. >> >> On Wed, Jul 7, 2010 at 7:12 PM, Li Li <[email protected]> wrote: >>> I use SegmentInfos to read the segment_N file and found the error is >>> that it try to load deletedDocs but the .del file's size is 0(because >>> of disk error) . So I use SegmentInfos to set delGen=-1 to ignore >>> deleted Docs. >>> But I think there is some bug. The logic of write my be -- it first >>> writes the .del file then write the segment_N file. But it only write >>> to buffer and don't flush to disk immediately. So when disk full. it >>> may happen that segment_N file is flushed but del file faild. >>> >>> 2010/7/8 Lance Norskog <[email protected]>: >>>> If autocommit does not to an automatic rollback, that is a serious bug. >>>> >>>> There should be a way to detect that an automatic rollback has >>>> happened, but I don't know what it is. Maybe something in the Solr >>>> MBeans? >>>> >>>> On Wed, Jul 7, 2010 at 5:41 AM, osocurious2 <[email protected]> >>>> wrote: >>>>> >>>>> I haven't used this myself, but Solr supports a >>>>> http://wiki.apache.org/solr/UpdateXmlMessages#A.22rollback.22 rollback >>>>> function. It is supposed to rollback to the state at the previous commit. >>>>> So >>>>> you may want to turn off auto-commit on the index you are updating if you >>>>> want to control what that last commit level is. >>>>> >>>>> However, in your case if the index gets corrupted due to a disk full >>>>> situation, I don't know what rollback would do, if anything, to help. You >>>>> may need to play with the scenario to see what would happen. >>>>> >>>>> If you are using the DataImportHandler it may handle the rollback for >>>>> you...again, however, it may not deal with disk full situations gracefully >>>>> either. >>>>> -- >>>>> View this message in context: >>>>> http://lucene.472066.n3.nabble.com/index-format-error-because-disk-full-tp948249p948968.html >>>>> Sent from the Solr - User mailing list archive at Nabble.com. >>>>> >>>> >>>> >>>> >>>> -- >>>> Lance Norskog >>>> [email protected] >>>> >>> >> >> >> >> -- >> Lance Norskog >> [email protected] >> > > > > -- > Lance Norskog > [email protected] > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
