On Mon, Oct 26, 2009 at 3:00 PM, Michael McCandless < luc...@mikemccandless.com> wrote:
> On Mon, Oct 26, 2009 at 2:55 PM, Peter Keegan <peterlkee...@gmail.com> > wrote: > > On Mon, Oct 26, 2009 at 2:50 PM, Michael McCandless < > > luc...@mikemccandless.com> wrote: > > > >> On Mon, Oct 26, 2009 at 10:44 AM, Peter Keegan <peterlkee...@gmail.com> > >> wrote: > >> > Even running in console mode, the exception is difficult to interpret. > >> > Here's an exception that I think occurred during an add document, > commit > >> or > >> > close: > >> > doc counts differ for segment _g: field Reader shows 137 but > segmentInfo > >> > shows 5777 > >> > >> That's spooky. Do you have the full exception for this one? What IO > >> system are you running on? (Is it just a local drive on your windows > >> computer?) It's almost as if the IO system is not generating an > >> IOException to Java when disk fills up. > >> > > > > Index and code are all on a local drive. There is no other exception > coming > > back - just what I reported. > > But, you didn't report a traceback for this first one? > Yes, I need to add some more printStackTrace calls. > > >> > I ensured that the disk space was low before updating the index. > >> > >> You mean, to intentionally test the disk-full case? > >> > > > > Yes, that's right. > > OK. Can you turn on IndexWriter's infoStream, get this disk full / > corruption to happen again, and post back the resulting output? Make > sure your index first passes CheckIndex before starting (so we don't > begin the test w/ any pre-existing index corruption). > Good point about CheckIndex. I've already found 2 bad ones. I will build new indexes from scratch. This will take a while. > >> > On another occasion, the exception was: > >> > background merge hit exception: _0:C1080260 _1:C139 _2:C123 _3:C107 > >> _4:C126 > >> > _5:C121 _6:C126 _7:C711 _8:C116 into _9 [optimize] [mergeDocStores] > >> > >> In this case, the SegmentMerger was trying to open this segment, but > >> on attempting to read the first int from the fdx (fields index) file > >> for one of the segments, it hit EOF. > >> > >> This is also spooky -- this looks like index corruption, which should > >> never happen on hitting disk full. > >> > > > > That's what I thought, too. Could Lucene be catching the IOException and > > turning it into a different exception? > > I think that's unlikely, but I guess possible. We have "disk full" > tests in the unit tests, that throw an IOException at different times. > > What exact windows version are you using? The local drive is NTFS? > Windows Server 2003 Enterprise x64 SP2. Local drive is NTFS > > Mike > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > >
