I noticed the exception is "Caused by: java.io.FileNotFoundException:
/SYS$SYSDEVICE/RISHI/melon_1600/_61.cfs (i/o error (errno:5))"

I searched for i/o error (errno:5) and found some information which
associates it w/ a more native IO problem, like corrupt file due to system
crash etc.

Did you experience that? Can you try to create the index somewhere else, or
on another drive?

Shai

On Thu, Aug 13, 2009 at 3:00 PM, rishisinghal <singhal.ri...@gmail.com>wrote:

>
> It is a local file system.
>
> We are using lucene 2.4 and java 1.5
>
> Regards,
> Rishi
>
>
> Shai Erera wrote:
> >
> > Is that a local file system, or a network share?
> >
> > On Thu, Aug 13, 2009 at 1:07 PM, rishisinghal
> > <singhal.ri...@gmail.com>wrote:
> >
> >>
> >> >>Is there any chance that two writers are open on this directory?
> >> No, thats not true.
> >>
> >> >>something external to Lucene is removing files from the directory.
> >> No this also has rare chances as I am the owner of these files and other
> >> then me no one can delete the, :-)
> >>
> >> Here are all the files in the index directory:
> >> segments.gen;1      segments_1.;1       segments_2.;1       _0.fdt;1
> >> _0.fdx;1            _37.cfs;1           _4c.cfs;1           _61.cfs;1
> >> _9e.cfs;1           _ay.cfs;1           _az.cfs;1           _b0.cfs;1
> >> _b2.tis;1           _b3.fdt;1           _b4.fdt;1
> >>
> >> I see _37 present.
> >>
> >> Regards,
> >> Rishi
> >>
> >>
> >> Michael McCandless-2 wrote:
> >> >
> >> > Is there any chance that two writers are open on this directory?  Or,
> >> > something external to Lucene is removing files from the directory.
> >> >
> >> > It looks like there were at least two missing files (_37
> >> >
> >> > On Thu, Aug 13, 2009 at 5:19 AM, rishisinghal<singhal.ri...@gmail.com
> >
> >> > wrote:
> >> >>
> >> >> Hi,
> >> >>
> >> >> I am trying to index documents and when all is complete and optimize
> >> is
> >> >> called I get
> >> >>
> >> >> IFD [main]: setInfoStream
> >> >>
> >>
> deletionpolicy=org.apache.lucene.index.keeponlylastcommitdeletionpol...@4fced0
> >> >> IW 0 [main]: setInfoStream:
> >> >>
> >> dir=org.apache.lucene.store.FSDirectory@/SYS$SYSDEVICE/RISHI/melon_1600
> >> >> autoCommit=false mergePolicy=org
> >> >> .apache.lucene.index.logbytesizemergepol...@92cb49
> >> >>
> mergescheduler=org.apache.lucene.index.concurrentmergeschedu...@6038e4
> >> >> ramBufferS
> >> >> izeMB=16.0 maxBufferedDocs=5 maxBuffereDeleteTerms=-1
> >> >> maxFieldLength=10000
> >> >> index=_37:c465->_0 _4c:c165->_0 _61:c245->_0 _9e:c485->_0
> >> >>  _ay:c225->_0 _az:c5->_0 _b0:c5->_0
> >> >> IW 0 [main]: optimize: index now _37:c465->_0 _4c:c165->_0
> >> _61:c245->_0
> >> >> _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0
> >> >> IW 0 [main]:   flush: segment=null docStoreSegment=null
> >> docStoreOffset=0
> >> >> flushDocs=false flushDeletes=true flushDocStores=false numD
> >> >> ocs=0 numBufDelTerms=0
> >> >> IW 0 [main]:   index before flush _37:c465->_0 _4c:c165->_0
> >> _61:c245->_0
> >> >> _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0
> >> >> IW 0 [main]: add merge to pendingMerges: _61:c245->_0 _9e:c485->_0
> >> >> _ay:c225->_0 _az:c5->_0 _b0:c5->_0 [optimize] [total 1 pending]
> >> >> IW 0 [main]: CMS: now merge
> >> >> IW 0 [main]: CMS:   index: _37:c465->_0 _4c:c165->_0 _61:c245->_0
> >> >> _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0
> >> >> IW 0 [main]: CMS:   consider merge _61:c245->_0 _9e:c485->_0
> >> _ay:c225->_0
> >> >> _az:c5->_0 _b0:c5->_0 into _b5 [optimize]
> >> >> IW 0 [main]: CMS:     launch new thread [Lucene Merge Thread #73]
> >> >> IW 0 [main]: CMS:   no more merges pending; now return
> >> >> IW 0 [Lucene Merge Thread #73]: CMS:   merge thread: start
> >> >> IW 0 [Lucene Merge Thread #73]: now merge
> >> >>  merge=_61:c245->_0 _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0
> >> into
> >> >> _b5 [optimize]
> >> >>  merge=org.apache.lucene.index.mergepolicy$oneme...@3725db
> >> >>  index=_37:c465->_0 _4c:c165->_0 _61:c245->_0 _9e:c485->_0
> >> _ay:c225->_0
> >> >> _az:c5->_0 _b0:c5->_0
> >> >> IW 0 [Lucene Merge Thread #73]: merging _61:c245->_0 _9e:c485->_0
> >> >> _ay:c225->_0 _az:c5->_0 _b0:c5->_0 into _b5 [optimize]
> >> >> IW 0 [Lucene Merge Thread #73]: hit exception during merge
> >> >> Exception in thread "Lucene Merge Thread #73"
> >> >> org.apache.lucene.index.MergePolicy$MergeException:
> >> >> java.io.FileNotFoundException: /SY
> >> >> S$SYSDEVICE/RISHI/melon_1600/_61.cfs (i/o error (errno:5))
> >> >>        at
> >> >>
> >>
> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:309)
> >> >>        at
> >> >>
> >>
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:286)
> >> >> Caused by: java.io.FileNotFoundException:
> >> >> /SYS$SYSDEVICE/RISHI/melon_1600/_61.cfs (i/o error (errno:5))
> >> >>        at java.io.RandomAccessFile.open(Native Method)
> >> >>        at java.io.RandomAccessFile.<init>(RandomAccessFile.java:231)
> >> >>        at
> >> >>
> >>
> org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.<init>(FSDirectory.java:552)
> >> >>        at
> >> >>
> >>
> org.apache.lucene.store.FSDirectory$FSIndexInput.<init>(FSDirectory.java:582)
> >> >>        at
> >> >> org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:488)
> >> >>        at
> >> >>
> >>
> org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:70)
> >> >>        at
> >> >>
> >> org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:321)
> >> >>        at
> >> >> org.apache.lucene.index.SegmentReader.get(SegmentReader.java:306)
> >> >>        at
> >> >> org.apache.lucene.index.SegmentReader.get(SegmentReader.java:260)
> >> >>        at
> >> >>
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4220)
> >> >>        at
> >> >> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3884)
> >> >>        at
> >> >>
> >>
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:205)
> >> >>        at
> >> >>
> >>
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:260)
> >> >> Aug 13, 2009 2:40:56 PM org.hp.openvms.lucene.index.FileIndexerJSP
> >> >> jspMain
> >> >> SEVERE: Indexing not completebackground merge hit exception:
> >> _61:c245->_0
> >> >> _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 into _b5 [
> >> >> optimize]
> >> >> java.io.IOException: background merge hit exception: _61:c245->_0
> >> >> _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 into _b5 [optimize
> >> >> ]
> >> >>        at
> >> >> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2273)
> >> >>        at
> >> >> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2218)
> >> >>        at
> >> >> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2198)
> >> >>        at org.hp.openvms.lucene.index.FileIndexerJSP.jspMain(Unknown
> >> >> Source)
> >> >>        at org.hp.openvms.lucene.index.FileIndexer.main(Unknown
> Source)
> >> >> Caused by: java.io.FileNotFoundException:
> >> >> /SYS$SYSDEVICE/RISHI/melon_1600/_61.cfs (i/o error (errno:5))
> >> >>        at java.io.RandomAccessFile.open(Native Method)
> >> >>        at java.io.RandomAccessFile.<init>(RandomAccessFile.java:231)
> >> >>        at
> >> >>
> >>
> org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.<init>(FSDirectory.java:552)
> >> >>        at
> >> >>
> >>
> org.apache.lucene.store.FSDirectory$FSIndexInput.<init>(FSDirectory.java:582)
> >> >>        at
> >> >> org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:488)
> >> >>        at
> >> >>
> >>
> org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:70)
> >> >>        at
> >> >>
> >> org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:321)
> >> >>        at
> >> >> org.apache.lucene.index.SegmentReader.get(SegmentReader.java:306)
> >> >>        at
> >> >> org.apache.lucene.index.SegmentReader.get(SegmentReader.java:260)
> >> >>        at
> >> >>
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4220)
> >> >>        at
> >> >> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3884)
> >> >>        at
> >> >>
> >>
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:205)
> >> >>        at
> >> >>
> >>
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:260)
> >> >> IW 0 [main]: now flush at close
> >> >> IW 0 [main]:   flush: segment=null docStoreSegment=null
> >> docStoreOffset=0
> >> >> flushDocs=false flushDeletes=true flushDocStores=false numD
> >> >> ocs=0 numBufDelTerms=0
> >> >> IW 0 [main]:   index before flush _37:c465->_0 _4c:c165->_0
> >> _61:c245->_0
> >> >> _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0
> >> >> IW 0 [main]: CMS: now merge
> >> >> IW 0 [main]: CMS:   index: _37:c465->_0 _4c:c165->_0 _61:c245->_0
> >> >> _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0
> >> >> IW 0 [main]: CMS:   no more merges pending; now return
> >> >> IW 0 [main]: now call final commit()
> >> >> IW 0 [main]: startCommit(): start sizeInBytes=0
> >> >> IW 0 [main]: startCommit index=_37:c465->_0 _4c:c165->_0 _61:c245->_0
> >> >> _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 changeCount=11
> >> >> 95
> >> >> IW 0 [main]: now sync _37.cfs
> >> >> IW 0 [main]: hit exception while closing
> >> >> Exception in thread "main" java.io.FileNotFoundException:
> >> >> /SYS$SYSDEVICE/RISHI/melon_1600/_37.cfs (i/o error (errno:5))
> >> >>        at java.io.RandomAccessFile.open(Native Method)
> >> >>        at java.io.RandomAccessFile.<init>(RandomAccessFile.java:231)
> >> >>        at
> >> org.apache.lucene.store.FSDirectory.sync(FSDirectory.java:456)
> >> >>        at
> >> >>
> org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4590)
> >> >>        at
> >> >> org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3369)
> >> >>        at
> >> >>
> >> org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:1654)
> >> >>        at
> >> >> org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1602)
> >> >>        at
> >> >> org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1578)
> >> >>        at org.hp.openvms.lucene.index.FileIndexerJSP.jspMain(Unknown
> >> >> Source)
> >> >>        at org.hp.openvms.lucene.index.FileIndexer.main(Unknown
> Source)
> >> >>
> >> >>
> >> >>
> >> >> I have setMaxBufferedDocs = 5 and setMergeFactor=5. I was trying
> >> >> different
> >> >> combinations of these to see if that helps but was unsuccessful.
> >> >>
> >> >>
> >> >> Regards,
> >> >> Rishi
> >> >> --
> >> >> View this message in context:
> >> >>
> >>
> http://www.nabble.com/Indexer-crashes-with-%22hit-exception-during-merge%22-tp24951667p24951667.html
> >> >> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
> >> >>
> >> >>
> >> >> ---------------------------------------------------------------------
> >> >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> >> >> For additional commands, e-mail: java-user-h...@lucene.apache.org
> >> >>
> >> >>
> >> >
> >> > ---------------------------------------------------------------------
> >> > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> >> > For additional commands, e-mail: java-user-h...@lucene.apache.org
> >> >
> >> >
> >> >
> >>
> >> --
> >> View this message in context:
> >>
> http://www.nabble.com/Indexer-crashes-with-%22hit-exception-during-merge%22-tp24951667p24952295.html
> >> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> >> For additional commands, e-mail: java-user-h...@lucene.apache.org
> >>
> >>
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/Indexer-crashes-with-%22hit-exception-during-merge%22-tp24951667p24953575.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> For additional commands, e-mail: java-user-h...@lucene.apache.org
>
>

Reply via email to