I tried creating the index in different disks but still i see the issue :-(
I tried to index documents in other disks also and got the same exception. I also tried $ java org.apache.lucene.index.CheckIndex /SYS$SYSDEVICE/RISHI/melon_1600/ -segment _61 NOTE: testing will be more thorough if you run java with '-ea:org.apache.lucene...', so assertions are enabled Opening index @ /SYS$SYSDEVICE/RISHI/melon_1600/ Segments file=segments_1 numSegments=0 version=FORMAT_HAS_PROX [Lucene 2.4] Checking only these segments: _61: No problems were detected with this index. Regards, Rishi Shai Erera wrote: > > I noticed the exception is "Caused by: java.io.FileNotFoundException: > /SYS$SYSDEVICE/RISHI/melon_1600/_61.cfs (i/o error (errno:5))" > > I searched for i/o error (errno:5) and found some information which > associates it w/ a more native IO problem, like corrupt file due to system > crash etc. > > Did you experience that? Can you try to create the index somewhere else, > or > on another drive? > > Shai > > On Thu, Aug 13, 2009 at 3:00 PM, rishisinghal > <singhal.ri...@gmail.com>wrote: > >> >> It is a local file system. >> >> We are using lucene 2.4 and java 1.5 >> >> Regards, >> Rishi >> >> >> Shai Erera wrote: >> > >> > Is that a local file system, or a network share? >> > >> > On Thu, Aug 13, 2009 at 1:07 PM, rishisinghal >> > <singhal.ri...@gmail.com>wrote: >> > >> >> >> >> >>Is there any chance that two writers are open on this directory? >> >> No, thats not true. >> >> >> >> >>something external to Lucene is removing files from the directory. >> >> No this also has rare chances as I am the owner of these files and >> other >> >> then me no one can delete the, :-) >> >> >> >> Here are all the files in the index directory: >> >> segments.gen;1 segments_1.;1 segments_2.;1 _0.fdt;1 >> >> _0.fdx;1 _37.cfs;1 _4c.cfs;1 _61.cfs;1 >> >> _9e.cfs;1 _ay.cfs;1 _az.cfs;1 _b0.cfs;1 >> >> _b2.tis;1 _b3.fdt;1 _b4.fdt;1 >> >> >> >> I see _37 present. >> >> >> >> Regards, >> >> Rishi >> >> >> >> >> >> Michael McCandless-2 wrote: >> >> > >> >> > Is there any chance that two writers are open on this directory? >> Or, >> >> > something external to Lucene is removing files from the directory. >> >> > >> >> > It looks like there were at least two missing files (_37 >> >> > >> >> > On Thu, Aug 13, 2009 at 5:19 AM, >> rishisinghal<singhal.ri...@gmail.com >> > >> >> > wrote: >> >> >> >> >> >> Hi, >> >> >> >> >> >> I am trying to index documents and when all is complete and >> optimize >> >> is >> >> >> called I get >> >> >> >> >> >> IFD [main]: setInfoStream >> >> >> >> >> >> deletionpolicy=org.apache.lucene.index.keeponlylastcommitdeletionpol...@4fced0 >> >> >> IW 0 [main]: setInfoStream: >> >> >> >> >> >> dir=org.apache.lucene.store.FSDirectory@/SYS$SYSDEVICE/RISHI/melon_1600 >> >> >> autoCommit=false mergePolicy=org >> >> >> .apache.lucene.index.logbytesizemergepol...@92cb49 >> >> >> >> mergescheduler=org.apache.lucene.index.concurrentmergeschedu...@6038e4 >> >> >> ramBufferS >> >> >> izeMB=16.0 maxBufferedDocs=5 maxBuffereDeleteTerms=-1 >> >> >> maxFieldLength=10000 >> >> >> index=_37:c465->_0 _4c:c165->_0 _61:c245->_0 _9e:c485->_0 >> >> >> _ay:c225->_0 _az:c5->_0 _b0:c5->_0 >> >> >> IW 0 [main]: optimize: index now _37:c465->_0 _4c:c165->_0 >> >> _61:c245->_0 >> >> >> _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 >> >> >> IW 0 [main]: flush: segment=null docStoreSegment=null >> >> docStoreOffset=0 >> >> >> flushDocs=false flushDeletes=true flushDocStores=false numD >> >> >> ocs=0 numBufDelTerms=0 >> >> >> IW 0 [main]: index before flush _37:c465->_0 _4c:c165->_0 >> >> _61:c245->_0 >> >> >> _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 >> >> >> IW 0 [main]: add merge to pendingMerges: _61:c245->_0 _9e:c485->_0 >> >> >> _ay:c225->_0 _az:c5->_0 _b0:c5->_0 [optimize] [total 1 pending] >> >> >> IW 0 [main]: CMS: now merge >> >> >> IW 0 [main]: CMS: index: _37:c465->_0 _4c:c165->_0 _61:c245->_0 >> >> >> _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 >> >> >> IW 0 [main]: CMS: consider merge _61:c245->_0 _9e:c485->_0 >> >> _ay:c225->_0 >> >> >> _az:c5->_0 _b0:c5->_0 into _b5 [optimize] >> >> >> IW 0 [main]: CMS: launch new thread [Lucene Merge Thread #73] >> >> >> IW 0 [main]: CMS: no more merges pending; now return >> >> >> IW 0 [Lucene Merge Thread #73]: CMS: merge thread: start >> >> >> IW 0 [Lucene Merge Thread #73]: now merge >> >> >> merge=_61:c245->_0 _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 >> >> into >> >> >> _b5 [optimize] >> >> >> merge=org.apache.lucene.index.mergepolicy$oneme...@3725db >> >> >> index=_37:c465->_0 _4c:c165->_0 _61:c245->_0 _9e:c485->_0 >> >> _ay:c225->_0 >> >> >> _az:c5->_0 _b0:c5->_0 >> >> >> IW 0 [Lucene Merge Thread #73]: merging _61:c245->_0 _9e:c485->_0 >> >> >> _ay:c225->_0 _az:c5->_0 _b0:c5->_0 into _b5 [optimize] >> >> >> IW 0 [Lucene Merge Thread #73]: hit exception during merge >> >> >> Exception in thread "Lucene Merge Thread #73" >> >> >> org.apache.lucene.index.MergePolicy$MergeException: >> >> >> java.io.FileNotFoundException: /SY >> >> >> S$SYSDEVICE/RISHI/melon_1600/_61.cfs (i/o error (errno:5)) >> >> >> at >> >> >> >> >> >> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:309) >> >> >> at >> >> >> >> >> >> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:286) >> >> >> Caused by: java.io.FileNotFoundException: >> >> >> /SYS$SYSDEVICE/RISHI/melon_1600/_61.cfs (i/o error (errno:5)) >> >> >> at java.io.RandomAccessFile.open(Native Method) >> >> >> at >> java.io.RandomAccessFile.<init>(RandomAccessFile.java:231) >> >> >> at >> >> >> >> >> >> org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.<init>(FSDirectory.java:552) >> >> >> at >> >> >> >> >> >> org.apache.lucene.store.FSDirectory$FSIndexInput.<init>(FSDirectory.java:582) >> >> >> at >> >> >> org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:488) >> >> >> at >> >> >> >> >> >> org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:70) >> >> >> at >> >> >> >> >> >> org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:321) >> >> >> at >> >> >> org.apache.lucene.index.SegmentReader.get(SegmentReader.java:306) >> >> >> at >> >> >> org.apache.lucene.index.SegmentReader.get(SegmentReader.java:260) >> >> >> at >> >> >> >> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4220) >> >> >> at >> >> >> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3884) >> >> >> at >> >> >> >> >> >> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:205) >> >> >> at >> >> >> >> >> >> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:260) >> >> >> Aug 13, 2009 2:40:56 PM org.hp.openvms.lucene.index.FileIndexerJSP >> >> >> jspMain >> >> >> SEVERE: Indexing not completebackground merge hit exception: >> >> _61:c245->_0 >> >> >> _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 into _b5 [ >> >> >> optimize] >> >> >> java.io.IOException: background merge hit exception: _61:c245->_0 >> >> >> _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 into _b5 [optimize >> >> >> ] >> >> >> at >> >> >> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2273) >> >> >> at >> >> >> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2218) >> >> >> at >> >> >> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2198) >> >> >> at >> org.hp.openvms.lucene.index.FileIndexerJSP.jspMain(Unknown >> >> >> Source) >> >> >> at org.hp.openvms.lucene.index.FileIndexer.main(Unknown >> Source) >> >> >> Caused by: java.io.FileNotFoundException: >> >> >> /SYS$SYSDEVICE/RISHI/melon_1600/_61.cfs (i/o error (errno:5)) >> >> >> at java.io.RandomAccessFile.open(Native Method) >> >> >> at >> java.io.RandomAccessFile.<init>(RandomAccessFile.java:231) >> >> >> at >> >> >> >> >> >> org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.<init>(FSDirectory.java:552) >> >> >> at >> >> >> >> >> >> org.apache.lucene.store.FSDirectory$FSIndexInput.<init>(FSDirectory.java:582) >> >> >> at >> >> >> org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:488) >> >> >> at >> >> >> >> >> >> org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:70) >> >> >> at >> >> >> >> >> >> org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:321) >> >> >> at >> >> >> org.apache.lucene.index.SegmentReader.get(SegmentReader.java:306) >> >> >> at >> >> >> org.apache.lucene.index.SegmentReader.get(SegmentReader.java:260) >> >> >> at >> >> >> >> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4220) >> >> >> at >> >> >> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3884) >> >> >> at >> >> >> >> >> >> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:205) >> >> >> at >> >> >> >> >> >> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:260) >> >> >> IW 0 [main]: now flush at close >> >> >> IW 0 [main]: flush: segment=null docStoreSegment=null >> >> docStoreOffset=0 >> >> >> flushDocs=false flushDeletes=true flushDocStores=false numD >> >> >> ocs=0 numBufDelTerms=0 >> >> >> IW 0 [main]: index before flush _37:c465->_0 _4c:c165->_0 >> >> _61:c245->_0 >> >> >> _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 >> >> >> IW 0 [main]: CMS: now merge >> >> >> IW 0 [main]: CMS: index: _37:c465->_0 _4c:c165->_0 _61:c245->_0 >> >> >> _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 >> >> >> IW 0 [main]: CMS: no more merges pending; now return >> >> >> IW 0 [main]: now call final commit() >> >> >> IW 0 [main]: startCommit(): start sizeInBytes=0 >> >> >> IW 0 [main]: startCommit index=_37:c465->_0 _4c:c165->_0 >> _61:c245->_0 >> >> >> _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 changeCount=11 >> >> >> 95 >> >> >> IW 0 [main]: now sync _37.cfs >> >> >> IW 0 [main]: hit exception while closing >> >> >> Exception in thread "main" java.io.FileNotFoundException: >> >> >> /SYS$SYSDEVICE/RISHI/melon_1600/_37.cfs (i/o error (errno:5)) >> >> >> at java.io.RandomAccessFile.open(Native Method) >> >> >> at >> java.io.RandomAccessFile.<init>(RandomAccessFile.java:231) >> >> >> at >> >> org.apache.lucene.store.FSDirectory.sync(FSDirectory.java:456) >> >> >> at >> >> >> >> org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4590) >> >> >> at >> >> >> org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3369) >> >> >> at >> >> >> >> >> >> org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:1654) >> >> >> at >> >> >> org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1602) >> >> >> at >> >> >> org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1578) >> >> >> at >> org.hp.openvms.lucene.index.FileIndexerJSP.jspMain(Unknown >> >> >> Source) >> >> >> at org.hp.openvms.lucene.index.FileIndexer.main(Unknown >> Source) >> >> >> >> >> >> >> >> >> >> >> >> I have setMaxBufferedDocs = 5 and setMergeFactor=5. I was trying >> >> >> different >> >> >> combinations of these to see if that helps but was unsuccessful. >> >> >> >> >> >> >> >> >> Regards, >> >> >> Rishi >> >> >> -- >> >> >> View this message in context: >> >> >> >> >> >> http://www.nabble.com/Indexer-crashes-with-%22hit-exception-during-merge%22-tp24951667p24951667.html >> >> >> Sent from the Lucene - Java Users mailing list archive at >> Nabble.com. >> >> >> >> >> >> >> >> >> >> --------------------------------------------------------------------- >> >> >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> >> >> For additional commands, e-mail: java-user-h...@lucene.apache.org >> >> >> >> >> >> >> >> > >> >> > >> --------------------------------------------------------------------- >> >> > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> >> > For additional commands, e-mail: java-user-h...@lucene.apache.org >> >> > >> >> > >> >> > >> >> >> >> -- >> >> View this message in context: >> >> >> http://www.nabble.com/Indexer-crashes-with-%22hit-exception-during-merge%22-tp24951667p24952295.html >> >> Sent from the Lucene - Java Users mailing list archive at Nabble.com. >> >> >> >> >> >> --------------------------------------------------------------------- >> >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> >> For additional commands, e-mail: java-user-h...@lucene.apache.org >> >> >> >> >> > >> > >> >> -- >> View this message in context: >> http://www.nabble.com/Indexer-crashes-with-%22hit-exception-during-merge%22-tp24951667p24953575.html >> Sent from the Lucene - Java Users mailing list archive at Nabble.com. >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-user-h...@lucene.apache.org >> >> > > -- View this message in context: http://www.nabble.com/Indexer-crashes-with-%22hit-exception-during-merge%22-tp24951667p24953833.html Sent from the Lucene - Java Users mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
