Is there any chance that two writers are open on this directory? Or, something external to Lucene is removing files from the directory.
It looks like there were at least two missing files (_37 On Thu, Aug 13, 2009 at 5:19 AM, rishisinghal<singhal.ri...@gmail.com> wrote: > > Hi, > > I am trying to index documents and when all is complete and optimize is > called I get > > IFD [main]: setInfoStream > deletionpolicy=org.apache.lucene.index.keeponlylastcommitdeletionpol...@4fced0 > IW 0 [main]: setInfoStream: > dir=org.apache.lucene.store.FSDirectory@/SYS$SYSDEVICE/RISHI/melon_1600 > autoCommit=false mergePolicy=org > .apache.lucene.index.logbytesizemergepol...@92cb49 > mergescheduler=org.apache.lucene.index.concurrentmergeschedu...@6038e4 > ramBufferS > izeMB=16.0 maxBufferedDocs=5 maxBuffereDeleteTerms=-1 maxFieldLength=10000 > index=_37:c465->_0 _4c:c165->_0 _61:c245->_0 _9e:c485->_0 > _ay:c225->_0 _az:c5->_0 _b0:c5->_0 > IW 0 [main]: optimize: index now _37:c465->_0 _4c:c165->_0 _61:c245->_0 > _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 > IW 0 [main]: flush: segment=null docStoreSegment=null docStoreOffset=0 > flushDocs=false flushDeletes=true flushDocStores=false numD > ocs=0 numBufDelTerms=0 > IW 0 [main]: index before flush _37:c465->_0 _4c:c165->_0 _61:c245->_0 > _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 > IW 0 [main]: add merge to pendingMerges: _61:c245->_0 _9e:c485->_0 > _ay:c225->_0 _az:c5->_0 _b0:c5->_0 [optimize] [total 1 pending] > IW 0 [main]: CMS: now merge > IW 0 [main]: CMS: index: _37:c465->_0 _4c:c165->_0 _61:c245->_0 > _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 > IW 0 [main]: CMS: consider merge _61:c245->_0 _9e:c485->_0 _ay:c225->_0 > _az:c5->_0 _b0:c5->_0 into _b5 [optimize] > IW 0 [main]: CMS: launch new thread [Lucene Merge Thread #73] > IW 0 [main]: CMS: no more merges pending; now return > IW 0 [Lucene Merge Thread #73]: CMS: merge thread: start > IW 0 [Lucene Merge Thread #73]: now merge > merge=_61:c245->_0 _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 into > _b5 [optimize] > merge=org.apache.lucene.index.mergepolicy$oneme...@3725db > index=_37:c465->_0 _4c:c165->_0 _61:c245->_0 _9e:c485->_0 _ay:c225->_0 > _az:c5->_0 _b0:c5->_0 > IW 0 [Lucene Merge Thread #73]: merging _61:c245->_0 _9e:c485->_0 > _ay:c225->_0 _az:c5->_0 _b0:c5->_0 into _b5 [optimize] > IW 0 [Lucene Merge Thread #73]: hit exception during merge > Exception in thread "Lucene Merge Thread #73" > org.apache.lucene.index.MergePolicy$MergeException: > java.io.FileNotFoundException: /SY > S$SYSDEVICE/RISHI/melon_1600/_61.cfs (i/o error (errno:5)) > at > org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:309) > at > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:286) > Caused by: java.io.FileNotFoundException: > /SYS$SYSDEVICE/RISHI/melon_1600/_61.cfs (i/o error (errno:5)) > at java.io.RandomAccessFile.open(Native Method) > at java.io.RandomAccessFile.<init>(RandomAccessFile.java:231) > at > org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.<init>(FSDirectory.java:552) > at > org.apache.lucene.store.FSDirectory$FSIndexInput.<init>(FSDirectory.java:582) > at > org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:488) > at > org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:70) > at > org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:321) > at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:306) > at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:260) > at > org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4220) > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3884) > at > org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:205) > at > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:260) > Aug 13, 2009 2:40:56 PM org.hp.openvms.lucene.index.FileIndexerJSP jspMain > SEVERE: Indexing not completebackground merge hit exception: _61:c245->_0 > _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 into _b5 [ > optimize] > java.io.IOException: background merge hit exception: _61:c245->_0 > _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 into _b5 [optimize > ] > at > org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2273) > at > org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2218) > at > org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2198) > at org.hp.openvms.lucene.index.FileIndexerJSP.jspMain(Unknown > Source) > at org.hp.openvms.lucene.index.FileIndexer.main(Unknown Source) > Caused by: java.io.FileNotFoundException: > /SYS$SYSDEVICE/RISHI/melon_1600/_61.cfs (i/o error (errno:5)) > at java.io.RandomAccessFile.open(Native Method) > at java.io.RandomAccessFile.<init>(RandomAccessFile.java:231) > at > org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.<init>(FSDirectory.java:552) > at > org.apache.lucene.store.FSDirectory$FSIndexInput.<init>(FSDirectory.java:582) > at > org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:488) > at > org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:70) > at > org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:321) > at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:306) > at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:260) > at > org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4220) > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3884) > at > org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:205) > at > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:260) > IW 0 [main]: now flush at close > IW 0 [main]: flush: segment=null docStoreSegment=null docStoreOffset=0 > flushDocs=false flushDeletes=true flushDocStores=false numD > ocs=0 numBufDelTerms=0 > IW 0 [main]: index before flush _37:c465->_0 _4c:c165->_0 _61:c245->_0 > _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 > IW 0 [main]: CMS: now merge > IW 0 [main]: CMS: index: _37:c465->_0 _4c:c165->_0 _61:c245->_0 > _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 > IW 0 [main]: CMS: no more merges pending; now return > IW 0 [main]: now call final commit() > IW 0 [main]: startCommit(): start sizeInBytes=0 > IW 0 [main]: startCommit index=_37:c465->_0 _4c:c165->_0 _61:c245->_0 > _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 changeCount=11 > 95 > IW 0 [main]: now sync _37.cfs > IW 0 [main]: hit exception while closing > Exception in thread "main" java.io.FileNotFoundException: > /SYS$SYSDEVICE/RISHI/melon_1600/_37.cfs (i/o error (errno:5)) > at java.io.RandomAccessFile.open(Native Method) > at java.io.RandomAccessFile.<init>(RandomAccessFile.java:231) > at org.apache.lucene.store.FSDirectory.sync(FSDirectory.java:456) > at > org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4590) > at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3369) > at > org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:1654) > at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1602) > at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1578) > at org.hp.openvms.lucene.index.FileIndexerJSP.jspMain(Unknown > Source) > at org.hp.openvms.lucene.index.FileIndexer.main(Unknown Source) > > > > I have setMaxBufferedDocs = 5 and setMergeFactor=5. I was trying different > combinations of these to see if that helps but was unsuccessful. > > > Regards, > Rishi > -- > View this message in context: > http://www.nabble.com/Indexer-crashes-with-%22hit-exception-during-merge%22-tp24951667p24951667.html > Sent from the Lucene - Java Users mailing list archive at Nabble.com. > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org