Re: Lucene 2.3.1 IndexReader and IndexWriter Synchronization Issues

Sat, 29 Mar 2008 00:14:21 -0700
That use case of Lucene should be fine, ie no further synchronization should be necessary. 


Your debug output below is great, but it doesn't seem to cover the  
occurrence of that exception.  Can you post the full debug output?



Can you describe how you reindex in more detail?  Are you manually  
removing files from the index and then opening a new IndexWriter with  
create=true?


Mike

Jamie wrote:

HI There


I keep getting the following error when simultaneously reindexing  
my documents and searching through the index.


java.io.IOException: Cannot overwrite: C:\index9121\_2.cfs

   at org.apache.lucene.store.FSDirectory.createOutput 
(FSDirectory.java:433)
   at org.apache.lucene.index.CompoundFileWriter.close 
(CompoundFileWriter.java:150)
   at org.apache.lucene.index.DocumentsWriter.createCompoundFile 
(DocumentsWriter.java:569)
   at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java: 
2565)

   at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:2397)

   at org.apache.lucene.index.IndexWriter.closeInternal 
(IndexWriter.java:1204)

   at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1178)
   at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1153)


Consequently, often documents are missing from the index. Does one  
have to perform any synchronized across IndexWriter and  
IndexReader? I assumed that Lucene took care of this. I sincerely  
appreciate any guidance on offer!'


Many thanks
Jamie

PS: Here is the Lucene indexing debug information.


IFD [http-8090-2]: setInfoStream  
deletionPolicy=org.apache.lucene.index.KeepOnlyLastCommitDeletionPolic 
[EMAIL PROTECTED]
IW 27 [http-8090-2]: setInfoStream:  
[EMAIL PROTECTED]:\index9121  
autoCommit=false  
[EMAIL PROTECTED]  
[EMAIL PROTECTED] 
 ramBufferSizeMB=16.0 maxBuffereDocs=-1 maxBuffereDeleteTerms=-1  
maxFieldLength=50000 index=_3:c113 _4:c70 _5:c44 _6:c68

IW 27 [Timer-0]: now flush at close

IW 27 [Timer-0]:   flush: segment=_7 docStoreSegment=_7  
docStoreOffset=0 flushDocs=true flushDeletes=false  
flushDocStores=true numDocs=27 numBufDelTerms=0

IW 27 [Timer-0]:   index before flush _3:c113 _4:c70 _5:c44 _6:c68

flush postings as segment _7 numDocs=27

closeDocStore: 2 files to flush to segment _7 numDocs=27

 oldRAMSize=212992 newFlushedSize=32784 docs/MB=863.578 new/ 
old=15.392%
IFD [Timer-0]: now checkpoint "segments_5" [5 segments ; isCommit =  
false]

IFD [Timer-0]: delete pending file _9s.cfs
IFD [Timer-0]: delete "_9s.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9s.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9s.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9y.cfs
IFD [Timer-0]: delete "_9y.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9y.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9y.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9x.cfs
IFD [Timer-0]: delete "_9x.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9x.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9x.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9v.cfs
IFD [Timer-0]: delete "_9v.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9v.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9v.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9z.cfs
IFD [Timer-0]: delete "_9z.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9z.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9z.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9t.cfs
IFD [Timer-0]: delete "_9t.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9t.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9t.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9x.cfx
IFD [Timer-0]: delete "_9x.cfx"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9x.cfx":  
java.io.IOException: Cannot delete C:\index9121\_9x.cfx; Will re- 
try later.

IFD [Timer-0]: delete pending file _9u.cfs
IFD [Timer-0]: delete "_9u.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9u.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9u.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9w.cfs
IFD [Timer-0]: delete "_9w.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9w.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9w.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9t.cfx
IFD [Timer-0]: delete "_9t.cfx"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9t.cfx":  
java.io.IOException: Cannot delete C:\index9121\_9t.cfx; Will re- 
try later.
IFD [Timer-0]: now checkpoint "segments_5" [5 segments ; isCommit =  
false]

IFD [Timer-0]: delete pending file _9s.cfs
IFD [Timer-0]: delete "_9s.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9s.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9s.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9y.cfs
IFD [Timer-0]: delete "_9y.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9y.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9y.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9x.cfs
IFD [Timer-0]: delete "_9x.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9x.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9x.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9v.cfs
IFD [Timer-0]: delete "_9v.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9v.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9v.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9z.cfs
IFD [Timer-0]: delete "_9z.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9z.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9z.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9t.cfs
IFD [Timer-0]: delete "_9t.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9t.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9t.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9x.cfx
IFD [Timer-0]: delete "_9x.cfx"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9x.cfx":  
java.io.IOException: Cannot delete C:\index9121\_9x.cfx; Will re- 
try later.

IFD [Timer-0]: delete pending file _9u.cfs
IFD [Timer-0]: delete "_9u.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9u.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9u.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9w.cfs
IFD [Timer-0]: delete "_9w.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9w.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9w.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9t.cfx
IFD [Timer-0]: delete "_9t.cfx"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9t.cfx":  
java.io.IOException: Cannot delete C:\index9121\_9t.cfx; Will re- 
try later.

IFD [Timer-0]: delete "_7.fnm"
IFD [Timer-0]: delete "_7.frq"
IFD [Timer-0]: delete "_7.prx"
IFD [Timer-0]: delete "_7.tis"
IFD [Timer-0]: delete "_7.tii"
IFD [Timer-0]: delete "_7.nrm"
IFD [Timer-0]: delete "_7.fdx"
IFD [Timer-0]: delete "_7.fdt"
IW 27 [Timer-0]: LMP: findMerges: 5 segments
IW 27 [Timer-0]: LMP:   level -1.0 to 5.250432: 5 segments
IW 27 [Timer-0]: CMS: now merge
IW 27 [Timer-0]: CMS:   index: _3:c113 _4:c70 _5:c44 _6:c68 _7:c27
IW 27 [Timer-0]: CMS:   no more merges pending; now return
IW 27 [Timer-0]: close: wrote segments file "segments_6"

IFD [Timer-0]: now checkpoint "segments_6" [5 segments ; isCommit =  
true]

IFD [Timer-0]: delete pending file _9s.cfs
IFD [Timer-0]: delete "_9s.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9s.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9s.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9y.cfs
IFD [Timer-0]: delete "_9y.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9y.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9y.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9x.cfs
IFD [Timer-0]: delete "_9x.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9x.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9x.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9v.cfs
IFD [Timer-0]: delete "_9v.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9v.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9v.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9z.cfs
IFD [Timer-0]: delete "_9z.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9z.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9z.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9t.cfs
IFD [Timer-0]: delete "_9t.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9t.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9t.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9x.cfx
IFD [Timer-0]: delete "_9x.cfx"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9x.cfx":  
java.io.IOException: Cannot delete C:\index9121\_9x.cfx; Will re- 
try later.

IFD [Timer-0]: delete pending file _9u.cfs
IFD [Timer-0]: delete "_9u.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9u.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9u.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9w.cfs
IFD [Timer-0]: delete "_9w.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9w.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9w.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9t.cfx
IFD [Timer-0]: delete "_9t.cfx"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9t.cfx":  
java.io.IOException: Cannot delete C:\index9121\_9t.cfx; Will re- 
try later.

IFD [Timer-0]: deleteCommits: now remove commit "segments_5"
IFD [Timer-0]: delete "segments_5"
IW 27 [Timer-0]: at close: _3:c113 _4:c70 _5:c44 _6:c68 _7:c27
IFD [Timer-0]: delete pending file _9s.cfs
IFD [Timer-0]: delete "_9s.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9s.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9s.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9y.cfs
IFD [Timer-0]: delete "_9y.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9y.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9y.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9x.cfs
IFD [Timer-0]: delete "_9x.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9x.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9x.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9v.cfs
IFD [Timer-0]: delete "_9v.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9v.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9v.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9z.cfs
IFD [Timer-0]: delete "_9z.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9z.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9z.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9t.cfs
IFD [Timer-0]: delete "_9t.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9t.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9t.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9x.cfx
IFD [Timer-0]: delete "_9x.cfx"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9x.cfx":  
java.io.IOException: Cannot delete C:\index9121\_9x.cfx; Will re- 
try later.

IFD [Timer-0]: delete pending file _9u.cfs
IFD [Timer-0]: delete "_9u.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9u.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9u.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9w.cfs
IFD [Timer-0]: delete "_9w.cfs"

IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9w.cfs":  
java.io.IOException: Cannot delete C:\index9121\_9w.cfs; Will re- 
try later.

IFD [Timer-0]: delete pending file _9t.cfx
IFD [Timer-0]: delete "_9t.cfx"
IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9t.cfx": jav



---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]























					Reply via email to