I've seen a couple of other people reporting similar issues when switching to the latest 1.3 tree. However, usually the problems come from lack of synchronization of multiple threads doing reads and writes against the index. But it's definetely possible that the compound index format makes this issue more visible or that it has its own bugs. In my testing and in the initial testing of the Lucene dev team when we were working on the compound index format we didn't see any such issues, but of course that could just mean that we didn't look in the right place.
Anyway, it would be extremely helpful if you could produce a self-contained program that demonstrates this behavior. I know that this can be hard to impossible with concurrency-related issues, but still.
Also, what OS is this running under? This issue is related to filesystem locking behavior which is different between Unix and Windows.
Does this happen every time or does it happen only under heavy load, etc.?
What is the system doing when this occurs (i.e. indexing and searching at the same time or just searching etc)?
Thanks. Dmitry.
Bruce Ritchie wrote:
We've started using the new compound indexing functionality available in lucene 1.3 final however we've been seeing an issue that I haven't quite resolved and was hoping that people on this list might be able to lend help on.
The stack trace of the issue is:
2004.01.13 18:15 [com.jivesoftware.forum.database.DbQuery.executeQuery(DbQuery.java:402)]
java.io.FileNotFoundException: /var/search/BfUfO9s1/_qb5.fnm (No such file or directory)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:204)
at org.apache.lucene.store.FSInputStream$Descriptor.<init>(FSDirectory.java:392)
at org.apache.lucene.store.FSInputStream.<init>(FSDirectory.java:421)
at org.apache.lucene.store.FSDirectory.openFile(FSDirectory.java:294)
at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:80)
at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:138)
at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:120)
at org.apache.lucene.index.IndexReader$1.doBody(IndexReader.java:122)
at org.apache.lucene.store.Lock$With.run(Lock.java:148)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:111)
at com.jivesoftware.forum.database.DbSearchManager.getSearcher(DbSearchManager.java:897)
...
The error seems obvious enough from a quick glance of the above code - the file was been deleted. However, since we are using compound indexing there should be a /var/search/BfUfO9s1/_qb5.cfs file, however that file does not exist. The only files in the search directory are:
-bash-2.05b$ ls -l total 6908 -rw-r--r-- 1 resin js 4 Jan 13 18:14 deletable -rw-r--r-- 1 resin js 7029055 Jan 13 10:03 _qau.cfs -rw-r--r-- 1 resin js 9 Jan 13 18:14 _qb5.del -rw-r--r-- 1 resin js 16060 Jan 13 18:14 _qb7.cfs -rw-r--r-- 1 resin js 34 Jan 13 18:14 segments
From what I can tell it seems to me that the segments are somehow getting out of sync with what is actually on the filesystem somehow.
Any suggestions are where to start looking for the cause of this bug?
Regards,
Bruce Ritchie
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
