I've recently noticed this major problem with our indexer.

It only seems to happen running on 1.3 final and not on 1.3 RC2.

Essentially we have about 80 threads doing indexing... they all add to a RAMDirectory which after 10k documents does a commit to disk.

This appears to be a problem with FSDirectory .. which was not modified in 1.3 final (same as in RC2)

If anyone has any suggestions they would be appreciated!

Here's the full stacktrace ...

"ksa-task-thread-32" prio=1 tid=0x89d04140 nid=0x4666 runnable [ba7ff000..ba7ff8d0]
at java.io.RandomAccessFile.readBytes(Native Method)
at java.io.RandomAccessFile.read(RandomAccessFile.java:307)
at org.apache.lucene.store.FSInputStream.readInternal(FSDirectory.java:433)
- locked <0x46df74a0> (a org.apache.lucene.store.FSInputStream$Descriptor)
at org.apache.lucene.store.InputStream.refill(InputStream.java:196)
at org.apache.lucene.store.InputStream.readByte(InputStream.java:81)
at org.apache.lucene.store.InputStream.readVLong(InputStream.java:141)
at org.apache.lucene.index.SegmentTermEnum.next(SegmentTermEnum.java:117)
at org.apache.lucene.index.TermInfosReader.readIndex(TermInfosReader.java:110)
at org.apache.lucene.index.TermInfosReader.<init>(TermInfosReader.java:82)
at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:141)
at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:120)
at org.apache.lucene.index.IndexReader$1.doBody(IndexReader.java:118)
at org.apache.lucene.store.Lock$With.run(Lock.java:148)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:111)
- locked <0x4ccc3790> (a org.apache.lucene.store.FSDirectory)
at ksa.index.AdvancedIndexWriter.doDeleteByResource(AdvancedIndexWriter.java:264)
at ksa.index.AdvancedIndexWriter.commit(AdvancedIndexWriter.java:172)
- locked <0x497cfde0> (a java.lang.Object)
at ksa.index.IndexManager.getAdvancedIndexWriter(IndexManager.java:282)
- locked <0x497e0a50> (a java.lang.Object)
at ksa.robot.FeedTaskParserListener.onItemEnd(FeedTaskParserListener.java:315)
at org.apache.commons.feedparser.RSSFeedParser.doParseItem(RSSFeedParser.java:260)
at org.apache.commons.feedparser.RSSFeedParser.parse(RSSFeedParser.java:126)
at org.apache.commons.feedparser.FeedParser.parse(FeedParser.java:146)
at org.apache.commons.feedparser.FeedParser.parse(FeedParser.java:113)
at ksa.robot.FeedTask.doUpdate(FeedTask.java:375)
at ksa.robot.FeedTask.doUpdate(FeedTask.java:331)
at ksa.robot.FeedTask.run(FeedTask.java:147)
at ksa.robot.TaskThread.doProcessTask(TaskThread.java:213)
at ksa.robot.TaskThread.run(TaskThread.java:78)

--

Please reply using PGP:

http://peerfear.org/pubkey.asc

NewsMonster - http://www.newsmonster.org/
Kevin A. Burton, Location - San Francisco, CA, Cell - 415.595.9965
AIM/YIM - sfburtonator, Web - http://peerfear.org/
GPG fingerprint: 5FB2 F3E2 760E 70A8 6174 D393 E84D 8D04 99F1 4412
IRC - freenode.net #infoanarchy | #p2p-hackers | #newsmonster

begin:vcard
fn:Kevin Burton
n:Burton;Kevin
email;internet:[EMAIL PROTECTED]
x-mozilla-html:TRUE
version:2.1
end:vcard

Attachment: signature.asc
Description: OpenPGP digital signature

Reply via email to