Lucene 2.4.x is quite ancient by now ... FSDirectory.FSIndexInput is single-threaded in seeking/reading bytes, which I think explains your 1 and 4. Try using MMapDirectory, if you are using a 64 bit JVM or if your index is tiny. Newer Lucene versions also have NIOFSDirectory, which is thread-friendly on Unix (but not on Windows due to a JVM bug).
For 2 and 3, creating a FieldCache entry is also single threaded, but this is a one-time event on the first search to the IndexReader requiring that entry. Lucene 4.x adds doc values which are much more efficient to init at search time. But, what changed in your app? Perhaps there's less RAM available to the OS for caching IO pages (this could explain 1 and 4)? Mike McCandless http://blog.mikemccandless.com On Thu, Jul 18, 2013 at 6:46 AM, RameshIyerV <[email protected]> wrote: > Hi All, > > I need some help in analyzing some contentions we observe in the Lucene > execution. > > We are supporting the Sterling 9.0 fulfillment application and it uses > Lucene 2.4 for catalog search functionality. > > ---The Issue---- > This system is Live in production since Nov 2012 and only recently (Mid June > 2013) our application is forming stuck threads during lucene invocations, > this causes our application to crash. > > This occurs 2 - 3 times a week, on other days we see spikes of very slow > performance on the exact places that causes stuck threads. > > ---The research--- > We have validated that the data or the usage has not grown between Jan 2012 > & now. > > We took snapshot of the code execution (through visual VM) and for slow > running treads we validated that too much time is spent at certain spots > (these very same spots appear in the stack trace of the stuck threads). > > ---Help needed--- > If you can guide me on what kind of contentions (heap, IO, Data, CPU, JVM > params) can cause such a behavior it will really help. > > > ---Lucene Invocation contentions observed--- > (We find stuck threads & slowness at the following spots, ordered in the > order of severity [high to low]) > 1. java.io.RandomAccessFile.readBytes(Native Method) > java.io.RandomAccessFile.read(RandomAccessFile.java:338) > > org.apache.lucene.store.FSDirectory$FSIndexInput.readInternal(FSDirectory.java:596) > > org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:136) > > 2. org.apache.lucene.index.TermBuffer.toTerm(TermBuffer.java:122) > org.apache.lucene.index.SegmentTermEnum.term(SegmentTermEnum.java:167) > > org.apache.lucene.search.FieldCacheImpl$10.createValue(FieldCacheImpl.java:373) > > org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:71) > > org.apache.lucene.search.FieldCacheImpl.getStringIndex(FieldCacheImpl.java:351) > > 3. org.apache.lucene.store.IndexInput.readVInt(IndexInput.java:80) > org.apache.lucene.index.TermBuffer.read(TermBuffer.java:65) > org.apache.lucene.index.SegmentTermEnum.next(SegmentTermEnum.java:127) > > org.apache.lucene.search.FieldCacheImpl$10.createValue(FieldCacheImpl.java:389) > > org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:71) > > org.apache.lucene.search.FieldCacheImpl.getStringIndex(FieldCacheImpl.java:351) > > 4. java.io.RandomAccessFile.seek(Native Method) > > org.apache.lucene.store.FSDirectory$FSIndexInput.readInternal(FSDirectory.java:591) > > org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:136) > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Contentions-observed-in-lucene-execution-tp4078796.html > Sent from the Lucene - Java Developer mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
