Hello all, I am using Lucene 4.10.4 in our custom software and faced a long-running GC while searching in an index.
I have attached the thread dump below *Index details*: Size of index = 7.8 GB Largest file size = 4.266GB (_0.fdt segment file) Results hit for the search query --> 4M records *Server RAM size*: 64 GB *Allotted heap size*: 16GB Due to this long-running GC, subsequent search calls to this server have failed. What can be the possible cause for this? Is this due to the larger segments? (We have a minimal number of segments in our index) If so, can you pls suggest to us the ideal index size and the number of segments? Thanks and Regards, Raahul
Stack Trace at java.lang.ref.ReferenceQueue.poll(ReferenceQueue.java:107) - locked <0x000000054ddd8a18> (a java.lang.ref.ReferenceQueue$Lock) at org.apache.lucene.util.WeakIdentityMap.reap(WeakIdentityMap.java:237) at org.apache.lucene.util.WeakIdentityMap.put(WeakIdentityMap.java:130) at org.apache.lucene.store.ByteBufferIndexInput.buildSlice(ByteBufferIndexInput.java:296) at org.apache.lucene.store.ByteBufferIndexInput.clone(ByteBufferIndexInput.java:259) at org.apache.lucene.store.ByteBufferIndexInput$SingleBufferImpl.clone(ByteBufferIndexInput.java:411) at org.apache.lucene.codecs.blocktree.SegmentTermsEnum.initIndexInput(SegmentTermsEnum.java:117) at org.apache.lucene.codecs.blocktree.SegmentTermsEnumFrame.loadBlock(SegmentTermsEnumFrame.java:143) at org.apache.lucene.codecs.blocktree.SegmentTermsEnum.seekExact(SegmentTermsEnum.java:506) at org.apache.lucene.index.TermContext.build(TermContext.java:95) at org.apache.lucene.search.TermQuery.createWeight(TermQuery.java:165) at org.apache.lucene.search.BooleanQuery$BooleanWeight.<init>(BooleanQuery.java:185) at org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:526) at org.apache.lucene.search.BooleanQuery$BooleanWeight.<init>(BooleanQuery.java:185) at org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:526) at org.apache.lucene.search.BooleanQuery$BooleanWeight.<init>(BooleanQuery.java:185) at org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:526) at org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:684) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:345) at com.**.Searcher.search(Searcher.java:868) at com.**.Searcher.search(Searcher.java:358) at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.dfs.DataNode$DataXceiver.invokeSearch(DataNode.java:9977) at org.apache.hadoop.dfs.DataNode$DataXceiver.doNewSearch(DataNode.java:10123) at org.apache.hadoop.dfs.DataNode$DataXceiver.access$3500(DataNode.java:2248) at org.apache.hadoop.dfs.DataNode$DataXceiver$CommonReqHandler.run(DataNode.java:2551) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) - <0x00000004d88ba3d8> (a java.util.concurrent.ThreadPoolExecutor$Worker)
--------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
