Have you tried a System.gc before opening the index? And check how much
memory is available before the open. Does the open seem to be taking more
memory that could reasonably be expected? How big is the index, how many
docs, how many unique terms in that index?
1.8G doesn't sound like a lot. You may simply have to bite the bullet and
move to an environment with more memory. Or, split the index and run on
multiple machines/JVMs.
Have many deletes been performed? How many segments are there? Maybe you can
merge using a standalone program that doesn't have the rest of your app
consuming memory.
-- Jack Krupansky
-----Original Message-----
From: nishesh.gu...@emc.com
Sent: Friday, June 01, 2012 7:53 PM
To: java-user@lucene.apache.org
Subject: OOM during IndexReader open
Hi,
I am getting the following OOM consistently whenever the index is opened .
Is it because now the index is holding too many terms ? Our application (
that has Lucene 2.9.3 ) already has reached 1.8G of virtual memory.
Is there a way I can avoid OOM during IndexReader open because of too many
terms ?
2012-05-30 14:12:46,139 [MC:10.2.2.1-1332015615933-1264
FS:parinfprdcl01k_655394_0] ERROR util - Context: MC: 10.2.2
.1-1332015615933-1264 failed at the state LocalIndexOptimizeAndCompress,
NumActiveThreads = 64. Error: java.lang.OutO fMemoryError: Java heap
space % STACK:
org.apache.lucene.index.TermInfosReader.<init>(TermInfosReader.java:90)
org.apache.lucene.index.SegmentReader$CoreReaders.<init>(SegmentReader.java:122)
org.apache.lucene.index.SegmentReader.get(SegmentReader.java:652)
org.apache.lucene.index.SegmentReader.get(SegmentReader.java:613)
org.apache.lucene.index.DirectoryReader.<init>(DirectoryReader.java:106)
org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:78)
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:683)
org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:71)
org.apache.lucene.index.IndexReader.open(IndexReader.java:482)
org.apache.lucene.index.IndexReader.open(IndexReader.java:320)
com.kazeon.search.indexingengine.context.MergerContext.deleteAgainstFiler(MergerContext.java:1357)
com.kazeon.search.indexingengine.context.MergerContext.deleteAgainstFiler(MergerContext.java:656)
com.kazeon.search.indexingengine.statemachine.modifiers.merger.LocalIndexOptimizeAndCompressModifier.modifyState
Aux(LocalIndexOptimizeAndCompressModifier.java:303)
com.kazeon.search.indexingengine.statemachine.modifiers.merger.LocalIndexOptimizeAndCompressModifier.mergeAllICs
(LocalIndexOptimizeAndCompressModifier.java:175)
com.kazeon.search.indexingengine.statemachine.modifiers.merger.LocalIndexOptimizeAndCompressModifier.modifyState
(LocalIndexOptimizeAndCompressModifier.java:106)
com.kazeon.util.scoreboard.WorkerThread.run(WorkerThread.java:31)
thanks,
Nishesh
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
