[
https://issues.apache.org/jira/browse/LUCENE-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854398#action_12854398
]
Michael McCandless commented on LUCENE-2376:
--------------------------------------------
Is this the same issue as LUCENE-2361? Or is something different here? Are
you using the same settings in IndexWriter?
The infoStream output is now much more complete -- eg I see where the OOME
happens at the end.
It does look like the cause here is an insane number of fields:
{code}
purge field=Cor414D5120514D475231202020202020204A94834B21C22E6B
purge field=Cor414D5120514D475231202020202020204A94834B21C22E6C
purge field=Cor414D5120514D475231202020202020204A94834B21C22E6D
purge field=Cor414D5120514D475231202020202020204A94834B21C22E6E
purge field=Cor414D5120514D475231202020202020204A94834B21C22E70
purge field=Cor414D5120514D475231202020202020204A94834B21C22E72
{code}
I count 43K purge fields lines in your log (and that's an undercount since
you've pruned some). Are you disabling norms for these fields (if not, this
will certainly lead to OOME)? In your design, is there any limit to the number
of fields? Or will each document have a new field name?
Can you run CheckIndex on the index and post the output?
> java.lang.OutOfMemoryError:Java heap space
> ------------------------------------------
>
> Key: LUCENE-2376
> URL: https://issues.apache.org/jira/browse/LUCENE-2376
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 2.9.1
> Environment: Windows
> Reporter: Shivender Devarakonda
> Attachments: InfoStreamOutput.txt
>
>
> I see an OutOfMemory error in our product and it is happening when we have
> some data objects on which we built the index. I see the following
> OutOfmemory error, this is happening after we call Indexwriter.optimize():
> 4/06/10 02:03:42.160 PM PDT [ERROR] [Lucene Merge Thread #12] In thread
> Lucene Merge Thread #12 and the message is
> org.apache.lucene.index.MergePolicy$MergeException:
> java.lang.OutOfMemoryError: Java heap space
> 4/06/10 02:03:42.207 PM PDT [VERBOSE] [Lucene Merge Thread #12] [Manager]
> Uncaught Exception in thread Lucene Merge Thread #12
> org.apache.lucene.index.MergePolicy$MergeException:
> java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:351)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:315)
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at java.util.HashMap.resize(HashMap.java:462)
> at java.util.HashMap.addEntry(HashMap.java:755)
> at java.util.HashMap.put(HashMap.java:385)
> at org.apache.lucene.index.FieldInfos.addInternal(FieldInfos.java:256)
> at org.apache.lucene.index.FieldInfos.read(FieldInfos.java:366)
> at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:71)
> at
> org.apache.lucene.index.SegmentReader$CoreReaders.<init>(SegmentReader.java:116)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:638)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:608)
> at
> org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:686)
> at
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4979)
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4614)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:235)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:291)
> 4/06/10 02:03:42.895 PM PDT [ERROR] this writer hit an OutOfMemoryError;
> cannot complete optimize
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org
