[
https://issues.apache.org/jira/browse/LUCENE-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12855615#action_12855615
]
Earwin Burrfoot commented on LUCENE-2376:
-----------------------------------------
A field is basically an index in itself. As such, it has a bunch of bookkeeping
data associated with it, which is always loaded into memory, unlike field
values, which are mostly kept to disk. Also, like I explained in LUCENE-2361,
field names are interned as they take part in string comparisons very often
internally.
So creating an insane amount of unique fields is like begging for either OOM or
OOPermGen. Can you elaborate, why do you need to store unique number in field
name rather then field value?
> java.lang.OutOfMemoryError:Java heap space
> ------------------------------------------
>
> Key: LUCENE-2376
> URL: https://issues.apache.org/jira/browse/LUCENE-2376
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 2.9.1
> Environment: Windows
> Reporter: Shivender Devarakonda
> Attachments: CheckIndex_JavaHeapOOM.txt,
> CheckIndex_PermGenSpaceOOM.txt, InfoStreamOutput.txt
>
>
> I see an OutOfMemory error in our product and it is happening when we have
> some data objects on which we built the index. I see the following
> OutOfmemory error, this is happening after we call Indexwriter.optimize():
> 4/06/10 02:03:42.160 PM PDT [ERROR] [Lucene Merge Thread #12] In thread
> Lucene Merge Thread #12 and the message is
> org.apache.lucene.index.MergePolicy$MergeException:
> java.lang.OutOfMemoryError: Java heap space
> 4/06/10 02:03:42.207 PM PDT [VERBOSE] [Lucene Merge Thread #12] [Manager]
> Uncaught Exception in thread Lucene Merge Thread #12
> org.apache.lucene.index.MergePolicy$MergeException:
> java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:351)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:315)
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at java.util.HashMap.resize(HashMap.java:462)
> at java.util.HashMap.addEntry(HashMap.java:755)
> at java.util.HashMap.put(HashMap.java:385)
> at org.apache.lucene.index.FieldInfos.addInternal(FieldInfos.java:256)
> at org.apache.lucene.index.FieldInfos.read(FieldInfos.java:366)
> at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:71)
> at
> org.apache.lucene.index.SegmentReader$CoreReaders.<init>(SegmentReader.java:116)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:638)
> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:608)
> at
> org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:686)
> at
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4979)
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4614)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:235)
> at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:291)
> 4/06/10 02:03:42.895 PM PDT [ERROR] this writer hit an OutOfMemoryError;
> cannot complete optimize
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org
