[ https://issues.apache.org/jira/browse/LUCENE-3953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adrien Grand resolved LUCENE-3953. ---------------------------------- Resolution: Fixed Fix Version/s: master (7.0) This looks mostly due to FieldCache storing data in a dense way. Now that FieldCache has been replaced with doc values, you would need many more fields in order to trigger OOMEs, but disk usage and indexing speed would still suffer from these sparse fields. I think we can consider it as fixed in master (7.0) thanks to the new sparse APIs for doc values and the new doc values format. That said, it is still advised to have dense fields whenever possible. We made sparse fields more cost-efficient, but there is still a penalty compared to dense fields. > Using many fields in filters, sorts causes OOM > ---------------------------------------------- > > Key: LUCENE-3953 > URL: https://issues.apache.org/jira/browse/LUCENE-3953 > Project: Lucene - Core > Issue Type: Bug > Components: core/search > Affects Versions: 4.0-ALPHA > Reporter: Des Lownds > Fix For: master (7.0) > > Attachments: FieldCacheLimitPatch.patch > > > I have an index with ~1M documents, and many uniquely named fields. Each > document only has a few fields populated, but there is a large set of dynamic > fields across all docs. In other words, theres many sparse fields. > Eventually the jvm will fall over due to OOM errors, because lucene's > fieldCache grows without bounds. Adding more heap does not help. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org