[
https://issues.apache.org/jira/browse/LUCENE-7834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16015692#comment-16015692
]
Adrien Grand commented on LUCENE-7834:
--------------------------------------
Agreed with Robert about the false sense of security. I remember I hated when
norms would only start using memory when they were used. I have seen some
threads about the memory usage of terms recently and I'm wondering whether this
is misusage of Lucene somehow such as too many fields/segments or whether the
terms index is hitting adversarial patterns of the terms dictionary that make
it use lots of memory. In the latter case maybe this is something we can
address?
> BlockTree's terms index should be loaded into memory lazily
> -----------------------------------------------------------
>
> Key: LUCENE-7834
> URL: https://issues.apache.org/jira/browse/LUCENE-7834
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/codecs
> Reporter: David Smiley
> Assignee: David Smiley
> Attachments: LUCENE_7834_BlockTreeLazyFST.patch
>
>
> I propose that BlockTree delay loading the FST prefix terms index into memory
> until {{terms(String)}} is first called. This seems like how it should work
> since if someone wants to eager load then they can use {{IndexReaderWarmer}}.
> By making the FST lazy load, we can be more NRT friendly (if some fields are
> rarely used), and also save memory (if some fields are rarely used).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
