[
https://issues.apache.org/jira/browse/LUCENE-8267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448272#comment-16448272
]
Robert Muir commented on LUCENE-8267:
-------------------------------------
There are a lot of other alternatives to putting data in heap memory directly
in the postings format.
The best (IMO) is for the user to use MMapDirectory.preload with the standard
index format. This way it doesn't impact their java heap and they use supported
index format. Users can also use RAMDirectory/FileSwitchDirectory to load
specified files into heap.
Finally, users can use FSTPostingsFormat which will load *term dictionary only*
into heap fst. This is way different than Memory/Direct which load not only
terms but also postings lists and positions and stuff all into heap RAM.
So i don't really see any technical merit for your objection: there are many
other ways to have a ram-resident terms dictionary, many of them better than
the inefficient Memory/Direct formats.
> Remove memory codecs from the codebase
> --------------------------------------
>
> Key: LUCENE-8267
> URL: https://issues.apache.org/jira/browse/LUCENE-8267
> Project: Lucene - Core
> Issue Type: Task
> Reporter: Dawid Weiss
> Priority: Major
>
> Memory codecs (MemoryPostings*, MemoryDocValues*) are part of random
> selection of codecs for tests and cause occasional OOMs when a test with huge
> data is selected. We don't use those memory codecs anywhere outside of tests,
> it has been suggested to just remove them to avoid maintenance costs and OOMs
> in tests. [1]
> [1] https://apache.markmail.org/thread/mj53os2ekyldsoy3
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
