Hi Justin,
This question is better for the OS community, as some of the settings are
specific to OpenSearch. Will really appreciate, if you can create an OpenSearch
issue <https://github.com/opensearch-project/OpenSearch/issues>.
We can always follow up with the Lucene community, if it turns out Lucene
related.
Thanks
Ankit
On Tue, May 6, 2025 at 6:45 PM Justin Borromeo
<justin.borro...@glean.com.invalid> wrote:
> Hi all,
>
> After upgrading our OpenSearch cluster from 2.16.0 to 2.19.1 (moving from
> Lucene 9.10 to Lucene 9.12), our largest clusters started crashing with the
> following error:
>
> # There is insufficient memory for the Java Runtime Environment to
> continue.
>
> # Native memory allocation (malloc) failed to allocate 2097152 bytes. Error
> detail: AllocateHeap
>
> We narrowed down the issue to the vm max map count (262144) being reached.
> Prior to server crash, we see map count (measured by `cat /proc/{pid}/maps
> | wc -l`) approach the 262144 limit we set. Looking at one of the outputs
> of `cat /proc/{pid}/maps`, we observed that 246K of the 252K maps are for
> deleted doc values (.dvd) files.
>
> Is this expected? If so, were there any changes in the Lucene codebase
> between those two versions that could have caused this? Any suggestions on
> debugging?
>
> Thanks in advance and sorry if this is a better question for the OS
> community or the Lucene developer list.
>
> Justin Borromeo
>
