uschindler commented on issue #15054:
URL: https://github.com/apache/lucene/issues/15054#issuecomment-3189192328
Hi,
we found a bug in the 9.x branch regarding the grouping of files belonging
to segments due to the lack of `READONCE` for `*.si` files. This consumes a lot
of mappings for si files which are not freed. Basically on every
readSegmentInfos a new mapping is created.
If you have a dump of all mapped files /proc/pid/maps can you check if you
see many `*.si` files? If this is the case the problem you are seing here is
just a side effect
In any case, to turn of the grouping completely, don't use
`...sharedArenaMaxPermits=1` and instead turn the grouping off completely for a
specific directory instance (e.g., the remote one) using:
```java
mmapDir.setGroupingFunction(MMapDirectory.NO_GROUPING);
```
To work around the possible 9.x issue with segment info files, you can try
on normal directories that are used for normal indexing to turn that off:
```java
mmapDir.setGroupingFunction(fn -> fn.endsWith(".si") ? Optional.empty() :
MMapDirectory.GROUP_BY_SEGMENT.apply(fn));
```
(or alternatively turn off grouping everywhere and completely to finally get
the root cause. If you then still get map failed and exhausted mappings, you
have a leak in your code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
