Hello,
>From the thread linked there, its not clear to me the problem relates
to lucene (vs being e.g. a bug in netty, or too many threads, or
potentially many other problems).
Can you first try to determine to breakdown your problematic "RSS"
from the operating system? Maybe this helps determine if your issue is
with an anonymous mapping (ByteBuffer.allocateDirect) or file mapping
(FileChannel.map).
WIth recent kernels you can break down RSS with /proc/pid/XXX/status
(RssAnon vs RssFile vs RssShmem):
http://man7.org/linux/man-pages/man5/proc.5.html
If your kernel is old you may have to go through more trouble (summing
up stuff from smaps or whatever)
On Wed, Aug 30, 2017 at 9:58 PM, Erik Stephens <mreriksteph...@gmail.com> wrote:
> Our elasticsearch processes have been slowly consuming memory until a kernel
> OOM kills it. Details are here:
>
> https://github.com/elastic/elasticsearch/issues/26269
> <https://github.com/elastic/elasticsearch/issues/26269>
>
> To summarize:
>
> - Explicit GC is enabled
> - MaxDirectMemorySize is set
> - Total memory usage for the process is roughly heap (30G) + mmap'd
> (unbounded) + 1-2G (meta space, threads, etc)
>
> The crowd is suggesting "Don't worry. You want to use all that memory." I
> understand that sentiment except for:
>
> - The process eventually gets OOM killed.
> - I need to support multiple processes on same machine and need a more
> predictable footprint.
>
> It seems to be relatively common knowledge that direct byte buffers require a
> GC to trigger their freedom. However, full GC's are happening but not
> resulting in a reduction of resident mmap'd memory.
>
> Any pointers to source code, settings, or tools are much appreciated. Thanks!
>
> --
> Erik
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
