Hi Thomas Thanks for your input. I took me a while to have a stable system again to repeat measurements.
I have tried setting G1HeapRegionSize to 16M on one instance (8M is default) and I notice lower GC memory usage: GC (reserved=1117MB -18MB, committed=1117MB -18MB) vs GC (reserved=1604MB +313MB, committed=1604MB +313MB) It seems more stable too. However, "Internal" is still relatively high for a 25G heap, and there is no much difference between instances: Internal (reserved=2132MB -7MB, committed=2132MB -7MB) Milan Mimica, Senior Software Engineer / Division Lead ________________________________________ From: Thomas Schatzl <thomas.scha...@oracle.com> Sent: Monday, January 23, 2017 11:41 To: Milan Mimica; hotspot-gc-use@openjdk.java.net Subject: Re: G1 native memory consumption Hi Milan, On Fri, 2017-01-20 at 20:24 +0000, Milan Mimica wrote: > Hi > > I'm inspecting memory consumption issues of a service running on > java-8u102, linux. The service is running for a few days now, and in > a few days more it would consume all of 32GB physical memory > available, and get killed by OOM Killer. > Questions: > - If the code is not allocating any significant off-heap memory, > neither by Unsafe.allocateMemory or by external library, isn't 7GB > native memory overhead supposed to be enough for a 25GB heap? > - Why so much memory spent on "Internal" category, apparently from G1 > thread? G1 remembered sets typically consume approximately 10% of java heap, depending on application, heap and your remembered set configuration. This remembered set contains information that is necessary to be able to do incremental and partial old generation compaction. So the 7GB should be sufficient. You can decrease remembered set overhead by e.g. increasing heap region size. Given the heap size you mentioned it seems it would be worth a try to go to 16M regions. > Find the attached jemalloc heap profile, showing "live" > allocations that happened in about 30 hour timespan, and a > NMT profile of approximately same period. > Profiling was done after some warm-up time time, and with a > manually triggered Full GC in between just to give the JVM a chance > to clean up everything. Remembered set memory consumption should level out after some time, where "some" may be quite a bit of time. I.e. it may take much longer than for other collectors. It also depends on the memory allocator. Do you have any new measurements after this weekend? It should have levelled out by this time. JDK9 contains some improvements on memory usage in exactly this area. There will likely be further improvements in this area going forward. Thanks, Thomas _______________________________________________ hotspot-gc-use mailing list hotspot-gc-use@openjdk.java.net http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use
