For humongous region waste, you can calculate the waste without observing the fillers by using the non-filler object size and region size, as you said (an explicit counter for this would be nice, and should be very easy to add).
For TLAB waste, it's logged directly: "[0.365s][debug][gc,tlab] GC(0) TLAB totals: thrds: 5 alloc-frac: 54.6% refills: 104 max: 100 slow allocs: 60 max 60 waste: 0.9% gc: 65832B(0.9%) max: 16416B refill: 3096B(0.0%) max: 3096B", and these counters are also available through hsPerfData. So it seems like the main causes will still be observable. Oli ________________________________ From: Kirk Pepperdine Sent: Thursday, August 27, 2026 13:12 To: Gillespie, Oli Cc: [email protected] Subject: RE: [EXTERNAL] [External] : Re: Omit GC filler objects from heap dump My first heuristic was finding humongous allocations and knowing the region size one can calculate the waste. From there one can find int[] arrays that are ~ the size of the tlab. It’s not a perfect heuristic but it’s been helpful. Kind regards, Kirk On Aug 27, 2026, at 10:56 AM, Gillespie, Oli <[email protected]> wrote: Thanks. Could you share how you find and distinguish them? I suppose, 'large unreachable int[]' is one heuristic. Perhaps the fact that the contents are unzeroed heap memory makes them look quite distinctive too? What sources do you look for and what are the remedies? I'm wondering if there's any more direct measurement that's either already available or could be made available. For example, for the filler at the end of humongous G1 regions, it would be easy for G1 to track and report directly. Do you always operate on hprof files? The filler objects will still be present in GC.class_histogram and JVMTI heap iteration, do those cover your needs? "I would leave it up to the tooling" - my problem is that the hprof doesn't retain the information to (perfectly) distinguish filler arrays from real int arrays. If they were clearly marked, then I might agree. But even then, the increased dump size and the fact that leaks random unused heap memory remain. Those could possibly be addressed by a flag to opt-in to dumping fillers, and to dump them zeroed, but I think that's too much. Oli ________________________________ From: Kirk Pepperdine Sent: Thursday, August 27, 2026 08:26 To: Gillespie, Oli Cc: [email protected] Subject: RE: [EXTERNAL] [External] : Re: Omit GC filler objects from heap dump Hi Ali, I share your frustration with filler objects, they are confusing. That said, I have tooling that finds these filler objects and works to sort out if anything can be done to reduce this type of heap “wastage”. I have some heuristics that help me distinguish source and propose possible remedies. So I would like to object to having this information hidden from me. I would leave it up to the tooling to decide if these filler classes should filtered out. Kind regards, Kirk Pepperdine On Aug 26, 2026, at 5:50 PM, Gillespie, Oli <[email protected]> wrote: Hearing no objections, I created https://github.com/openjdk/jdk/pull/32542 to implement this. I found the existinghttps://bugs.openjdk.org/browse/JDK-8372389 which had the same idea. ________________________________ From: Gillespie, Oli <[email protected]<mailto:[email protected]>> Sent: 24 August 2026 10:20 To: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> Subject: Re: Omit GC filler objects from heap dump Bumping this - does anyone have any objection to excluding filler objects from heap dumps? Oli ________________________________ From: Gillespie, Oli Sent: 18 August 2026 15:29 To: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> Subject: Omit GC filler objects from heap dump Proposal: exclude filler objects from heap dumps. They confuse more than they help. I've had a few cases where filler arrays in heap dumps confused me for quite a while. It's okay in things like GC.class_histogram as they have the identifying name of "[Ljdk.internal.vm.FillerArray", but in other tools like Eclipse MAT they still show up as plain int arrays. Normally unreachable objects would not be looked at much, I get that, but here are the cases I heard about recently. 1 - HeapDumpOnOutOfMemoryError produced a heap dump, which only had around 20% space filled with reachable objects, the rest unreachable. I figured the unreachable stuff must be relevant to the OOM. Actually, the 'OutOfMemoryError: java heap space' was triggered by GCLocker retry count exceeded, the JVM was not out of heap space at all. 2 - a service had an issue with periodic allocation spikes causing G1 full GC. They enabled HeapDumpBeforeFullGC to understand the garbage that was triggering the full gc. Many large int arrays seemed suspicious, but in the end they were mostly fillers. I propose the simple option of excluding filler objects from heap dumps, but if there's a way to instead make these identifiable in MAT and other tools, that would also be fine. I wonder if Valhalla might provide any options here to represent the filler elements not as primitives but as values? Feedback appreciated! Oli (Filler object background - https://tschatzl.github.io/2022/09/26/jdk-vm-internal-fillerarray.html) Amazon Development Centre (London) Ltd.Registered in England and Wales with registration number 04543232 with its registered office at 1 Principal Place, Worship Street, London EC2A 2FA, United Kingdom. Amazon Development Centre (London) Ltd.Registered in England and Wales with registration number 04543232 with its registered office at 1 Principal Place, Worship Street, London EC2A 2FA, United Kingdom. Amazon Development Centre (London) Ltd. Registered in England and Wales with registration number 04543232 with its registered office at 1 Principal Place, Worship Street, London EC2A 2FA, United Kingdom.
