On 08/06/2026 17:27, Tomas Vondra wrote: > On 6/8/26 15:32, Andrei Lepikhov wrote: >> Instead, I used the heaptrack tool, which gave me the following > Hmm, how does heaptrack deal with out memory pools? I was worried > existing memory profilers (like heaptrack) would get confused by our > memory contexts, attributing the whole block to the palloc that just > happens to allocate a new block. But that's not really right.
In theory, heaptrack does not catch pallocs inside a memory context and only tracks malloc and free calls. However, in my experience, it still helps identify the issue. This might be because the planner uses local memory contexts, and with [1] added, it could be even more localised. Usually, the problem is a cycle that grows too long, with many allocations within it that go unreleased, causing overconsumption and memory context growth. In this case, heaptrack may not point to the exact spot, but it still provides useful hints. Compared to other profilers (gperftools, jemalloc, valgrind massif), I like it more for its flexibility, flamegraphs, and an interface that the support team frequently uses (even on Windows somehow) - it is beneficial for the inter-department communication. Profilers don’t understand malloc/palloc postgres separation anyway, and to use them we need to annotate postgres calls. > > I see the heaptrack README claims it can work with memory pools after > annotating the code in some way. But there's not much details about > that. Also, it suggests valgrind/massif can already do that. > Looking into the interface [2,3], it looks quite clear to me - it even includes a Valgrind macros set that makes it more natively integrated. Thinking about implementation, my first wish was to create an extension and see how it actually works in practice, but we can’t inject a wrapper into the static MemoryContextMethods, which limits the ability of such an extension. Maybe with an umbrella planning memory context … Core-based annotation looks more interesting. There are questions about proper palloc/malloc counting that exist - I think it can be generally solved by employing heaptrack_pause / heaptrack_resume calls, if this topic sparks an interest. Or we can discuss some API flexibility on the HeapTrack side. [1] https://www.postgresql.org/message-id/[email protected] [2] https://github.com/KDE/heaptrack/blob/master/src/track/libheaptrack.h [3] https://github.com/KDE/heaptrack/blob/master/src/track/heaptrack_api.h -- regards, Andrei Lepikhov, pgEdge
