Hi, On 2023-05-23 18:23:00 +0200, Tomas Vondra wrote: > This means that for an UPDATE with triggers, we may end up calling this > for each row, possibly multiple bitmaps. And those bitmaps are allocated > in ExecutorState, so won't be freed until the end of the query :-(
Ugh. I've wondered about some form of instrumentation to detect such issues before. It's obviously a problem that we can have fairly large leaks, like the one you just discovered, without detecting it for a couple years. It's kinda made worse by the memory context infrastructure, because it hides such issues. Could it help to have a mode where the executor shutdown hook checks how much memory is allocated in ExecutorState and warns if its too much? There's IIRC a few places that allocate large things directly in it, but most of those probably should be dedicated contexts anyway. Something similar could be useful for some other long-lived contexts. > The bitmaps are typically fairly small (a couple bytes), but for wider > tables it can be a couple dozen bytes. But it's primarily driven by > number of updated rows. Random aside: I've been wondering whether it'd be worth introducing an in-place representation of Bitmap (e.g. if the low bit is set, the low 63 bits are in-place, if unset, it's a pointer). > Attached is a patch, restoring the pre-12 behavior for me. Hm. Somehow this doesn't seem quite right. Shouldn't we try to use a shorter lived memory context instead? Otherwise we'll just end up with the same problem in a few years. Greetings, Andres Freund
