Re: Lots of memory allocated when reassigning Large Objects

[Justin Pryzby]

On Mon, Nov 29, 2021 at 01:49:24PM +0100, Guillaume Lelarge wrote:
> One of our customers had a very bad issue while trying to reassign objects
> from user A to user B. He had a lot of them, and the backend got very
> hungry for memory. It finally all went down when the linux kernel decided
> to kill the backend (-9 of course).

> Memory is released at the end of the reassignment. So it's definitely not
> leaked forever, but only during the operation, which looks like a missing
> pfree (or something related). I've tried to find something like that in the
> code somewhere, but to no avail. I'm pretty sure I missed something, which
> is the reason for this email :)

I reproduced the issue like this.

psql postgres -c 'CREATE ROLE two WITH login superuser'
psql postgres two -c "SELECT lo_import('/dev/null') FROM 
generate_series(1,22111)" >/dev/null
psql postgres -c 'SET client_min_messages=debug; SET log_statement_stats=on;' 
-c 'begin; REASSIGN OWNED BY two TO pryzbyj; rollback;'

I didn't find the root problem, but was able to avoid the issue by creating a
new mem context.  I wonder if there are a bunch more issues like this.

unpatched:
!       33356 kB max resident size

patched:
!       21352 kB max resident size

diff --git a/src/backend/catalog/pg_shdepend.c 
b/src/backend/catalog/pg_shdepend.c
index 9ea42f805f..cbe7f04983 100644
--- a/src/backend/catalog/pg_shdepend.c
+++ b/src/backend/catalog/pg_shdepend.c
@@ -65,6 +65,7 @@
 #include "storage/lmgr.h"
 #include "utils/acl.h"
 #include "utils/fmgroids.h"
+#include "utils/memutils.h"
 #include "utils/syscache.h"
 
 typedef enum
@@ -1497,6 +1498,11 @@ shdepReassignOwned(List *roleids, Oid newrole)
                while ((tuple = systable_getnext(scan)) != NULL)
                {
                        Form_pg_shdepend sdepForm = (Form_pg_shdepend) 
GETSTRUCT(tuple);
+                       MemoryContext cxt, oldcxt;
+
+                       cxt = AllocSetContextCreate(CurrentMemoryContext, 
"shdepReassignOwned cxt",
+                                       ALLOCSET_DEFAULT_SIZES);
+                       oldcxt = MemoryContextSwitchTo(cxt);
 
                        /*
                         * We only operate on shared objects and objects in the 
current
@@ -1598,8 +1604,12 @@ shdepReassignOwned(List *roleids, Oid newrole)
                                        elog(ERROR, "unexpected classid %u", 
sdepForm->classid);
                                        break;
                        }
+
                        /* Make sure the next iteration will see my changes */
                        CommandCounterIncrement();
+
+                       MemoryContextSwitchTo(oldcxt);
+                       MemoryContextDelete(cxt);
                }
 
                systable_endscan(scan);