$SUBJECT, if the table has, say, 2000 partitions. The main reason seems to be that RelationBuildPartitionDesc() will be called that many times within the same transaction, which perhaps we cannot do much about right away. But one thing we could do is to reduce the impact of memory allocations it does. They are currently leaked into the caller's context, which may not be reset immediately (such as PortalHeapMemory). Instead of doing it in the caller's context, use a temporary context that is deleted before returning. Attached is a patch for that. On my local development VM, `drop table table_with_2000_partitions` finished in 27 seconds with the patch instead of more than 20 minutes that it currently takes.
Thoughts? Adding this to the open items list. Thanks, Amit PS: this was actually mentioned by Ragnar Ouchterlony who reported some bugs back in declarative partitioning in January [1] [1] https://www.postgresql.org/message-id/17d89e08-874b-c1b1-aa46-12d5afb26235%40agama.tv
>From d435dbe04de935ceb092d4a05cfb083f2ede7f19 Mon Sep 17 00:00:00 2001 From: amit <amitlangot...@gmail.com> Date: Fri, 13 Jan 2017 17:47:04 +0900 Subject: [PATCH] Set up a temp context in RelationBuildPartitionDesc Memory allocations while reading PartitionBoundSpec from the catalog and converting to the internal representation to be put into relcache currently leak to whatever context it was when we entered RelationBuildPartitionDesc(). Instead set up a temporary context as a workspace for those allocations and delete it before returning. --- src/backend/catalog/partition.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/src/backend/catalog/partition.c b/src/backend/catalog/partition.c index e0d2665a91..900b144af5 100644 --- a/src/backend/catalog/partition.c +++ b/src/backend/catalog/partition.c @@ -159,7 +159,8 @@ RelationBuildPartitionDesc(Relation rel) nparts; PartitionKey key = RelationGetPartitionKey(rel); PartitionDesc result; - MemoryContext oldcxt; + MemoryContext oldcxt, + tmpcxt; int ndatums = 0; @@ -178,6 +179,16 @@ RelationBuildPartitionDesc(Relation rel) if (key == NULL) return; + /* + * Create a temporary context to not leak into the outer potentially long- + * running context + */ + /* Now build the actual relcache partition descriptor */ + tmpcxt = AllocSetContextCreate(CurrentMemoryContext, + "RelationBuildPartitionDesc_workspace", + ALLOCSET_DEFAULT_SIZES); + oldcxt = MemoryContextSwitchTo(tmpcxt); + /* Get partition oids from pg_inherits */ inhoids = find_inheritance_children(RelationGetRelid(rel), NoLock); @@ -431,7 +442,7 @@ RelationBuildPartitionDesc(Relation rel) rel->rd_pdcxt = AllocSetContextCreate(CacheMemoryContext, RelationGetRelationName(rel), ALLOCSET_DEFAULT_SIZES); - oldcxt = MemoryContextSwitchTo(rel->rd_pdcxt); + MemoryContextSwitchTo(rel->rd_pdcxt); result = (PartitionDescData *) palloc0(sizeof(PartitionDescData)); result->nparts = nparts; @@ -580,6 +591,7 @@ RelationBuildPartitionDesc(Relation rel) } MemoryContextSwitchTo(oldcxt); + MemoryContextDelete(tmpcxt); rel->rd_partdesc = result; } -- 2.11.0
-- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
