Hi,
I'm starting a new thread for one of the issues reported by Sawada-san at [1].
This is a memory leak on CacheMemoryContext when using pgoutput via SQL APIs:
/* Map must live as long as the session does. */
oldctx = MemoryContextSwitchTo(CacheMemoryContext);
entry->attrmap = build_attrmap_by_name_if_req(indesc, outdesc, false);
MemoryContextSwitchTo(oldctx);
RelationClose(ancestor);
entry->attrmap is pfree'd only when validating the RelationSyncEntry
so remains even after logical decoding API calls.
This issue can be reproduced with the following test:
-- Create table
create table t1( c1 int, c2 int, c3 int) PARTITION BY LIST (c1);
create table t1_1( c2 int, c1 int, c3 int);
ALTER TABLE t1 ATTACH PARTITION t1_1 FOR VALUES IN (0, 1, 2, 3);
-- Create publication
create publication pub1 FOR TABLE t1, t1_1 WITH
(publish_via_partition_root = true);
-- Create replication slot
SELECT 'init' FROM pg_create_logical_replication_slot('test', 'pgoutput');
-- Run the below steps between Test start and Test end many times to
see the memory usage getting increased
-- Test start
insert into t1 values( 1);
SELECT count(*) FROM pg_logical_slot_get_binary_changes('test', NULL,
NULL, 'proto_version', '4', 'publication_names', 'pub1');
-- Memory increases after each invalidation and insert
SELECT * FROM pg_backend_memory_contexts where name = 'CacheMemoryContext';
-- Test end
The attached patch resolves a memory leak by ensuring that the
attribute map is properly freed during plugin shutdown. This process
is triggered by the SQL API when the decoding context is being
released. Additionally, I am conducting a review to identify and
address any similar memory leaks that may exist elsewhere in the code.
[1] -
https://www.postgresql.org/message-id/CAD21AoDkAhQVSukOfH3_reuF-j4EU0-HxMqU3dU%2BbSTxsqT14Q%40mail.gmail.com
Regards,
Vignesh
From 36e5c10105d934da0d51474b0ad7c5bd9087e2aa Mon Sep 17 00:00:00 2001
From: Vignesh <vignes...@gmail.com>
Date: Wed, 11 Dec 2024 08:57:06 +0530
Subject: [PATCH v1] Fix memory leak in pgoutput relation attribute map
The pgoutput module caches relation attribute map and frees it upon
invalidation. However, this was not getting freed incase of SQL functions like
pg_logical_slot_{get,peek}_changes() which would bloat its cache memory usage.
Every pg_logical_slot_{get,peek}_changes() call for changes on partition table
creates more bloat. To address this, this relation attribute map is freed while
the output plugin is shutdown.
---
src/backend/replication/pgoutput/pgoutput.c | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index b50b3d62e3..f1ef13d313 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -1747,6 +1747,16 @@ pgoutput_shutdown(LogicalDecodingContext *ctx)
{
if (RelationSyncCache)
{
+ RelationSyncEntry *entry;
+ HASH_SEQ_STATUS status;
+
+ hash_seq_init(&status, RelationSyncCache);
+ while ((entry = (RelationSyncEntry *) hash_seq_search(&status)) != NULL)
+ {
+ if (entry->attrmap)
+ free_attrmap(entry->attrmap);
+ }
+
hash_destroy(RelationSyncCache);
RelationSyncCache = NULL;
}
--
2.43.0
