Hi, In [1], REPACK (CONCURRENTLY) was restricted to heap because nothing in TableAmRoutine says whether an AM can be logically decoded, and without decoding the changes made during the rewrite are silently lost. Masahiko suggested a callback for HEAD, and Bharath that it wait for a field report. Here is one: a table AM we are developing has its own resource manager with rm_decode, and the heap-only check refuses it.
The attached patch adds an optional relation_supports_logical_decoding callback, last in TableAmRoutine. An AM without it is taken not to support decoding, so existing AMs are unaffected. heap provides it, and REPACK asks the AM instead of comparing its routine with heap's. With the callback set in that AM, REPACK (CONCURRENTLY) with concurrent DML converges row for row with a heap oracle, and a logical slot attached during it sees no duplicate changes. The test module has two AMs that copy heap's routine, one without the callback. heap_getnext() insists on heap's own routine, so their tables can have no index or TOAST table; the test shows one refused and the other past the check. This is for v20 only. It is the second small TableAmRoutine addition that out-of-tree AMs need, after the reloptions patch in [2]. The two don't conflict (this one is last in the struct), and could be reviewed, or committed, as one series if that's easier. Maybe we only need one new test module instead of two. I'd welcome views on the shape: a callback, as suggested, or a plain flag. [1] https://postgr.es/m/apBkixO4xAGFoiT3@nathan [2] https://postgr.es/m/[email protected] ("Allow table AMs to define their own reloptions") cheers andrew -- Andrew Dunstan EDB: https://www.enterprisedb.com
From 1aef727a7e820e602cc77865bedceb72542a6bce Mon Sep 17 00:00:00 2001 From: Andrew Dunstan <[email protected]> Date: Thu, 19 Nov 2026 09:42:47 -0500 Subject: [PATCH] Add a table AM callback for logical decoding support. REPACK (CONCURRENTLY) loses the changes made during its rewrite if the table's AM can't be logically decoded. Commit e8636f2bed2 restricted it to heap for want of a way to ask the AM, which also refuses AMs that do support decoding. Add an optional relation_supports_logical_decoding callback, which heap provides, and have REPACK ask it. An AM without it is taken not to support decoding, so existing AMs are unaffected. Add test_tableam_logical, with two AMs that copy heap's routine, one without the callback. Discussion: https://postgr.es/m/apBkixO4xAGFoiT3@nathan --- doc/src/sgml/ref/repack.sgml | 4 +- doc/src/sgml/tableam.sgml | 12 ++++ src/backend/access/heap/heapam_handler.c | 12 +++- src/backend/commands/repack.c | 7 +-- src/include/access/tableam.h | 27 +++++++++ src/test/modules/Makefile | 1 + src/test/modules/meson.build | 1 + .../modules/test_tableam_logical/Makefile | 20 +++++++ .../test_tableam_logical/expected/repack.out | 45 +++++++++++++++ .../modules/test_tableam_logical/meson.build | 33 +++++++++++ .../test_tableam_logical/sql/repack.sql | 33 +++++++++++ .../test_tableam_logical--1.0.sql | 19 +++++++ .../test_tableam_logical.c | 56 +++++++++++++++++++ .../test_tableam_logical.control | 5 ++ 14 files changed, 268 insertions(+), 7 deletions(-) create mode 100644 src/test/modules/test_tableam_logical/Makefile create mode 100644 src/test/modules/test_tableam_logical/expected/repack.out create mode 100644 src/test/modules/test_tableam_logical/meson.build create mode 100644 src/test/modules/test_tableam_logical/sql/repack.sql create mode 100644 src/test/modules/test_tableam_logical/test_tableam_logical--1.0.sql create mode 100644 src/test/modules/test_tableam_logical/test_tableam_logical.c create mode 100644 src/test/modules/test_tableam_logical/test_tableam_logical.control diff --git a/doc/src/sgml/ref/repack.sgml b/doc/src/sgml/ref/repack.sgml index 4cd3cfa0b11..ab26cb5917e 100644 --- a/doc/src/sgml/ref/repack.sgml +++ b/doc/src/sgml/ref/repack.sgml @@ -417,7 +417,9 @@ REPACK [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] USING <listitem> <para> - The table's access method is not <literal>heap</literal>. + The table's access method does not support logical decoding of its + changes (see <xref linkend="tableam"/>). The + <literal>heap</literal> access method does. </para> </listitem> diff --git a/doc/src/sgml/tableam.sgml b/doc/src/sgml/tableam.sgml index 9ccf5b739ed..ac5c0007c4a 100644 --- a/doc/src/sgml/tableam.sgml +++ b/doc/src/sgml/tableam.sgml @@ -138,6 +138,18 @@ my_tableam_handler(PG_FUNCTION_ARGS) implemented. </para> + <para> + Changes to a table can be <link linkend="logicaldecoding">logically + decoded</link> only if the access method's resource manager provides an + <function>rm_decode</function> callback that turns its WAL records into row + changes. An access method that does so can say so by providing the + <structfield>relation_supports_logical_decoding</structfield> callback. + Commands that need to see every change made to a table while they work, + such as <link linkend="sql-repack"><command>REPACK + (CONCURRENTLY)</command></link>, refuse to run on a table of an access + method that doesn't provide it. + </para> + <para> To implement transactional support in a manner that allows different table access methods be accessed within a single transaction, it likely is diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 05a81e4557a..de86d6c8285 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -2653,6 +2653,14 @@ BitmapHeapScanNextBlock(TableScanDesc scan, * ------------------------------------------------------------------------ */ +/* Heap's resource managers support logical decoding. */ +static bool +heapam_relation_supports_logical_decoding(Relation rel) +{ + return true; +} + + static const TableAmRoutine heapam_methods = { .type = T_TableAmRoutine, @@ -2708,7 +2716,9 @@ static const TableAmRoutine heapam_methods = { .scan_bitmap_next_tuple = heapam_scan_bitmap_next_tuple, .scan_sample_next_block = heapam_scan_sample_next_block, - .scan_sample_next_tuple = heapam_scan_sample_next_tuple + .scan_sample_next_tuple = heapam_scan_sample_next_tuple, + + .relation_supports_logical_decoding = heapam_relation_supports_logical_decoding }; diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c index 8cf339a9cc7..053a50548a8 100644 --- a/src/backend/commands/repack.c +++ b/src/backend/commands/repack.c @@ -987,16 +987,13 @@ check_concurrent_repack_requirements(Relation rel, Oid *ident_idx_p) /* * A table AM that doesn't support logical decoding would cause REPACK * (CONCURRENTLY) to silently lose the changes made during the rewrite. - * Nothing in TableAmRoutine tells us whether it does, so for now restrict - * to heap. Check the routine rather than the AM OID, so that an AM - * reusing the heap handler still works. */ - if (rel->rd_tableam != GetHeapamTableAmRoutine()) + if (!table_relation_supports_logical_decoding(rel)) ereport(ERROR, errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("cannot execute %s on relation \"%s\"", "REPACK (CONCURRENTLY)", RelationGetRelationName(rel)), - errdetail("This operation is only supported for the \"heap\" access method.")); + errdetail("The table's access method does not support logical decoding.")); /* Data changes in system relations are not logically decoded. */ if (IsCatalogRelation(rel)) diff --git a/src/include/access/tableam.h b/src/include/access/tableam.h index 9d2fc7f9acb..5d2d397cd91 100644 --- a/src/include/access/tableam.h +++ b/src/include/access/tableam.h @@ -878,6 +878,22 @@ typedef struct TableAmRoutine SampleScanState *scanstate, TupleTableSlot *slot); + /* ------------------------------------------------------------------------ + * Logical decoding related functions. + * ------------------------------------------------------------------------ + */ + + /* + * See table_relation_supports_logical_decoding(). + * + * Optional; if NULL, the AM is taken not to support logical decoding. + * Return true only if every change to the relation's rows can be decoded: + * its resource manager needs an rm_decode callback. REPACK + * (CONCURRENTLY) checks this, as it would otherwise lose the changes made + * while it runs. + */ + bool (*relation_supports_logical_decoding) (Relation rel); + } TableAmRoutine; @@ -2040,6 +2056,17 @@ table_relation_estimate_size(Relation rel, int32 *attr_widths, allvisfrac); } +/* + * table_relation_supports_logical_decoding - can the relation's changes be + * logically decoded? An AM without the callback can't. + */ +static inline bool +table_relation_supports_logical_decoding(Relation rel) +{ + return rel->rd_tableam->relation_supports_logical_decoding != NULL && + rel->rd_tableam->relation_supports_logical_decoding(rel); +} + /* ---------------------------------------------------------------------------- * Executor related functionality diff --git a/src/test/modules/Makefile b/src/test/modules/Makefile index 6c1504ac932..ebf9d86ac3c 100644 --- a/src/test/modules/Makefile +++ b/src/test/modules/Makefile @@ -53,6 +53,7 @@ SUBDIRS = \ test_shmem \ test_shm_mq \ test_slru \ + test_tableam_logical \ test_tidstore \ test_wait_lsn \ unsafe_tests \ diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build index ef3cdb8faf9..9dc5cc18c88 100644 --- a/src/test/modules/meson.build +++ b/src/test/modules/meson.build @@ -55,6 +55,7 @@ subdir('test_saslprep') subdir('test_shmem') subdir('test_shm_mq') subdir('test_slru') +subdir('test_tableam_logical') subdir('test_tidstore') subdir('test_wait_lsn') subdir('typcache') diff --git a/src/test/modules/test_tableam_logical/Makefile b/src/test/modules/test_tableam_logical/Makefile new file mode 100644 index 00000000000..081c5a7de3f --- /dev/null +++ b/src/test/modules/test_tableam_logical/Makefile @@ -0,0 +1,20 @@ +# src/test/modules/test_tableam_logical/Makefile + +MODULES = test_tableam_logical + +EXTENSION = test_tableam_logical +DATA = test_tableam_logical--1.0.sql +PGFILEDESC = "test_tableam_logical - table access methods with and without logical decoding support" + +REGRESS = repack + +ifdef USE_PGXS +PG_CONFIG = pg_config +PGXS := $(shell $(PG_CONFIG) --pgxs) +include $(PGXS) +else +subdir = src/test/modules/test_tableam_logical +top_builddir = ../../../.. +include $(top_builddir)/src/Makefile.global +include $(top_srcdir)/contrib/contrib-global.mk +endif diff --git a/src/test/modules/test_tableam_logical/expected/repack.out b/src/test/modules/test_tableam_logical/expected/repack.out new file mode 100644 index 00000000000..878059e6ee3 --- /dev/null +++ b/src/test/modules/test_tableam_logical/expected/repack.out @@ -0,0 +1,45 @@ +-- +-- REPACK (CONCURRENTLY) needs the table AM's relation_supports_logical_decoding +-- callback. The test AMs copy heap's routine, which heap's index builds and +-- TOAST code reject, so their tables have no index and no TOAST table. +-- +CREATE EXTENSION test_tableam_logical; +CREATE TABLE repack_heap (i int PRIMARY KEY, j int); +CREATE TABLE repack_decode (i int, j int) USING test_decode; +CREATE TABLE repack_nodecode (i int, j int) USING test_nodecode; +INSERT INTO repack_heap SELECT g, g * 2 FROM generate_series(1, 100) g; +INSERT INTO repack_decode SELECT g, g * 2 FROM generate_series(1, 100) g; +INSERT INTO repack_nodecode SELECT g, g * 2 FROM generate_series(1, 100) g; +-- heap has it +REPACK (CONCURRENTLY) repack_heap; +-- an AM without it is refused +REPACK (CONCURRENTLY) repack_nodecode; +ERROR: cannot execute REPACK (CONCURRENTLY) on relation "repack_nodecode" +DETAIL: The table's access method does not support logical decoding. +-- one with it passes that check, then fails for want of an identity index +REPACK (CONCURRENTLY) repack_decode; +ERROR: cannot execute REPACK (CONCURRENTLY) on relation "repack_decode" +DETAIL: Relation "repack_decode" has no identity index. +-- the rest of REPACK doesn't need it +REPACK repack_nodecode; +VACUUM FULL repack_nodecode; +SELECT count(*) FROM repack_heap; + count +------- + 100 +(1 row) + +SELECT count(*) FROM repack_decode; + count +------- + 100 +(1 row) + +SELECT count(*) FROM repack_nodecode; + count +------- + 100 +(1 row) + +DROP TABLE repack_heap, repack_decode, repack_nodecode; +DROP EXTENSION test_tableam_logical; diff --git a/src/test/modules/test_tableam_logical/meson.build b/src/test/modules/test_tableam_logical/meson.build new file mode 100644 index 00000000000..4409ed65513 --- /dev/null +++ b/src/test/modules/test_tableam_logical/meson.build @@ -0,0 +1,33 @@ +# Copyright (c) 2026, PostgreSQL Global Development Group + +test_tableam_logical_sources = files( + 'test_tableam_logical.c', +) + +if host_system == 'windows' + test_tableam_logical_sources += rc_lib_gen.process(win32ver_rc, extra_args: [ + '--NAME', 'test_tableam_logical', + '--FILEDESC', 'test_tableam_logical - table access methods with and without logical decoding support',]) +endif + +test_tableam_logical = shared_module('test_tableam_logical', + test_tableam_logical_sources, + kwargs: pg_test_mod_args, +) +test_install_libs += test_tableam_logical + +test_install_data += files( + 'test_tableam_logical.control', + 'test_tableam_logical--1.0.sql', +) + +tests += { + 'name': 'test_tableam_logical', + 'sd': meson.current_source_dir(), + 'bd': meson.current_build_dir(), + 'regress': { + 'sql': [ + 'repack', + ], + }, +} diff --git a/src/test/modules/test_tableam_logical/sql/repack.sql b/src/test/modules/test_tableam_logical/sql/repack.sql new file mode 100644 index 00000000000..7f4b3b565db --- /dev/null +++ b/src/test/modules/test_tableam_logical/sql/repack.sql @@ -0,0 +1,33 @@ +-- +-- REPACK (CONCURRENTLY) needs the table AM's relation_supports_logical_decoding +-- callback. The test AMs copy heap's routine, which heap's index builds and +-- TOAST code reject, so their tables have no index and no TOAST table. +-- +CREATE EXTENSION test_tableam_logical; + +CREATE TABLE repack_heap (i int PRIMARY KEY, j int); +CREATE TABLE repack_decode (i int, j int) USING test_decode; +CREATE TABLE repack_nodecode (i int, j int) USING test_nodecode; +INSERT INTO repack_heap SELECT g, g * 2 FROM generate_series(1, 100) g; +INSERT INTO repack_decode SELECT g, g * 2 FROM generate_series(1, 100) g; +INSERT INTO repack_nodecode SELECT g, g * 2 FROM generate_series(1, 100) g; + +-- heap has it +REPACK (CONCURRENTLY) repack_heap; + +-- an AM without it is refused +REPACK (CONCURRENTLY) repack_nodecode; + +-- one with it passes that check, then fails for want of an identity index +REPACK (CONCURRENTLY) repack_decode; + +-- the rest of REPACK doesn't need it +REPACK repack_nodecode; +VACUUM FULL repack_nodecode; + +SELECT count(*) FROM repack_heap; +SELECT count(*) FROM repack_decode; +SELECT count(*) FROM repack_nodecode; + +DROP TABLE repack_heap, repack_decode, repack_nodecode; +DROP EXTENSION test_tableam_logical; diff --git a/src/test/modules/test_tableam_logical/test_tableam_logical--1.0.sql b/src/test/modules/test_tableam_logical/test_tableam_logical--1.0.sql new file mode 100644 index 00000000000..69344dfbb77 --- /dev/null +++ b/src/test/modules/test_tableam_logical/test_tableam_logical--1.0.sql @@ -0,0 +1,19 @@ +/* src/test/modules/test_tableam_logical/test_tableam_logical--1.0.sql */ + +-- complain if script is sourced in psql, rather than via CREATE EXTENSION +\echo Use "CREATE EXTENSION test_tableam_logical" to load this file. \quit + +CREATE FUNCTION test_tableam_decode_handler(internal) +RETURNS table_am_handler +AS 'MODULE_PATHNAME' +LANGUAGE C; + +CREATE FUNCTION test_tableam_nodecode_handler(internal) +RETURNS table_am_handler +AS 'MODULE_PATHNAME' +LANGUAGE C; + +CREATE ACCESS METHOD test_decode TYPE TABLE + HANDLER test_tableam_decode_handler; +CREATE ACCESS METHOD test_nodecode TYPE TABLE + HANDLER test_tableam_nodecode_handler; diff --git a/src/test/modules/test_tableam_logical/test_tableam_logical.c b/src/test/modules/test_tableam_logical/test_tableam_logical.c new file mode 100644 index 00000000000..34f941360ac --- /dev/null +++ b/src/test/modules/test_tableam_logical/test_tableam_logical.c @@ -0,0 +1,56 @@ +/*-------------------------------------------------------------------------- + * + * test_tableam_logical.c + * Table AMs that reuse heap's code, with and without the + * relation_supports_logical_decoding callback. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/modules/test_tableam_logical/test_tableam_logical.c + * + * ------------------------------------------------------------------------- + */ +#include "postgres.h" + +#include "access/tableam.h" +#include "fmgr.h" + +PG_MODULE_MAGIC; + +PG_FUNCTION_INFO_V1(test_tableam_decode_handler); +PG_FUNCTION_INFO_V1(test_tableam_nodecode_handler); + +static TableAmRoutine decode_methods; +static TableAmRoutine nodecode_methods; +static bool methods_initialized = false; + +static void init_methods(void); + +/* Copies of heap's routine, the second without the callback. */ +static void +init_methods(void) +{ + memcpy(&decode_methods, GetHeapamTableAmRoutine(), sizeof(TableAmRoutine)); + + memcpy(&nodecode_methods, GetHeapamTableAmRoutine(), sizeof(TableAmRoutine)); + nodecode_methods.relation_supports_logical_decoding = NULL; + + methods_initialized = true; +} + +Datum +test_tableam_decode_handler(PG_FUNCTION_ARGS) +{ + if (!methods_initialized) + init_methods(); + PG_RETURN_POINTER(&decode_methods); +} + +Datum +test_tableam_nodecode_handler(PG_FUNCTION_ARGS) +{ + if (!methods_initialized) + init_methods(); + PG_RETURN_POINTER(&nodecode_methods); +} diff --git a/src/test/modules/test_tableam_logical/test_tableam_logical.control b/src/test/modules/test_tableam_logical/test_tableam_logical.control new file mode 100644 index 00000000000..f38ab722948 --- /dev/null +++ b/src/test/modules/test_tableam_logical/test_tableam_logical.control @@ -0,0 +1,5 @@ +# test_tableam_logical extension +comment = 'test_tableam_logical - table access methods with and without logical decoding support' +default_version = '1.0' +module_pathname = '$libdir/test_tableam_logical' +relocatable = true -- 2.43.0
