On Thu, Oct 22, 2026 at 5:23 PM Nisha Moond <[email protected]> wrote: > > On Fri, Oct 16, 2026 at 8:03 PM Nisha Moond <[email protected]> wrote: > > > > Attached is v82, which addresses the comments on v81: > > > > We have not yet documented that, in rare cases, the subscriber can > still fail with the 1GB error when recording a conflict. > > The 64kB cap on replica identity values is checked after the key > value's output function has produced its text, so it limits what we > store, not what we render. Normally that's fine: the publisher already > rendered the same value to send it, which keeps the text well within > the limit. That doesn't hold in a few rare cases: > - the key column has a different data type on the subscriber (e.g. > text published into a numeric[] column); > - binary = true, where the publisher never produced the text; > - the apply worker's bytea_output or quote_all_identifiers differs > from the publisher's. > > In these cases a small stored value can render to more than 1GB, and > the apply worker errors out. > > The attached top-up patch 0004 documents this limitation and adds an > XXX comment in build_index_key_json() describing these cases. > Patches 0001-0003 are unchanged.
I agree with these scenarios and the need for documentation. But I have polished the doc change in patch 0004 a bit and attached the updated patch as text. Please take it if you agree. If we prefer not to elaborate on the details, we can drop the scenarios while retaining the rest of the wording. thanks Shveta
From 84fa79529b0a84e2fc7618aa9afef3e9aa5cf455 Mon Sep 17 00:00:00 2001 From: Nisha Moond <[email protected]> Date: Wed, 21 Oct 2026 13:56:51 +0530 Subject: [PATCH] Document and comment that replica identity rendering can still exceed 1GB The 64kB cap on replica identity values is checked after a key value's text is produced, so in rare cases the text itself can exceed 1GB and conflict logging fails with an error. Mention this in the docs, and add an XXX comment in build_index_key_json() listing when it can happen. --- doc/src/sgml/logical-replication.sgml | 16 +++++++++++++++- src/backend/replication/logical/conflict.c | 14 ++++++++++++-- 2 files changed, 27 insertions(+), 3 deletions(-) diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml index 8bc0e9e2e9c..fd4730ed420 100644 --- a/doc/src/sgml/logical-replication.sgml +++ b/doc/src/sgml/logical-replication.sgml @@ -2483,7 +2483,21 @@ SELECT * FROM pg_conflict.pg_conflict_log_16392 WHERE has_omitted_values; A value omitted this way cannot be recovered from the conflict log table. Some limit is needed because <literal>replica_identity</literal> is a <type>json</type> value, which cannot exceed 1GB, and the text of a value - can be much longer than the value itself. + can be much longer than the value itself. However, the 64kB limit is + checked only after a key value's text representation has been produced. + Thus, it limits the amount of text retained in the conflict log, but does + not prevent the output function from producing a text representation + exceeding 1GB. In almost all cases this text is well within the limit, + because the publisher produced the same value as text, within the same + bound, in order to send it. In rare cases that does not hold, for + example when the subscriber uses a different data type for a key column, + the subscription uses binary format + (<link linkend="sql-createsubscription-params-with-binary"><literal>binary = true</literal></link>, + so the value is not sent as text), or the subscriber's + <xref linkend="guc-bytea-output"/> or + <xref linkend="guc-quote-all-identifiers"/> differs from the publisher's. + In these (and other such possible) cases, the text produced on the + subscriber can exceed 1GB and conflict logging fails with an error. </para> </sect3> </sect2> diff --git a/src/backend/replication/logical/conflict.c b/src/backend/replication/logical/conflict.c index a2037ba716a..20cee95dd50 100644 --- a/src/backend/replication/logical/conflict.c +++ b/src/backend/replication/logical/conflict.c @@ -1115,8 +1115,9 @@ build_index_key_json(EState *estate, Relation localrel, * a json representation -- so its result is unrelated to the size of the * value we received, and a tiny key could render a json value above the * 1GB limit and error out the apply worker. Using the output function - * keeps us to the same representation the publisher already produced, as - * the server log does in BuildIndexValueDescription(). + * keeps us to the same representation the publisher already produced + * (when the column types and output settings match, see below), as the + * server log does in BuildIndexValueDescription(). * * Only the key attributes are recorded. FormIndexDatum() above also * fills in any non-key (INCLUDE) columns, but those are not part of the @@ -1155,6 +1156,15 @@ build_index_key_json(EState *estate, Relation localrel, * instead index's tupleDesc. */ getTypeOutputInfo(indexDesc->rd_opcintype[i], &outfuncoid, &typisvarlena); + + /* + * XXX The cap below is checked only after the value is rendered. + * The publisher rendered the same value to send it, which bounds + * its length, unless the column types differ, binary = true, or + * the apply worker's bytea_output or quote_all_identifiers + * differs. Then the render can exceed 1GB and error out the apply + * worker. + */ outputstr = OidOutputFunctionCall(outfuncoid, values[i]); outputlen = strlen(outputstr); -- 2.34.1
