On Thu, 13 Aug 2026 at 17:19, Greg Sabino Mullane <[email protected]> wrote: > > On Thu, Aug 13, 2026 at 9:50 AM Matthias van de Meent > <[email protected]> wrote: >> >> Or, an option D: Forbid the creation (and use) of filtered publication table >> definitions for tables which contain a non-identity varlena-typed column >> (i.e. the type's typlen is -1). > > > I think it's too late for that: option A seems better until we get a proper > fix.
What do you mean by "too late for that"? If you mean "there are already systems with publications with filters on tables with published varlena non-identity columns" then you're right that those systems exist, but that shouldn't preclude us from starting to raise errors when the user wants to create a new (or start using an existing) publication that we can assume to be broken; We've disabled and removed inherently broken features before, why shouldn't we do that here? I don't think there is a more proper fix than this option D. Publications are downstream of the LR decoder (plugins can use publications, but , and including publication information in the decision-making process upstream of that (such as, in tableam's update handler) we would have to break through several layers of abstractions. And, after all of that, it'd still leave decoders which don't use the publication tables with broken data, because those don't necessarily track the data they export through pg_publication et al. >> we should not allow users to create publications of which we know ahead of >> time that the >> data stream is likely to break on our side. > > > Maybe instead* we issue a warning on creation (again, until we get a real > fix, which I think is doable and probably needed for more than just this use > case) I don't see how we can get a "real" fix. Detoasting and WAL-logging all external columns "because a logical plugin may need to see this column if it has filtering" would explode the amount of WAL used in updates; it'd be indistinguishable from REPLICATION IDENTITY FULL. > * To be clear, I'm recommending the warning (call it option E) in addition to > option A The warning would have to say something along the lines of "Hey, your table has varlena columns and a filter, be aware your column data may be lost. Oh, don't worry, we'll detect it when we lost your data and stop the replication stream.", and if the message says something along those lines then we'd better just plainly disallow such publications by throwing errors, so that we don't leave a known and documented way to lose data or halt publications when the user is using PG normally in all possible ways. "Sorry, TOAST happened" is not a good argument to structurally halt replication slots. Kind regards, Matthias van de Meent Databricks (https://www.databricks.com)
