On Wed, Feb 10, 2021 at 3:43 PM Ashutosh Bapat <ashutosh.ba...@enterprisedb.com> wrote:
> I think we need to treat a prepared transaction slightly different from an > uncommitted transaction when sending downstream. We need to send a whole > uncommitted transaction downstream again because previously applied changes > must have been aborted and hence lost by the downstream and thus it needs to > get all of those again. But when a downstream prepares a transaction, even if > it's not committed, those changes are not lost even after restart of a > walsender. If the downstream confirms that it has "flushed" PREPARE, there is > no need to send all the changes again. But the other side of the problem is that ,without this, if the prepared transaction is prior to a consistent snapshot when decoding starts/restarts, then only the "commit prepared" is sent to downstream (as seen in the test scenario I shared above), and downstream has to error away the commit prepared because it does not have the corresponding prepared transaction. We did not find an easy way to distinguish between these two scenarios for prepared transactions. a. A consistent snapshot being formed in between a prepare and a commit prepared for the first time. b. Decoder restarting between a prepare and a commit prepared. For plugins to be able to handle this, we have added a special callback "Begin Prepare" as explained in [1] section 49.6.4.10 "The required begin_prepare_cb callback is called whenever the start of a prepared transaction has been decoded. The gid field, which is part of the txn parameter can be used in this callback to check if the plugin has already received this prepare in which case it can skip the remaining changes of the transaction. This can only happen if the user restarts the decoding after receiving the prepare for a transaction but before receiving the commit prepared say because of some error." The pgoutput plugin is also being updated to be able to handle this situation of duplicate prepared transactions. regards, Ajin Cherian Fujitsu Australia
