On Thu, Sep 8, 2022 at 8:32 PM Ashutosh Sharma <ashu.coe...@gmail.com> wrote: > > On Thu, Sep 8, 2022 at 6:23 PM Ashutosh Bapat > <ashutosh.bapat....@gmail.com> wrote: > > > > On Thu, Sep 8, 2022 at 4:14 PM Ashutosh Sharma <ashu.coe...@gmail.com> > > wrote: > > > > > > Hi All, > > > > > > The logically decoded data are sent to the logical subscriber at the time > > > of transaction commit, assuming that the data is small. However, before > > > the transaction commit is performed, the LSN representing the data that > > > is yet to be received by the logical subscriber appears in the > > > confirmed_flush_lsn column of pg_replication_slots catalog. Isn't the > > > information seen in the confirmed_flush_lsn column while the transaction > > > is in progress incorrect ? esp considering the description given in the > > > pg doc for this column. > > > > > > Actually, while the transaction is running, the publisher keeps on > > > sending keepalive messages containing LSN of the last decoded data saved > > > in reorder buffer and the subscriber responds with the same LSN as the > > > last received LSN which is then updated as confirmed_flush_lsn by the > > > publisher. I think the LSN that we are sending with the keepalive message > > > should be the one representing the transaction begin message, not the LSN > > > of the last decoded data which is yet to be sent. Please let me know if I > > > am missing something here. > > > > The transactions with commit lsn < confirmed_flush_lsn are confirmed > > to be received (and applied by the subscriber. Setting LSN > > corresponding to a WAL record within a transaction in progress as > > confirmed_flush should be ok. Since the transactions are interleaved > > in WAL stream, it's quite possible that LSNs of some WAL records of an > > inflight transaction are lesser than commit LSN of some another > > transaction. So setting commit LSN of another effectively same as > > setting it to any of the LSNs of any previous WAL record irrespective > > of the transaction that it belongs to. > > Thank you Ashutosh for the explanation. I still feel that the > documentation on confirmed_flush_lsn needs some improvement. It > actually claims that all the data before the confirmed_flush_lsn has > been received by the logical subscriber, but that's not the case. It > actually means that all the data belonging to the transactions with > commit lsn < confirmed_flush_lsn has been received and applied by the > subscriber. So setting confirmed_flush_lsn to the lsn of wal records > generated by running transaction might make people think that the wal > records belonging to previous data of the same running transaction has > already been received and applied by the subscriber node, but that's > not true. >
Can you please point to the documentation. It's true that it needs to be clarified. But what you are saying may not be entirely true in case of streamed transaction. In that case we might send logically decoded changes of an ongoing transaction as well. They may even get applied but not necessarily committed. It's a bit complicated. :) -- Best Wishes, Ashutosh Bapat
