Re: Track replica origin progress for Rollback Prepared

2021-03-07 Thread Michael Paquier
On Thu, Mar 04, 2021 at 08:03:25PM +0530, Amit Kapila wrote: > I think here we have a slight timing thing which is if the checkpoint > happens before restart then the problem won't occur, not sure but > maybe increase the checkpoint_timeout as well to make it reproducible. Thanks for providing

Re: Track replica origin progress for Rollback Prepared

2021-03-07 Thread Amit Kapila
On Thu, Mar 4, 2021 at 1:40 PM Michael Paquier wrote: > > On Wed, Mar 03, 2021 at 09:06:11AM +0530, Amit Kapila wrote: > > Michael, I want to push this patch soon unless you have any concerns. > > No concerns from me... > Pushed! I have tried but couldn't come up with a good way other than

Re: Track replica origin progress for Rollback Prepared

2021-03-04 Thread Amit Kapila
On Fri, Mar 5, 2021 at 7:11 AM Amit Kapila wrote: > > On Thu, Mar 4, 2021 at 8:03 PM Amit Kapila wrote: > > > > I have just checked via code coverage that we don't seem to have tests > for recovery of replication origin advance for commit [1], see > function xact_redo_commit. Similarly, a

Re: Track replica origin progress for Rollback Prepared

2021-03-04 Thread Amit Kapila
On Thu, Mar 4, 2021 at 8:03 PM Amit Kapila wrote: > > Having said that, I think we use > replication origins to test it. For example: > > Create Table t1(c1 int); > > SELECT pg_replication_origin_create('regress'); > SELECT pg_replication_origin_session_setup('regress'); > SELECT

Re: Track replica origin progress for Rollback Prepared

2021-03-04 Thread Amit Kapila
On Thu, Mar 4, 2021 at 1:40 PM Michael Paquier wrote: > > On Wed, Mar 03, 2021 at 09:06:11AM +0530, Amit Kapila wrote: > That's perhaps not enough to be an actual objection, but I am not > really comfortable with the concept of pushing into the tree code that > would remain untested until all the

Re: Track replica origin progress for Rollback Prepared

2021-03-04 Thread Michael Paquier
On Wed, Mar 03, 2021 at 09:06:11AM +0530, Amit Kapila wrote: > Michael, I want to push this patch soon unless you have any concerns. No concerns from me... > This is required for correctness if any plugin uses 2PC and origins to > track the progress. Now, that we have exposed the two_phase

Re: Track replica origin progress for Rollback Prepared

2021-03-02 Thread Amit Kapila
On Wed, Jan 13, 2021 at 2:03 PM Ajin Cherian wrote: > > On Wed, Jan 13, 2021 at 6:28 PM Ajin Cherian wrote: > > > I added the below log: > Small correction, I sent the wrong code change for logging, this was > the correct one: > Okay, thanks. I have rebased the patch and updated comments as

Re: Track replica origin progress for Rollback Prepared

2021-01-13 Thread Ajin Cherian
On Wed, Jan 13, 2021 at 6:28 PM Ajin Cherian wrote: > I added the below log: Small correction, I sent the wrong code change for logging, this was the correct one: @@ -5976,6 +5977,14 @@ xact_redo_abort(xl_xact_parsed_abort *parsed, TransactionId xid)

Re: Track replica origin progress for Rollback Prepared

2021-01-12 Thread Ajin Cherian
On Wed, Jan 13, 2021 at 12:11 AM Amit Kapila wrote: > Thanks for doing these tests. I think you can put an elog in the below > code change as well to show that the recovery code path is also hit: > > +xact_redo_abort(xl_xact_parsed_abort *parsed, TransactionId xid, > + XLogRecPtr lsn,

Re: Track replica origin progress for Rollback Prepared

2021-01-12 Thread Amit Kapila
On Tue, Jan 12, 2021 at 3:18 PM Ajin Cherian wrote: > > On Wed, Jan 6, 2021 at 11:56 PM Amit Kapila wrote: > > > > Now, let us see how the tests mentioned by me cover this code. In the > > first test (check that 2PC gets replicated to subscriber then ROLLBACK > > PREPARED), we do below on

Re: Track replica origin progress for Rollback Prepared

2021-01-12 Thread Ajin Cherian
On Wed, Jan 6, 2021 at 11:56 PM Amit Kapila wrote: > Now, let us see how the tests mentioned by me cover this code. In the > first test (check that 2PC gets replicated to subscriber then ROLLBACK > PREPARED), we do below on publisher and wait for it to be applied on > the subscriber. > BEGIN; >

Re: Track replica origin progress for Rollback Prepared

2021-01-06 Thread Amit Kapila
On Wed, Jan 6, 2021 at 5:18 PM Michael Paquier wrote: > > On Tue, Jan 05, 2021 at 04:24:21PM +0530, Amit Kapila wrote: > > There are already tests [1] in one of the upcoming patches for logical > > decoding of 2PC which covers this code using which I have found this > > problem. So, I thought

Re: Track replica origin progress for Rollback Prepared

2021-01-06 Thread Michael Paquier
On Tue, Jan 05, 2021 at 04:24:21PM +0530, Amit Kapila wrote: > There are already tests [1] in one of the upcoming patches for logical > decoding of 2PC which covers this code using which I have found this > problem. So, I thought those would be sufficient. I have not checked > the feasibility of

Re: Track replica origin progress for Rollback Prepared

2021-01-05 Thread Amit Kapila
On Tue, Jan 5, 2021 at 12:32 PM Michael Paquier wrote: > > On Tue, Jan 05, 2021 at 09:35:21AM +0530, Amit Kapila wrote: > > As noted in [1], without this the subscriber might again ask for > > rollback prepared lsn after restart. > > > > Attached patch addresses this problem. > > Is it possible

Re: Track replica origin progress for Rollback Prepared

2021-01-04 Thread Michael Paquier
On Tue, Jan 05, 2021 at 09:35:21AM +0530, Amit Kapila wrote: > As noted in [1], without this the subscriber might again ask for > rollback prepared lsn after restart. > > Attached patch addresses this problem. Is it possible to add some tests in test_decoding? /* dump transaction origin

Track replica origin progress for Rollback Prepared

2021-01-04 Thread Amit Kapila
While reviewing logical decoding of 2PC xacts work, I noticed that we need $SUBJECT [1]. Commit 1eb6d6527a [2] allowed to track replica origin replay progress for 2PC but it was not complete. It misses to properly track the progress for rollback prepared especially it missed to update the code for