Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Thu, Oct 27, 2016 at 7:38 PM, Michael Paquier wrote: >> I committed and back-patched this with some additional work on the >> comments, but I don't understand this remark. That comment seems like >> it should refer to the checkpointer in modern branches, but isn't

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Fri, Oct 28, 2016 at 5:08 AM, Michael Paquier wrote: > On Fri, Oct 28, 2016 at 1:16 AM, Robert Haas wrote: >> On Thu, Oct 27, 2016 at 9:05 AM, Michael Paquier >> wrote: >>> On Thu, Oct 27, 2016 at 7:16 PM, Amit

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Fri, Oct 28, 2016 at 1:16 AM, Robert Haas wrote: > On Thu, Oct 27, 2016 at 9:05 AM, Michael Paquier > wrote: >> On Thu, Oct 27, 2016 at 7:16 PM, Amit Kapila wrote: >>> This can create problem if the checkpoint record

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Thu, Oct 27, 2016 at 9:05 AM, Michael Paquier wrote: > On Thu, Oct 27, 2016 at 7:16 PM, Amit Kapila wrote: >> This can create problem if the checkpoint record spans across multiple >> segments, because you are updating minRecoveryPoint to

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Thu, Oct 27, 2016 at 7:16 PM, Amit Kapila wrote: > This can create problem if the checkpoint record spans across multiple > segments, because you are updating minRecoveryPoint to start of > checkpoint record. We need to update it to end+1 of checkpoint > record.

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Thu, Oct 27, 2016 at 9:51 AM, Michael Paquier wrote: > On Thu, Oct 27, 2016 at 2:59 AM, Robert Haas wrote: >> On Wed, Oct 26, 2016 at 2:06 AM, Michael Paquier >> wrote: >>> But yes, thinking *harder*, I agree that

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Wed, Oct 26, 2016 at 6:13 PM, Amit Kapila wrote: > If my understanding is right, then the changes proposed by you as > below is not what is intended here. I think you need to do something > as I have mentioned above. Ah OK I have spotted the subtility; - 0 means

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Thu, Oct 27, 2016 at 2:59 AM, Robert Haas wrote: > On Wed, Oct 26, 2016 at 2:06 AM, Michael Paquier > wrote: >> But yes, thinking *harder*, I agree that updating minRecoveryPoint >> just after the checkpoint record would be fine and removes

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Wed, Oct 26, 2016 at 2:06 AM, Michael Paquier wrote: > But yes, thinking *harder*, I agree that updating minRecoveryPoint > just after the checkpoint record would be fine and removes the need to > have more WAL than necessary in for a backup taken from a standby. >

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Wed, Oct 26, 2016 at 12:39 PM, Michael Paquier wrote: > On Wed, Oct 26, 2016 at 1:40 PM, Amit Kapila wrote: >> If you are inclined towards this solution, then I think what we need >> to do is to change the API UpdateMinRecoveryPoint() such

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Wed, Oct 26, 2016 at 1:40 PM, Amit Kapila wrote: > If you are inclined towards this solution, then I think what we need > to do is to change the API UpdateMinRecoveryPoint() such that it's > second parameter can take three values. 0 means update > minRecoveryPoint to

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Wed, Oct 26, 2016 at 1:53 PM, Amit Kapila wrote: > On Mon, Oct 24, 2016 at 12:25 PM, Michael Paquier > wrote: >> On Mon, Oct 24, 2016 at 1:39 PM, Amit Kapila wrote: >>> >>> I think what you are saying is not

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Wed, Oct 26, 2016 at 1:53 PM, Amit Kapila wrote: > On Mon, Oct 24, 2016 at 12:25 PM, Michael Paquier > wrote: >> On Mon, Oct 24, 2016 at 1:39 PM, Amit Kapila wrote: >>> >>> I think what you are saying is not

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Mon, Oct 24, 2016 at 12:25 PM, Michael Paquier wrote: > On Mon, Oct 24, 2016 at 1:39 PM, Amit Kapila wrote: >> >> I think what you are saying is not completely right, because we do >> update minRecoveryPoint when we don't perform a new

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Tue, Oct 25, 2016 at 10:43 PM, Robert Haas wrote: > On Mon, Oct 24, 2016 at 12:26 AM, Amit Kapila wrote: >> >> You are right that it will include additional WAL than strictly >> necessary, but that can happen today as well because

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Mon, Oct 24, 2016 at 2:55 AM, Michael Paquier wrote: > Yep, minRecoveryPoint still gets updated when the last checkpoint > record is the last restart point to avoid a hot standby to allow > read-only connections at a LSN-point earlier than the last shutdown. >

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Mon, Oct 24, 2016 at 12:26 AM, Amit Kapila wrote: >> The consensus solution on this thread seems to be that we should have >> pg_do_stop_backup() return the last-replayed XLOG location as the >> backup end point. If the control file has been updated with a newer >>

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Sun, Oct 23, 2016 at 11:48 PM, Kyotaro HORIGUCHI wrote: >> I can think of two solutions that would be "tighter": >> >> 1. When performing a restartpoint, update the minimum recovery point >> to just beyond the checkpoint record. I think this can't hurt anyone

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Tue, Oct 25, 2016 at 11:07 AM, Kyotaro HORIGUCHI wrote: > At Mon, 24 Oct 2016 15:55:58 +0900, Michael Paquier > wrote in > >> Anyway, we can clearly reject 1. in

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

Hi, At Mon, 24 Oct 2016 15:55:58 +0900, Michael Paquier wrote in > On Mon, Oct 24, 2016 at 1:39 PM, Amit Kapila wrote: > > On Mon, Oct 24, 2016 at 9:18 AM, Kyotaro HORIGUCHI

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Mon, Oct 24, 2016 at 1:26 PM, Amit Kapila wrote: > On Fri, Oct 21, 2016 at 10:32 PM, Robert Haas wrote: >> 2. In perform_base_backup(), if the endptr returned by >> do_pg_stop_backup() precedes the end of the checkpoint record returned >> by

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Mon, Oct 24, 2016 at 1:39 PM, Amit Kapila wrote: > On Mon, Oct 24, 2016 at 9:18 AM, Kyotaro HORIGUCHI > wrote: >> Thank you for looking and retelling this. +1. >> At Fri, 21 Oct 2016 13:02:21 -0400, Robert Haas

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Mon, Oct 24, 2016 at 9:18 AM, Kyotaro HORIGUCHI wrote: > Thank you for looking and retelling this. > > At Fri, 21 Oct 2016 13:02:21 -0400, Robert Haas wrote > in >> On

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Fri, Oct 21, 2016 at 10:32 PM, Robert Haas wrote: > On Sun, Oct 2, 2016 at 8:52 AM, Michael Paquier > wrote: >>> So, if I understand correctly, then we can mark the version posted by >>> you upthread [1] which includes a test along with

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

Thank you for looking and retelling this. At Fri, 21 Oct 2016 13:02:21 -0400, Robert Haas wrote in > On Sun, Oct 2, 2016 at 8:52 AM, Michael Paquier > wrote: > >> So, if I

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Sun, Oct 2, 2016 at 8:52 AM, Michael Paquier wrote: >> So, if I understand correctly, then we can mark the version posted by >> you upthread [1] which includes a test along with Kyotaro's fix can be >> marked as Ready for committer. If so, then please change the

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

> So, if I understand correctly, then we can mark the version posted by > you upthread [1] which includes a test along with Kyotaro's fix can be > marked as Ready for committer. If so, then please change the status > of patch accordingly. Patch moved to next CF 2016-11, still with status "ready

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

Hello, At Thu, 21 Jul 2016 22:32:08 +0900, Michael Paquier wrote in > On Thu, Jul 21, 2016 at 5:19 PM, Kyotaro HORIGUCHI > wrote: > > + * minRecoveryPoint can go

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Thu, Jul 21, 2016 at 5:19 PM, Kyotaro HORIGUCHI wrote: > + * minRecoveryPoint can go behind the last checkpoint's redo location when > + * the checkpoint writes out no buffer. This does no harm to performing a > + * recovery but such inversion seems

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

Hello, At Mon, 11 Jul 2016 16:47:53 +0900, Michael Paquier wrote in

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

Sorry for the absense. I've reached here. At Thu, 21 Jul 2016 12:20:30 +0900, Michael Paquier wrote in > On Thu, Jul 21, 2016 at 11:56 AM, Amit Kapila wrote: > > On Thu, Jul

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Thu, Jul 21, 2016 at 11:56 AM, Amit Kapila wrote: > On Thu, Jul 21, 2016 at 7:28 AM, Michael Paquier > wrote: >> On Wed, Jul 20, 2016 at 8:56 PM, Michael Paquier >> wrote: Yeah, I think that is totally

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Thu, Jul 21, 2016 at 7:28 AM, Michael Paquier wrote: > On Wed, Jul 20, 2016 at 8:56 PM, Michael Paquier > wrote: >>> >>> Yeah, I think that is totally different angle to fix this issue, so >>> don't you think it is better to start a

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Wed, Jul 20, 2016 at 8:56 PM, Michael Paquier wrote: > On Wed, Jul 20, 2016 at 8:40 PM, Amit Kapila wrote: >> On Wed, Jul 20, 2016 at 5:12 AM, Michael Paquier >> wrote: >>> On Tue, Jul 19, 2016 at 9:08 PM, Amit

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Wed, Jul 20, 2016 at 8:40 PM, Amit Kapila wrote: > On Wed, Jul 20, 2016 at 5:12 AM, Michael Paquier > wrote: >> On Tue, Jul 19, 2016 at 9:08 PM, Amit Kapila wrote: >>> On Tue, Jul 19, 2016 at 10:43 AM, Michael

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Wed, Jul 20, 2016 at 5:12 AM, Michael Paquier wrote: > On Tue, Jul 19, 2016 at 9:08 PM, Amit Kapila wrote: >> On Tue, Jul 19, 2016 at 10:43 AM, Michael Paquier >> wrote: >>> On Sat, Jul 16, 2016 at 9:20 PM, Amit

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Tue, Jul 19, 2016 at 9:08 PM, Amit Kapila wrote: > On Tue, Jul 19, 2016 at 10:43 AM, Michael Paquier > wrote: >> On Sat, Jul 16, 2016 at 9:20 PM, Amit Kapila wrote: >>> On Wed, Jul 13, 2016 at 8:56 AM, Michael

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Tue, Jul 19, 2016 at 10:43 AM, Michael Paquier wrote: > On Sat, Jul 16, 2016 at 9:20 PM, Amit Kapila wrote: >> On Wed, Jul 13, 2016 at 8:56 AM, Michael Paquier >> wrote: >>> Another way that just popped into my

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Sat, Jul 16, 2016 at 9:20 PM, Amit Kapila wrote: > On Wed, Jul 13, 2016 at 8:56 AM, Michael Paquier > wrote: >> If we want to tackle the case I mentioned above, one way is to just >> update minRecoveryPoint when an exclusive or a

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Wed, Jul 13, 2016 at 8:56 AM, Michael Paquier wrote: > On Tue, Jul 12, 2016 at 12:22 PM, Amit Kapila wrote: >> I think updating minRecoveryPoint unconditionally can change it's >> purpose in some cases. Refer below comments in code: >> >> *

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Tue, Jul 12, 2016 at 12:22 PM, Amit Kapila wrote: > I think updating minRecoveryPoint unconditionally can change it's > purpose in some cases. Refer below comments in code: > > * minRecoveryPoint is updated to the latest replayed LSN whenever we > * flush a data

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Thu, Jun 23, 2016 at 12:20 PM, Michael Paquier wrote: > On Wed, Jun 15, 2016 at 4:52 PM, Kyotaro HORIGUCHI > wrote: >> Hello, > > Sorry for the late reply, Horiguchi-san. I have finally been able to > put some mind power into that. >

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Mon, Jul 11, 2016 at 4:40 PM, Kyotaro HORIGUCHI wrote: > That's true, but we don't always have a perfectly comprehensive > test suite, consciously or unconsciously. The sentence was > inattentive but the "bug" was just the negative comparable to > "feature" in

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

Hello, thank you for the comment. At Fri, 8 Jul 2016 14:42:20 -0400, Alvaro Herrera wrote in <20160708184220.GA733807@alvherre.pgsql> > Kyotaro HORIGUCHI wrote: > > > At Fri, 10 Jun 2016 17:39:59 +0900, Michael Paquier > > wrote in > >

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

Kyotaro HORIGUCHI wrote: > At Fri, 10 Jun 2016 17:39:59 +0900, Michael Paquier > wrote in > > > On Thu, Jun 9, 2016 at 9:55 PM, Kyotaro HORIGUCHI > > wrote: > >

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

Sorry for the absense. Thank you for registering it. regards. At Fri, 24 Jun 2016 14:46:25 +0900, Michael Paquier wrote in > On Thu, Jun 23, 2016 at 3:51 PM, Michael Paquier >

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Thu, Jun 23, 2016 at 3:51 PM, Michael Paquier wrote: > By the way, do you mind if I add this patch to the next CF? Better not > to lose track of it... Well, I have added an entry here at the end: https://commitfest.postgresql.org/10/654/ Better doing it now before I

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Thu, Jun 23, 2016 at 3:50 PM, Michael Paquier wrote: > On Wed, Jun 15, 2016 at 4:52 PM, Kyotaro HORIGUCHI > wrote: >> Hello, > > Sorry for the late reply, Horiguchi-san. I have finally been able to > put some mind power into that. >

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Wed, Jun 15, 2016 at 4:52 PM, Kyotaro HORIGUCHI wrote: > Hello, Sorry for the late reply, Horiguchi-san. I have finally been able to put some mind power into that. > This is somewhat artificial but the same situation could be made > also in the nature. The

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

Hello, At Tue, 14 Jun 2016 21:24:58 +0900, Michael Paquier wrote in

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

Sorry, I'm confused about the minRecoveryPoint. Reconsidered a bit. At Tue, 14 Jun 2016 20:31:11 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20160614.203111.229211034.horiguchi.kyot...@lab.ntt.co.jp> > > > After looking more closely, I found that

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Tue, Jun 14, 2016 at 8:31 PM, Kyotaro HORIGUCHI wrote: >> +# Take a second backup of the standby while the master is offline. >> +$node_master->stop; >> +$node_standby_1->backup('my_backup_2'); >> +$node_master->start; > > I'm not sure that adding the test case

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

Hello, thank you for looking this. At Fri, 10 Jun 2016 17:39:59 +0900, Michael Paquier wrote in > On Thu, Jun 9, 2016 at 9:55 PM, Kyotaro HORIGUCHI > wrote: > >

Re: [HACKERS] [BUG] pg_basebackup from disconnected standby fails

On Thu, Jun 9, 2016 at 9:55 PM, Kyotaro HORIGUCHI wrote: > Hello, I found that pg_basebackup from a replication standby > fails after the following steps, on 9.3 and the master. > > - start a replication master > - start a replication standby > - stop the master