Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-04-03 Thread Michael Banck
Am Samstag, den 01.04.2017, 17:29 +0200 schrieb Magnus Hagander: > I've applied a backpatch to 9.4. Prior to that pretty much the entire > patch is a conflict, so it would need a full rewrite. Thanks! Michael -- Michael Banck Projektleiter / Senior Berater Tel.: +49 2166 9901-171 Fax: +49 21

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-04-01 Thread Magnus Hagander
On Fri, Mar 31, 2017 at 8:59 AM, Magnus Hagander wrote: > > On Wed, Mar 29, 2017 at 1:05 PM, Michael Banck > wrote: > >> Hi, >> >> Am Montag, den 27.02.2017, 16:20 +0100 schrieb Magnus Hagander: >> > On Sun, Feb 26, 2017 at 9:59 PM, Tom Lane wrote: >> > Is there an argument for back-pat

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-04-01 Thread Magnus Hagander
On Mon, Feb 27, 2017 at 7:46 PM, Jeff Janes wrote: > On Sun, Feb 26, 2017 at 12:32 PM, Magnus Hagander > wrote: > >> >> On Sun, Feb 26, 2017 at 8:27 PM, Michael Banck > > wrote: >> >>> Hi, >>> >>> Am Dienstag, den 14.02.2017, 18:18 -0500 schrieb Robert Haas: >>> > On Tue, Feb 14, 2017 at 4:06 PM

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-03-31 Thread Magnus Hagander
On Wed, Mar 29, 2017 at 1:05 PM, Michael Banck wrote: > Hi, > > Am Montag, den 27.02.2017, 16:20 +0100 schrieb Magnus Hagander: > > On Sun, Feb 26, 2017 at 9:59 PM, Tom Lane wrote: > > Is there an argument for back-patching this? > > > > > > Seems you were typing that at the same time as

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-03-29 Thread Michael Banck
Hi, Am Montag, den 27.02.2017, 16:20 +0100 schrieb Magnus Hagander: > On Sun, Feb 26, 2017 at 9:59 PM, Tom Lane wrote: > Is there an argument for back-patching this? > > > Seems you were typing that at the same time as we did. > > > I'm considering it, but not swayed in either directi

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-27 Thread Jeff Janes
On Sun, Feb 26, 2017 at 12:32 PM, Magnus Hagander wrote: > > On Sun, Feb 26, 2017 at 8:27 PM, Michael Banck > wrote: > >> Hi, >> >> Am Dienstag, den 14.02.2017, 18:18 -0500 schrieb Robert Haas: >> > On Tue, Feb 14, 2017 at 4:06 PM, Alvaro Herrera >> > wrote: >> > > I'd rather have a --quiet mod

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-27 Thread Simon Riggs
On 26 February 2017 at 20:55, Magnus Hagander wrote: > What do others think? Changing the output behaviour of a command isn't something we usually do as a backpatch. This change doesn't affect the default behaviour so probably wouldn't make a difference to the outcome of the situation that gene

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-27 Thread Tom Lane
Magnus Hagander writes: > On Sun, Feb 26, 2017 at 9:59 PM, Tom Lane wrote: >> Is there an argument for back-patching this? > I'm considering it, but not swayed in either direction. Should I take your > comment as a vote that we should back-patch it? Yeah, I'd vote for it.

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-27 Thread Magnus Hagander
On Sun, Feb 26, 2017 at 9:59 PM, Tom Lane wrote: > Magnus Hagander writes: > > On Sun, Feb 26, 2017 at 8:27 PM, Michael Banck < > michael.ba...@credativ.de> > > wrote: > >> ISTM the consensus is that there should be no output in regular mode, > >> but a message should be displayed in verbose and

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-26 Thread Tom Lane
Magnus Hagander writes: > On Sun, Feb 26, 2017 at 8:27 PM, Michael Banck > wrote: >> ISTM the consensus is that there should be no output in regular mode, >> but a message should be displayed in verbose and progress mode. > Agreed, and applied as one patch. Is there an argument for back-patchin

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-26 Thread Magnus Hagander
On Sun, Feb 26, 2017 at 9:53 PM, Michael Banck wrote: > Hi, > > Am Sonntag, den 26.02.2017, 21:32 +0100 schrieb Magnus Hagander: > > > On Sun, Feb 26, 2017 at 8:27 PM, Michael Banck > > wrote: > > > Agreed, and applied as one patch. Except I noticed you also fixed a > > couple of entries which w

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-26 Thread Michael Banck
Hi, Am Sonntag, den 26.02.2017, 21:32 +0100 schrieb Magnus Hagander: > On Sun, Feb 26, 2017 at 8:27 PM, Michael Banck > wrote: > Agreed, and applied as one patch. Except I noticed you also fixed a > couple of entries which were missing the progname in the messages -- I > broke those out to a se

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-26 Thread Magnus Hagander
On Sun, Feb 26, 2017 at 8:27 PM, Michael Banck wrote: > Hi, > > Am Dienstag, den 14.02.2017, 18:18 -0500 schrieb Robert Haas: > > On Tue, Feb 14, 2017 at 4:06 PM, Alvaro Herrera > > wrote: > > > I'd rather have a --quiet mode instead. If you're running it by hand, > > > you're likely to omit th

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-26 Thread Michael Banck
Hi, Am Dienstag, den 14.02.2017, 18:18 -0500 schrieb Robert Haas: > On Tue, Feb 14, 2017 at 4:06 PM, Alvaro Herrera > wrote: > > I'd rather have a --quiet mode instead. If you're running it by hand, > > you're likely to omit the switch, whereas when writing the cron job > > you're going to notic

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-19 Thread Robert Haas
On Sat, Feb 18, 2017 at 4:52 AM, Tomas Vondra wrote: > I have my doubts about this actually addressing gitlab-like mistakes, > though, because it's a helluva jump from "It's waiting and not doing > anything," to "We need to remove the datadir." (One of the reasons being > that non-empty directory

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-17 Thread David G. Johnston
On Fri, Feb 17, 2017 at 4:22 PM, Tomas Vondra wrote: > What about adding a paragraph into pg_basebackup docs, explaining that > with 'fast' it does immediate checkpoint, while with 'spread' it'll wait > for a spread checkpoint. > I agree that a better, and self-contained, explanation of the beha

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-17 Thread Tomas Vondra
On 02/17/2017 08:17 PM, Jim Nasby wrote: > On 2/14/17 5:18 PM, Robert Haas wrote: >> On Tue, Feb 14, 2017 at 4:06 PM, Alvaro Herrera >> wrote: >>> I'd rather have a --quiet mode instead. If you're running it by hand, >>> you're likely to omit the switch, whereas when writing the cron job >>> you

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-17 Thread Jim Nasby
On 2/14/17 5:18 PM, Robert Haas wrote: On Tue, Feb 14, 2017 at 4:06 PM, Alvaro Herrera wrote: I'd rather have a --quiet mode instead. If you're running it by hand, you're likely to omit the switch, whereas when writing the cron job you're going to notice lack of switch even before you let the

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-15 Thread Jeff Janes
On Tue, Feb 14, 2017 at 9:06 AM, Magnus Hagander wrote: Yeah, that's my view as well. I'm all for including it in verbose mode. > > *Iff* we can get a progress indicator through the checkpoint we could > include that in --progress mode. But that's a different patch, of course, > but it shouldn't

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-14 Thread Robert Haas
On Tue, Feb 14, 2017 at 4:06 PM, Alvaro Herrera wrote: > I'd rather have a --quiet mode instead. If you're running it by hand, > you're likely to omit the switch, whereas when writing the cron job > you're going to notice lack of switch even before you let the job run > once. Well, that might've

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-14 Thread Alvaro Herrera
Robert Haas wrote: > On Tue, Feb 14, 2017 at 12:06 PM, Magnus Hagander wrote: > > However, outputing this info by default will make it show up in things like > > everybodys cronjobs by default. Right now a successful pg_basebackup run > > will come out with no output at all, which is how most Unix

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-14 Thread Robert Haas
On Tue, Feb 14, 2017 at 12:06 PM, Magnus Hagander wrote: > However, outputing this info by default will make it show up in things like > everybodys cronjobs by default. Right now a successful pg_basebackup run > will come out with no output at all, which is how most Unix commands work, > and bring

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-14 Thread Magnus Hagander
On Mon, Feb 13, 2017 at 10:33 AM, Michael Banck wrote: > Hi, > > Am Montag, den 13.02.2017, 09:31 +0100 schrieb Magnus Hagander: > > On Mon, Feb 13, 2017 at 3:29 AM, Jim Nasby > > wrote: > > On 2/11/17 4:36 AM, Michael Banck wrote: > > I guess you're right, I've moved it

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-13 Thread Michael Banck
Hi, Am Montag, den 13.02.2017, 09:31 +0100 schrieb Magnus Hagander: > On Mon, Feb 13, 2017 at 3:29 AM, Jim Nasby > wrote: > On 2/11/17 4:36 AM, Michael Banck wrote: > I guess you're right, I've moved it further down. > There is in fact a > m

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-13 Thread Magnus Hagander
On Mon, Feb 13, 2017 at 3:29 AM, Jim Nasby wrote: > On 2/11/17 4:36 AM, Michael Banck wrote: > >> I guess you're right, I've moved it further down. There is in fact a >> message about the xlog location (unless you switch off wal entirely), >> but having another one right before that mentioning th

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-12 Thread Jim Nasby
On 2/11/17 4:36 AM, Michael Banck wrote: I guess you're right, I've moved it further down. There is in fact a message about the xlog location (unless you switch off wal entirely), but having another one right before that mentioning the completed checkpoint sounds ok to me. 1) I don't think this

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-11 Thread Michael Banck
Hi, Am Samstag, den 11.02.2017, 11:25 +0100 schrieb Michael Banck: > Am Samstag, den 11.02.2017, 11:07 +0100 schrieb Magnus Hagander: > > As for the code, while I haven't tested it, isn't the "checkpoint > > completed" message in the wrong place? Doesn't PQsendQuery() complete > > immediately, and

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-11 Thread Michael Banck
Hi, Am Samstag, den 11.02.2017, 11:07 +0100 schrieb Magnus Hagander: > As for the code, while I haven't tested it, isn't the "checkpoint > completed" message in the wrong place? Doesn't PQsendQuery() complete > immediately, and the check needs to be put *after* the PQgetResult() > call? I gues

Re: [HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-11 Thread Magnus Hagander
On Sat, Feb 11, 2017 at 10:38 AM, Michael Banck wrote: > Hi, > > one take-away from the Gitlab Post-Mortem[1] appears to be that after > their secondary lost replication, they were confused about what > pg_basebackup was doing when they tried to rebuild it. It just sat there > and did nothing (ev

[HACKERS] gitlab post-mortem: pg_basebackup waiting for checkpoint

2017-02-11 Thread Michael Banck
Hi, one take-away from the Gitlab Post-Mortem[1] appears to be that after their secondary lost replication, they were confused about what pg_basebackup was doing when they tried to rebuild it. It just sat there and did nothing (even with --verbose), so they assumed something was wrong with either