Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-04-13 Thread Fujii Masao
On Tue, Apr 13, 2010 at 10:27 PM, Robert Haas wrote: > Can you explain how to recreate the problem that this patch fixes? 1. Configure and start the primary server. 2. Configure the standby server. 3. Remove all of the WAL files in pg_xlog of the standby. 4. Start the standby. 5. Request smart sh

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-04-13 Thread Robert Haas
On Tue, Apr 13, 2010 at 9:18 AM, Fujii Masao wrote: > On Thu, Apr 1, 2010 at 8:24 PM, Robert Haas wrote: >> On Thu, Apr 1, 2010 at 7:18 AM, Simon Riggs wrote: >>> I'm not willing to investigate this further myself at this stage. This >>> looks like risk for little benefit. >> >> That's kind of w

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-04-13 Thread Fujii Masao
On Thu, Apr 1, 2010 at 8:24 PM, Robert Haas wrote: > On Thu, Apr 1, 2010 at 7:18 AM, Simon Riggs wrote: >> I'm not willing to investigate this further myself at this stage. This >> looks like risk for little benefit. > > That's kind of what I figured.  I'll see about fixing up Fujii-san's > patch

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-04-01 Thread Robert Haas
On Thu, Apr 1, 2010 at 7:18 AM, Simon Riggs wrote: > I'm not willing to investigate this further myself at this stage. This > looks like risk for little benefit. That's kind of what I figured. I'll see about fixing up Fujii-san's patch and documenting the behavior; but it won't happen before the

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-04-01 Thread Simon Riggs
On Thu, 2010-04-01 at 06:48 -0400, Robert Haas wrote: > On Thu, Apr 1, 2010 at 4:42 AM, Fujii Masao wrote: > > On Thu, Apr 1, 2010 at 12:16 AM, Robert Haas wrote: > >> On Wed, Mar 31, 2010 at 5:02 AM, Simon Riggs wrote: > > >From what I have seen, the comment about PM_WAIT_BACKENDS is > >>

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-04-01 Thread Robert Haas
On Thu, Apr 1, 2010 at 4:42 AM, Fujii Masao wrote: > On Thu, Apr 1, 2010 at 12:16 AM, Robert Haas wrote: >> On Wed, Mar 31, 2010 at 5:02 AM, Simon Riggs wrote: > >From what I have seen, the comment about PM_WAIT_BACKENDS is incorrect. > "backends might be waiting for the WAL record tha

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-04-01 Thread Fujii Masao
On Thu, Apr 1, 2010 at 12:16 AM, Robert Haas wrote: > On Wed, Mar 31, 2010 at 5:02 AM, Simon Riggs wrote: >>> > >From what I have seen, the comment about PM_WAIT_BACKENDS is incorrect. >>> > "backends might be waiting for the WAL record that conflicts with their >>> > queries to be replayed". Rec

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-03-31 Thread Robert Haas
On Wed, Mar 31, 2010 at 5:02 AM, Simon Riggs wrote: >> > >From what I have seen, the comment about PM_WAIT_BACKENDS is incorrect. >> > "backends might be waiting for the WAL record that conflicts with their >> > queries to be replayed". Recovery sometimes waits for backends, but >> > backends neve

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-03-31 Thread Robert Haas
On Wed, Mar 31, 2010 at 4:00 AM, Simon Riggs wrote: > Please add some docs that a) explains what the patch does and b) notes > any changes from behaviour in previous releases. ISTM this is a major > change in behaviour. I guess I see this a little bit differently. If you do a smart shutdown on 8

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-03-31 Thread Fujii Masao
On Wed, Mar 31, 2010 at 6:02 PM, Simon Riggs wrote: > On Wed, 2010-03-31 at 17:48 +0900, Fujii Masao wrote: >> On Wed, Mar 31, 2010 at 5:00 PM, Simon Riggs wrote: >> > Please add some docs that a) explains what the patch does and b) notes >> > any changes from behaviour in previous releases. ISTM

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-03-31 Thread Simon Riggs
On Wed, 2010-03-31 at 17:48 +0900, Fujii Masao wrote: > On Wed, Mar 31, 2010 at 5:00 PM, Simon Riggs wrote: > > Please add some docs that a) explains what the patch does and b) notes > > any changes from behaviour in previous releases. ISTM this is a major > > change in behaviour. > > How about a

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-03-31 Thread Fujii Masao
On Wed, Mar 31, 2010 at 5:00 PM, Simon Riggs wrote: > Please add some docs that a) explains what the patch does and b) notes > any changes from behaviour in previous releases. ISTM this is a major > change in behaviour. How about adding the following description into "17.5. Shutting Down the Serv

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-03-31 Thread Simon Riggs
On Wed, 2010-03-31 at 10:48 +0900, Fujii Masao wrote: > On Wed, Mar 31, 2010 at 9:47 AM, Robert Haas wrote: > > On Tue, Mar 30, 2010 at 5:09 AM, Fujii Masao wrote: > >> I rebased the patch to HEAD. Is the patch still required for 9.0? > >> If not, I'd remove the open item of the smart shutdown du

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-03-30 Thread Robert Haas
On Tue, Mar 30, 2010 at 9:48 PM, Fujii Masao wrote: > On Wed, Mar 31, 2010 at 9:47 AM, Robert Haas wrote: >> On Tue, Mar 30, 2010 at 5:09 AM, Fujii Masao wrote: >>> I rebased the patch to HEAD. Is the patch still required for 9.0? >>> If not, I'd remove the open item of the smart shutdown during

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-03-30 Thread Fujii Masao
On Wed, Mar 31, 2010 at 9:47 AM, Robert Haas wrote: > On Tue, Mar 30, 2010 at 5:09 AM, Fujii Masao wrote: >> I rebased the patch to HEAD. Is the patch still required for 9.0? >> If not, I'd remove the open item of the smart shutdown during >> recovery. > > I am by no means an expert on this area

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-03-30 Thread Robert Haas
On Tue, Mar 30, 2010 at 5:09 AM, Fujii Masao wrote: > I rebased the patch to HEAD. Is the patch still required for 9.0? > If not, I'd remove the open item of the smart shutdown during > recovery. I am by no means an expert on this area of the code, but in the interest of moving things along I rev

pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-03-30 Thread Fujii Masao
On Mon, Feb 1, 2010 at 11:49 AM, Fujii Masao wrote: > On Sat, Jan 30, 2010 at 12:54 PM, Fujii Masao wrote: >>> HOWEVER, I do believe this is an issue we could live with for 9.0 if >>> it's going to lead to a whole lot of additional debugging of SR.  But if >>> it's an easy fix, it'll avoid a lot

Re: [HACKERS] HS/SR and smart shutdown

2010-03-04 Thread Robert Haas
On Thu, Mar 4, 2010 at 12:39 PM, Greg Stark wrote: > On Thu, Mar 4, 2010 at 3:56 PM, Robert Haas wrote: >> On Thu, Mar 4, 2010 at 10:17 AM, Fujii Masao wrote: >>> >>> Yes. More precisely, smart shutdown during recovery does not complete >>> until recovery ends. >> >> Well, I don't think we shoul

Re: [HACKERS] HS/SR and smart shutdown

2010-03-04 Thread Greg Stark
On Thu, Mar 4, 2010 at 3:56 PM, Robert Haas wrote: > On Thu, Mar 4, 2010 at 10:17 AM, Fujii Masao wrote: >> >> Yes. More precisely, smart shutdown during recovery does not complete >> until recovery ends. > > Well, I don't think we should let smart shutdown just never terminate > when standby_mod

Re: [HACKERS] HS/SR and smart shutdown

2010-03-04 Thread Robert Haas
On Thu, Mar 4, 2010 at 10:17 AM, Fujii Masao wrote: > On Thu, Mar 4, 2010 at 11:55 PM, Greg Stark wrote: >> On Thu, Mar 4, 2010 at 12:11 PM, Fujii Masao wrote: >>> There is no post about this for over a month. Can I remove this >>> from TODO item of SR for 9.0? Thought? Objection? >>> >> >> Does

Re: [HACKERS] HS/SR and smart shutdown

2010-03-04 Thread Fujii Masao
On Thu, Mar 4, 2010 at 11:55 PM, Greg Stark wrote: > On Thu, Mar 4, 2010 at 12:11 PM, Fujii Masao wrote: >> There is no post about this for over a month. Can I remove this >> from TODO item of SR for 9.0? Thought? Objection? >> > > Does smart shutdown still fail to shut down a slave? Yes. More p

Re: [HACKERS] HS/SR and smart shutdown

2010-03-04 Thread Greg Stark
On Thu, Mar 4, 2010 at 12:11 PM, Fujii Masao wrote: > There is no post about this for over a month. Can I remove this > from TODO item of SR for 9.0? Thought? Objection? > Does smart shutdown still fail to shut down a slave? -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgr

Re: [HACKERS] HS/SR and smart shutdown

2010-03-04 Thread Fujii Masao
On Mon, Feb 1, 2010 at 11:49 AM, Fujii Masao wrote: > On Sat, Jan 30, 2010 at 12:54 PM, Fujii Masao wrote: >>> HOWEVER, I do believe this is an issue we could live with for 9.0 if >>> it's going to lead to a whole lot of additional debugging of SR.  But if >>> it's an easy fix, it'll avoid a lot

Re: [HACKERS] HS/SR and smart shutdown

2010-01-31 Thread Fujii Masao
On Sat, Jan 30, 2010 at 12:54 PM, Fujii Masao wrote: >> HOWEVER, I do believe this is an issue we could live with for 9.0 if >> it's going to lead to a whole lot of additional debugging of SR. But if >> it's an easy fix, it'll avoid a lot of complaints on pgsql-general. > > I think that the latte

Re: [HACKERS] HS/SR and smart shutdown

2010-01-31 Thread Magnus Hagander
On Sat, Jan 30, 2010 at 01:05, Robert Haas wrote: > On Fri, Jan 29, 2010 at 7:01 PM, Josh Berkus wrote: It's a good question if that still makes sense with Hot Standby. Perhaps we should redefine smart shutdown in standby mode to shut down as soon as all read-only connections have

Re: [HACKERS] HS/SR and smart shutdown

2010-01-29 Thread Fujii Masao
On Sat, Jan 30, 2010 at 9:01 AM, Josh Berkus wrote: > I don't think it's clear, or intuitive for users.  In SR, recovery is > *never* done, so smart shutdown never completes (even if the master is > shut down, when I tested it). If you specify the trigger_file parameter in the recovery.conf, the

Re: [HACKERS] HS/SR and smart shutdown

2010-01-29 Thread Robert Haas
On Fri, Jan 29, 2010 at 7:01 PM, Josh Berkus wrote: >>> It's a good question if that still makes sense with Hot Standby. >>> Perhaps we should redefine smart shutdown in standby mode to shut down >>> as soon as all read-only connections have died. >> It's clear that "smart" shutdown doesn't work w

Re: [HACKERS] HS/SR and smart shutdown

2010-01-29 Thread Josh Berkus
>> It's a good question if that still makes sense with Hot Standby. >> Perhaps we should redefine smart shutdown in standby mode to shut down >> as soon as all read-only connections have died. > > It's clear that "smart" shutdown doesn't work while something is active. > Recovery is "active" and

Re: [HACKERS] HS/SR and smart shutdown

2010-01-29 Thread Simon Riggs
On Thu, 2010-01-21 at 09:27 +0200, Heikki Linnakangas wrote: > Right, that's the way a standby server (= one still in recovery) has > always behaved. It has made sense in the past: it's not in the spirit > of smart shutdown to kill the WAL replay immediately. "smart" means > wait for recovery to f

Re: [HACKERS] HS/SR and smart shutdown

2010-01-29 Thread Heikki Linnakangas
Josh Berkus wrote: >> I guess that the startup process and the walreceiver should wait >> for all read only backends to exit in smart shutdown case. It's >> because those backends might be waiting for the record that conflicts >> with their queries to be replayed. Is this OK? Or we should kill the

Re: [HACKERS] HS/SR and smart shutdown

2010-01-29 Thread Josh Berkus
Fujii, > I guess that the startup process and the walreceiver should wait > for all read only backends to exit in smart shutdown case. It's > because those backends might be waiting for the record that conflicts > with their queries to be replayed. Is this OK? Or we should kill the > startup proce

Re: [HACKERS] HS/SR and smart shutdown

2010-01-29 Thread Fujii Masao
On Thu, Jan 21, 2010 at 4:27 PM, Heikki Linnakangas wrote: > It's a good question if that still makes sense with Hot Standby. Perhaps > we should redefine smart shutdown in standby mode to shut down as soon > as all read-only connections have died. Okay. Let's work out the details. I guess that

Re: [HACKERS] HS/SR and smart shutdown

2010-01-21 Thread Greg Smith
Heikki Linnakangas wrote: It's a good question if that still makes sense with Hot Standby. Perhaps we should redefine smart shutdown in standby mode to shut down as soon as all read-only connections have died. I've advocated in the past that an escalating shutdown procedure would be helpful

Re: [HACKERS] HS/SR and smart shutdown

2010-01-20 Thread Heikki Linnakangas
Fujii Masao wrote: > On Thu, Jan 21, 2010 at 10:44 AM, Josh Berkus wrote: >>> If it's "standby", it's a previously-existing behavior that a "smart" >>> shutdown doesn't work immediately during recovery. After a recovery >>> has been completed, it would work. Of course, I agree that such a >>> beha

Re: [HACKERS] HS/SR and smart shutdown

2010-01-20 Thread Fujii Masao
On Thu, Jan 21, 2010 at 10:44 AM, Josh Berkus wrote: > >> If it's "standby", it's a previously-existing behavior that a "smart" >> shutdown doesn't work immediately during recovery. After a recovery >> has been completed, it would work. Of course, I agree that such a >> behavior should be document

Re: [HACKERS] HS/SR and smart shutdown

2010-01-20 Thread Mark Kirkwood
Tom Lane wrote: Robert Haas writes: On Wed, Jan 20, 2010 at 8:44 PM, Josh Berkus wrote: Well, as long as streaming rep is running, you can't do a smart shutdown ... smart shutdown seems to treat the walreciever as a client connection. At the very least, this should be in the document

Re: [HACKERS] HS/SR and smart shutdown

2010-01-20 Thread Robert Haas
On Wed, Jan 20, 2010 at 8:56 PM, Tom Lane wrote: > Robert Haas writes: >> On Wed, Jan 20, 2010 at 8:44 PM, Josh Berkus wrote: >>> Well, as long as streaming rep is running, you can't do a smart shutdown >>> ... smart shutdown seems to treat the walreciever as a client >>> connection.  At the ver

Re: [HACKERS] HS/SR and smart shutdown

2010-01-20 Thread Tom Lane
Robert Haas writes: > On Wed, Jan 20, 2010 at 8:44 PM, Josh Berkus wrote: >> Well, as long as streaming rep is running, you can't do a smart shutdown >> ... smart shutdown seems to treat the walreciever as a client >> connection.  At the very least, this should be in the documentation. > How har

Re: [HACKERS] HS/SR and smart shutdown

2010-01-20 Thread Robert Haas
On Wed, Jan 20, 2010 at 8:44 PM, Josh Berkus wrote: >> If it's "standby", it's a previously-existing behavior that a "smart" >> shutdown doesn't work immediately during recovery. After a recovery >> has been completed, it would work. Of course, I agree that such a >> behavior should be documented.

Re: [HACKERS] HS/SR and smart shutdown

2010-01-20 Thread Josh Berkus
> If it's "standby", it's a previously-existing behavior that a "smart" > shutdown doesn't work immediately during recovery. After a recovery > has been completed, it would work. Of course, I agree that such a > behavior should be documented. Well, as long as streaming rep is running, you can't d

Re: [HACKERS] HS/SR and smart shutdown

2010-01-20 Thread Fujii Masao
On Thu, Jan 21, 2010 at 8:04 AM, Josh Berkus wrote: > I've been working on my demo, and I'm discovering that due to the > connection from the walsender and walreceiver, "smart" shutdown from > pg_ctl doesn't work if replication is active. > > This seems worth fixing; if we don't fix it, we should