Re: [HACKERS] max_standby_delay considered harmful

2010-05-14 Thread Robert Haas
On Fri, May 14, 2010 at 5:51 PM, Josh Berkus wrote: > >>> This would be OK as long as we document it well.  We patched the >>> shutdown the way we did specifically because Fujii thought it would be >>> an easy fix; if it's complicated, we should revert it and document the >>> issue for DBAs. >> >>

Re: [HACKERS] max_standby_delay considered harmful

2010-05-14 Thread Josh Berkus
>> This would be OK as long as we document it well. We patched the >> shutdown the way we did specifically because Fujii thought it would be >> an easy fix; if it's complicated, we should revert it and document the >> issue for DBAs. > > I don't understand this comment. In other words, I'm sayi

Re: [HACKERS] max_standby_delay considered harmful

2010-05-14 Thread Robert Haas
On Thu, May 13, 2010 at 1:12 PM, Josh Berkus wrote: > On 5/12/10 8:07 PM, Robert Haas wrote: >> I think that would be a good thing to check (it'll confirm whether >> this is the same bug), but I'm not convinced we should actually fix it >> that way.  Prior to 8.4, we handled a smart shutdown durin

Re: [HACKERS] max_standby_delay considered harmful

2010-05-13 Thread Josh Berkus
On 5/12/10 8:07 PM, Robert Haas wrote: > I think that would be a good thing to check (it'll confirm whether > this is the same bug), but I'm not convinced we should actually fix it > that way. Prior to 8.4, we handled a smart shutdown during recovery > at the conclusion of recovery, just prior to

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Robert Haas
On Wed, May 12, 2010 at 10:46 PM, Fujii Masao wrote: > On Thu, May 13, 2010 at 4:55 AM, Robert Haas wrote: >> I am wondering if we are not correctly handling the case where we get >> a shutdown request while we are still in the PM_STARTUP state.  It >> looks like we might go ahead and switch to P

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Fujii Masao
On Thu, May 13, 2010 at 4:55 AM, Robert Haas wrote: > I am wondering if we are not correctly handling the case where we get > a shutdown request while we are still in the PM_STARTUP state.  It > looks like we might go ahead and switch to PM_RECOVERY and then > PM_RECOVERY_CONSISTENT without notici

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Robert Haas
On Wed, May 12, 2010 at 5:34 PM, Simon Riggs wrote: > On Wed, 2010-05-12 at 14:43 -0400, Robert Haas wrote: > >> I thought that it >> would be a good idea for Simon to look at it because, on the surface, >> it APPEARS to have something to do with Hot Standby, since that's what >> Stefan was testin

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Joshua D. Drake
On Wed, 2010-05-12 at 22:34 +0100, Simon Riggs wrote: > On Wed, 2010-05-12 at 14:43 -0400, Robert Haas wrote: > > > I thought that it > > would be a good idea for Simon to look at it because, on the surface, > > it APPEARS to have something to do with Hot Standby, since that's what > > Stefan was

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Josh Berkus
Simon, Robert, > He was also testing SR, yet you haven't breathed a word about that for > some strange reason. It didn't APPEAR like it was HS at all, not from > basic logic or from technical knowledge. So you'll have to forgive me if > I don't leap into action when you say something is an HS prob

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Simon Riggs
On Wed, 2010-05-12 at 14:43 -0400, Robert Haas wrote: > I thought that it > would be a good idea for Simon to look at it because, on the surface, > it APPEARS to have something to do with Hot Standby, since that's what > Stefan was testing when he found it. He was also testing SR, yet you haven't

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Robert Haas
On Wed, May 12, 2010 at 3:51 PM, Robert Haas wrote: > On Wed, May 12, 2010 at 3:36 PM, Alvaro Herrera > wrote: >> Excerpts from Stefan Kaltenbrunner's message of mié may 12 15:10:28 -0400 >> 2010: >> >>> the startup process has the following backtrace: >>> >>> (gdb) bt >>> #0  0x7fbe24cb2c8

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Robert Haas
On Wed, May 12, 2010 at 3:36 PM, Alvaro Herrera wrote: > Excerpts from Stefan Kaltenbrunner's message of mié may 12 15:10:28 -0400 > 2010: > >> the startup process has the following backtrace: >> >> (gdb) bt >> #0  0x7fbe24cb2c83 in select () from /lib/libc.so.6 >> #1  0x006e811a in p

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Simon Riggs
On Wed, 2010-05-12 at 15:36 -0400, Alvaro Herrera wrote: > I just noticed that we have some code assigning the return value of > time() to a pg_time_t variable. Is this supposed to work reliably? > (xlog.c lines 9267ff) Code's used that for a while now. Checkpoints and everywhere. -- Simon Rig

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Alvaro Herrera
Excerpts from Stefan Kaltenbrunner's message of mié may 12 15:10:28 -0400 2010: > the startup process has the following backtrace: > > (gdb) bt > #0 0x7fbe24cb2c83 in select () from /lib/libc.so.6 > #1 0x006e811a in pg_usleep () > #2 0x0048c333 in XLogPageRead () > #3 0x00

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Simon Riggs
On Wed, 2010-05-12 at 21:10 +0200, Stefan Kaltenbrunner wrote: > > There is no evidence to link this behaviour with HS, as yet, and you > > should be considering the possibility the problem lies elsewhere, > > especially since it could be code you committed that is at fault. > > Well I'm not sure

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Stefan Kaltenbrunner
On 05/12/2010 05:28 PM, Simon Riggs wrote: On Wed, 2010-05-12 at 14:18 +0100, Simon Riggs wrote: On Wed, 2010-05-12 at 08:52 -0400, Robert Haas wrote: On Wed, May 12, 2010 at 7:26 AM, Simon Riggs wrote: On Wed, 2010-05-12 at 07:10 -0400, Robert Haas wrote: I'm not sure what to make of this.

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Robert Haas
On Wed, May 12, 2010 at 1:21 PM, Simon Riggs wrote: > On Wed, 2010-05-12 at 18:05 +0100, Greg Stark wrote: > >> I'm not sure who to blame for the shouting match over whose commit >> introduced the bug -- it doesn't seem like a relevant or useful thing >> to argue about, please both stop. > > I hav

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Joshua D. Drake
On Wed, 2010-05-12 at 17:49 +0100, Simon Riggs wrote: > On Wed, 2010-05-12 at 12:04 -0400, Robert Haas wrote: > Normal shutdown didn't work on a standby before HS was committed and it > didn't work afterwards either. Use all the capitals you like but if you > use poor arguments and combine that wi

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Simon Riggs
On Wed, 2010-05-12 at 18:05 +0100, Greg Stark wrote: > I'm not sure who to blame for the shouting match over whose commit > introduced the bug -- it doesn't seem like a relevant or useful thing > to argue about, please both stop. I haven't blamed Robert's code, merely asked him to consider that i

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Greg Stark
On Wed, May 12, 2010 at 5:49 PM, Simon Riggs wrote: > On Wed, 2010-05-12 at 12:04 -0400, Robert Haas wrote: > >> Huh?? The evidence that this bug is linked with HS is that it occurs >> on a server running in HS mode, and not otherwise.  As for whether the >> bug is code I committed, that's certain

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Simon Riggs
On Wed, 2010-05-12 at 12:04 -0400, Robert Haas wrote: > Huh?? The evidence that this bug is linked with HS is that it occurs > on a server running in HS mode, and not otherwise. As for whether the > bug is code I committed, that's certainly possible, but keep in mind > it didn't work at all befor

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Robert Haas
On Wed, May 12, 2010 at 11:28 AM, Simon Riggs wrote: > On Wed, 2010-05-12 at 14:18 +0100, Simon Riggs wrote: >> On Wed, 2010-05-12 at 08:52 -0400, Robert Haas wrote: >> > On Wed, May 12, 2010 at 7:26 AM, Simon Riggs wrote: >> > > On Wed, 2010-05-12 at 07:10 -0400, Robert Haas wrote: >> > > >> > >

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Simon Riggs
On Wed, 2010-05-12 at 14:18 +0100, Simon Riggs wrote: > On Wed, 2010-05-12 at 08:52 -0400, Robert Haas wrote: > > On Wed, May 12, 2010 at 7:26 AM, Simon Riggs wrote: > > > On Wed, 2010-05-12 at 07:10 -0400, Robert Haas wrote: > > > > > >> I'm not sure what to make of this. Sometimes not shutting

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Simon Riggs
On Wed, 2010-05-12 at 16:03 +0200, Stefan Kaltenbrunner wrote: > Simon Riggs wrote: > > On Wed, 2010-05-12 at 08:52 -0400, Robert Haas wrote: > >> On Wed, May 12, 2010 at 7:26 AM, Simon Riggs wrote: > >>> On Wed, 2010-05-12 at 07:10 -0400, Robert Haas wrote: > >>> > I'm not sure what to make

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Stefan Kaltenbrunner
Simon Riggs wrote: On Wed, 2010-05-12 at 08:52 -0400, Robert Haas wrote: On Wed, May 12, 2010 at 7:26 AM, Simon Riggs wrote: On Wed, 2010-05-12 at 07:10 -0400, Robert Haas wrote: I'm not sure what to make of this. Sometimes not shutting down doesn't sound like a feature to me. It acts exac

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Simon Riggs
On Wed, 2010-05-12 at 08:52 -0400, Robert Haas wrote: > On Wed, May 12, 2010 at 7:26 AM, Simon Riggs wrote: > > On Wed, 2010-05-12 at 07:10 -0400, Robert Haas wrote: > > > >> I'm not sure what to make of this. Sometimes not shutting down > >> doesn't sound like a feature to me. > > > > It acts ex

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Robert Haas
On Wed, May 12, 2010 at 7:26 AM, Simon Riggs wrote: > On Wed, 2010-05-12 at 07:10 -0400, Robert Haas wrote: > >> I'm not sure what to make of this.  Sometimes not shutting down >> doesn't sound like a feature to me. > > It acts exactly the same in recovery as in normal running. It is not a > speci

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Greg Stark
On Wed, May 12, 2010 at 12:26 PM, Simon Riggs wrote: > On Wed, 2010-05-12 at 07:10 -0400, Robert Haas wrote: > >> I'm not sure what to make of this.  Sometimes not shutting down >> doesn't sound like a feature to me. > > It acts exactly the same in recovery as in normal running. It is not a > spec

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Simon Riggs
On Wed, 2010-05-12 at 07:10 -0400, Robert Haas wrote: > I'm not sure what to make of this. Sometimes not shutting down > doesn't sound like a feature to me. It acts exactly the same in recovery as in normal running. It is not a special feature of recovery at all, bug or otherwise. You may think

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Robert Haas
On Wed, May 12, 2010 at 2:50 AM, Simon Riggs wrote: > On Tue, 2010-05-11 at 14:01 +0900, Fujii Masao wrote: >> On Mon, May 10, 2010 at 3:27 PM, Simon Riggs wrote: >> > I already explained that killing the startup process first is a bad idea >> > for many reasons when shutdown was discussed. Can't

Re: [HACKERS] max_standby_delay considered harmful

2010-05-11 Thread Simon Riggs
On Tue, 2010-05-11 at 14:01 +0900, Fujii Masao wrote: > On Mon, May 10, 2010 at 3:27 PM, Simon Riggs wrote: > > I already explained that killing the startup process first is a bad idea > > for many reasons when shutdown was discussed. Can't remember who added > > the new standby shutdown code rece

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Fujii Masao
On Mon, May 10, 2010 at 3:27 PM, Simon Riggs wrote: > I already explained that killing the startup process first is a bad idea > for many reasons when shutdown was discussed. Can't remember who added > the new standby shutdown code recently, but it sounds like their design > was pretty poor if it

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Florian Pflug
On May 10, 2010, at 17:39 , Kevin Grittner wrote: > Bruce Momjian wrote: >> Robert Haas wrote: > >>> Overall I would say opinion is about evenly split between: >>> >>> - leave it as-is >>> - make it a Boolean >>> - change it in some way but to something more expressive than a >>> Boolean > > I

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Josh Berkus
> 1) Replace max_standby_delay with a boolean as per heikki's suggestion > > 2) Add an explicitly experimental option like max_standby_delay or > recovery_conflict_timeout which is only effective if you've chosen > recovery_conflict="pause recovery" > option and is explicitly documented as being

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Greg Stark
On Mon, May 10, 2010 at 5:20 PM, Bruce Momjian wrote: > You are right that we are much more flexible about changing > administrative configuration parameters (like this one) than SQL. In the > past, we even renamed logging parameters to be more consistent, and I > think that proves the bar is quit

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Bruce Momjian
Kevin Grittner wrote: > Bruce Momjian wrote: > > Robert Haas wrote: > > >> Overall I would say opinion is about evenly split between: > >> > >> - leave it as-is > >> - make it a Boolean > >> - change it in some way but to something more expressive than a > >> Boolean > > I think a boolean w

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Stephen Frost
* Aidan Van Dyk (ai...@highrise.ca) wrote: > * Heikki Linnakangas [100510 06:03]: > > A problem with using the name "max_standby_delay" for Tom's suggestion > > is that it sounds like a hard limit, which it isn't. But if we name it > > something like: > > I'ld still rather an "if your killing som

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Kevin Grittner
Bruce Momjian wrote: > Robert Haas wrote: >> Overall I would say opinion is about evenly split between: >> >> - leave it as-is >> - make it a Boolean >> - change it in some way but to something more expressive than a >> Boolean I think a boolean would limit the environments in which HS woul

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Mike Rylander
On Mon, May 10, 2010 at 6:03 AM, Heikki Linnakangas wrote: > Robert Haas wrote: >> On Thu, May 6, 2010 at 2:47 PM, Josh Berkus wrote: Now that I've realized what the real problem is with max_standby_delay (namely, that inactivity on the master can use up the delay), I think we shou

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Bruce Momjian
Robert Haas wrote: > Wultsch (who doesn't ever want to kill queries and therefore would be > happy with a boolean), Yeb Havinga (who never wants to stall recovery > and therefore would also be happy with a boolean), and Florian Pflug > (who points out that pause/resume is actually a nontrivial feat

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Bruce Momjian
Simon Riggs wrote: > Bruce has used the word crippleware for the current state. Raising a > problem and then blocking solutions is the best way I know to cripple a > release. It should be clear that I've done my best to avoid this FYI, it was Robert Haas who used the term "crippleware" to describe

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Andres Freund
On Monday 10 May 2010 14:00:45 Heikki Linnakangas wrote: > Florian Pflug wrote: > > On May 10, 2010, at 11:43 , Heikki Linnakangas wrote: > >> If you're not going to apply any more WAL records before shutdown, you > >> could also just release all the AccessExclusiveLocks held by the startup > >> pr

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Heikki Linnakangas
Florian Pflug wrote: > On May 10, 2010, at 11:43 , Heikki Linnakangas wrote: >> If you're not going to apply any more WAL records before shutdown, you >> could also just release all the AccessExclusiveLocks held by the startup >> process. Whatever the transaction was doing with the locked relation,

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Robert Haas
On Mon, May 10, 2010 at 6:13 AM, Florian Pflug wrote: > On May 10, 2010, at 11:43 , Heikki Linnakangas wrote: >> If you're not going to apply any more WAL records before shutdown, you >> could also just release all the AccessExclusiveLocks held by the startup >> process. Whatever the transaction w

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Robert Haas
On Mon, May 10, 2010 at 6:03 AM, Heikki Linnakangas wrote: > Yeah, I could live with that. > > A problem with using the name "max_standby_delay" for Tom's suggestion > is that it sounds like a hard limit, which it isn't. But if we name it > something like: > > # -1 = no timeout > # 0 = kill confli

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Robert Haas
On Mon, May 10, 2010 at 2:27 AM, Simon Riggs wrote: > I already explained that killing the startup process first is a bad idea > for many reasons when shutdown was discussed. Can't remember who added > the new standby shutdown code recently, but it sounds like their design > was pretty poor if it

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Aidan Van Dyk
* Heikki Linnakangas [100510 06:03]: > A problem with using the name "max_standby_delay" for Tom's suggestion > is that it sounds like a hard limit, which it isn't. But if we name it > something like: I'ld still rather an "if your killing something, make sure you kill enough to get all the way

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Florian Pflug
On May 10, 2010, at 11:43 , Heikki Linnakangas wrote: > If you're not going to apply any more WAL records before shutdown, you > could also just release all the AccessExclusiveLocks held by the startup > process. Whatever the transaction was doing with the locked relation, if > we're not going to r

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Heikki Linnakangas
Robert Haas wrote: > On Thu, May 6, 2010 at 2:47 PM, Josh Berkus wrote: >>> Now that I've realized what the real problem is with max_standby_delay >>> (namely, that inactivity on the master can use up the delay), I think >>> we should do what Tom originally suggested here. It's not as good as >>>

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Heikki Linnakangas
Robert Haas wrote: > On Sun, May 9, 2010 at 6:58 PM, Andres Freund wrote: >> On Monday 10 May 2010 00:25:44 Florian Pflug wrote: >>> On May 9, 2010, at 22:01 , Robert Haas wrote: On Sun, May 9, 2010 at 3:09 PM, Dimitri Fontaine >> wrote: Seems like it could take FOREVER on a busy system

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Dimitri Fontaine
Robert Haas writes: > On Sun, May 9, 2010 at 6:58 PM, Andres Freund wrote: >> The difference is that in HS you have to wait for a moment where *no >> exclusive >> lock at all* exist, possibly without contending for any of them, while on the >> master you might not even blocked by the existence o

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Simon Riggs
On Sun, 2010-05-09 at 20:56 -0400, Robert Haas wrote: > >> > Seems like it could take FOREVER on a busy system. Surely that's not > >> > OK. The fact that Hot Standby has to take exclusive locks that can't > >> > be released until WAL replay has progressed to a certain point seems > >> > like a

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Robert Haas
On Sun, May 9, 2010 at 6:58 PM, Andres Freund wrote: > On Monday 10 May 2010 00:25:44 Florian Pflug wrote: >> On May 9, 2010, at 22:01 , Robert Haas wrote: >> > On Sun, May 9, 2010 at 3:09 PM, Dimitri Fontaine > wrote: >> >> Florian Pflug writes: >> >>> The only remaining option is to continue a

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Andres Freund
On Monday 10 May 2010 00:25:44 Florian Pflug wrote: > On May 9, 2010, at 22:01 , Robert Haas wrote: > > On Sun, May 9, 2010 at 3:09 PM, Dimitri Fontaine wrote: > >> Florian Pflug writes: > >>> The only remaining option is to continue applying WAL until you reach > >>> a point where no locks are

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Florian Pflug
On May 9, 2010, at 22:01 , Robert Haas wrote: > On Sun, May 9, 2010 at 3:09 PM, Dimitri Fontaine > wrote: >> Florian Pflug writes: >>> The only remaining option is to continue applying WAL until you reach >>> a point where no locks are held, then pause. But from a user's POV >>> that is nearly i

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Simon Riggs
On Sun, 2010-05-09 at 16:01 -0400, Robert Haas wrote: > The fact that Hot Standby has to take exclusive locks that can't > be released until WAL replay has progressed to a certain point seems > like a fairly serious wart. LOL And people lecture me about design. -- Simon Riggs www.2n

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Robert Haas
On Sun, May 9, 2010 at 3:09 PM, Dimitri Fontaine wrote: > Florian Pflug writes: >> The only remaining option is to continue applying WAL until you reach >> a point where no locks are held, then pause. But from a user's POV >> that is nearly indistinguishable from simply setting >> hot_standby_con

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Florian Pflug
On May 9, 2010, at 21:04 , Simon Riggs wrote: > On Sun, 2010-05-09 at 16:10 +0200, Florian Pflug wrote: > >> Adding pause/resume seems to introduce some non-trivial locking >> problems, though. How would you handle a pause request if the recovery >> process currently held a lock? > > (We are only

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Dimitri Fontaine
Florian Pflug writes: > The only remaining option is to continue applying WAL until you reach > a point where no locks are held, then pause. But from a user's POV > that is nearly indistinguishable from simply setting > hot_standby_conflict_winner to in the first place I think. Not really, the us

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Simon Riggs
On Sun, 2010-05-09 at 16:10 +0200, Florian Pflug wrote: > Adding pause/resume seems to introduce some non-trivial locking > problems, though. How would you handle a pause request if the recovery > process currently held a lock? (We are only talking about AccessExclusiveLocks here. No LWlocks are

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Simon Riggs
On Sat, 2010-05-08 at 20:57 -0400, Tom Lane wrote: > Andres Freund writes: > > On Sunday 09 May 2010 01:34:18 Bruce Momjian wrote: > >> I think everyone agrees the current code is unusable, per Heikki's > >> comment about a WAL file arriving after a period of no WAL activity, and > >> look how lon

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Robert Haas
On Sun, May 9, 2010 at 12:47 PM, Greg Stark wrote: > On Sun, May 9, 2010 at 4:00 AM, Greg Smith wrote: >>  The use cases are covered as best they can be without better support from >> expected future SR features like heartbeats and XID loopback. > > For what it's worth I think deferring these ext

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Greg Stark
On Sun, May 9, 2010 at 4:00 AM, Greg Smith wrote: >  The use cases are covered as best they can be without better support from > expected future SR features like heartbeats and XID loopback. For what it's worth I think deferring these extra complications is a very useful exercise. I would like to

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Florian Pflug
On May 9, 2010, at 13:59 , Dimitri Fontaine wrote: > Tom Lane writes: >> I like the proposal of a boolean because it provides only the minimal >> feature set of two cases that are both clearly needed and easily >> implementable. Whatever we do later is certain to provide a superset >> of those tw

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Dimitri Fontaine
Tom Lane writes: > I like the proposal of a boolean because it provides only the minimal > feature set of two cases that are both clearly needed and easily > implementable. Whatever we do later is certain to provide a superset > of those two cases. If we do something else (and that includes my o

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Kevin Grittner
Bruce Momjian wrote: > I think everyone agrees the current code is unusable, per Heikki's > comment about a WAL file arriving after a period of no WAL > activity I don't. I am curious to hear how many complaints we've had from alpha and beta testers of HS regarding this issue. I know that i

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Simon Riggs
On Sat, 2010-05-08 at 14:48 -0400, Bruce Momjian wrote: > I think the consensus is to change this setting to a boolean. If you > don't want to do it, I am sure we can find someone who will. You expect others to act on consensus and follow rules, yet ignore them yourself when it suits your purpos

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Robert Haas
On Sun, May 9, 2010 at 12:08 AM, Bruce Momjian wrote: > Robert Haas wrote: >> > Clearly, anything is more feature-full than boolean --- the big question >> > is whether Tom's proposal is significantly better than boolean that we >> > should spend the time designing and implementing it, with the >>

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Bruce Momjian
Robert Haas wrote: > > Clearly, anything is more feature-full than boolean --- the big question > > is whether Tom's proposal is significantly better than boolean that we > > should spend the time designing and implementing it, with the > > possibility it will all be changed in 9.1. > > I doubt it

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Robert Haas
On Sat, May 8, 2010 at 6:51 PM, Bruce Momjian wrote: > Robert Haas wrote: >> On Sat, May 8, 2010 at 3:40 PM, Bruce Momjian wrote: >> > Robert Haas wrote: >> >> On Sat, May 8, 2010 at 2:48 PM, Bruce Momjian wrote: >> >> > I think the concensus is to change this setting to a boolean. ?If you >> >>

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Bruce Momjian
Greg Smith wrote: > Tom Lane wrote: > > > Taking out features after they've been in a release is very hard, even if > > we realize they're badly > > designed. > > > > It doesn't have to be; that's the problem the "release often" part takes > care of. If a release has only been out a year, a

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Greg Smith
Tom Lane wrote: Taking out features after they've been in a release is very hard, even if we realize they're badly designed. It doesn't have to be; that's the problem the "release often" part takes care of. If a release has only been out a year, and a new one comes out saying "oh, that

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Tom Lane
Andres Freund writes: > On Sunday 09 May 2010 01:34:18 Bruce Momjian wrote: >> I think everyone agrees the current code is unusable, per Heikki's >> comment about a WAL file arriving after a period of no WAL activity, and >> look how long it took our group to even understand why that fails so >> b

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Andres Freund
On Sunday 09 May 2010 01:34:18 Bruce Momjian wrote: > I think everyone agrees the current code is unusable, per Heikki's > comment about a WAL file arriving after a period of no WAL activity, and > look how long it took our group to even understand why that fails so > badly. To be honest its not *t

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Bruce Momjian
Greg Smith wrote: > Bruce Momjian wrote: > > I think the big question is whether this issue is significant enough > > that we should ignore our policy of no feature design during beta > > The idea that you're considering removal of a feature that we already > have people using in beta and making

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Greg Smith
Bruce Momjian wrote: I think the big question is whether this issue is significant enough that we should ignore our policy of no feature design during beta The idea that you're considering removal of a feature that we already have people using in beta and making plans around is a policy violat

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Bruce Momjian
Robert Haas wrote: > On Sat, May 8, 2010 at 3:40 PM, Bruce Momjian wrote: > > Robert Haas wrote: > >> On Sat, May 8, 2010 at 2:48 PM, Bruce Momjian wrote: > >> > I think the concensus is to change this setting to a boolean. ?If you > >> > don't want to do it, I am sure we can find someone who wil

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Dimitri Fontaine
Bruce Momjian writes: > I have no idea why an objection from you should mean more than an > objection from anyone else in the community, and I have no idea what an > "extreme reaction" means, or why anyone should care. Maybe I shouldn't say anything here. But clearly while you're spot on that Sim

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Robert Haas
On Sat, May 8, 2010 at 3:40 PM, Bruce Momjian wrote: > Robert Haas wrote: >> On Sat, May 8, 2010 at 2:48 PM, Bruce Momjian wrote: >> > I think the concensus is to change this setting to a boolean. ?If you >> > don't want to do it, I am sure we can find someone who will. >> >> I still think we sho

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Bruce Momjian
Robert Haas wrote: > On Sat, May 8, 2010 at 2:48 PM, Bruce Momjian wrote: > > I think the concensus is to change this setting to a boolean. ?If you > > don't want to do it, I am sure we can find someone who will. > > I still think we should revert to Tom's original proposal. And Tom's proposal w

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Robert Haas
On Sat, May 8, 2010 at 2:48 PM, Bruce Momjian wrote: > I think the concensus is to change this setting to a boolean.  If you > don't want to do it, I am sure we can find someone who will. I still think we should revert to Tom's original proposal. -- Robert Haas EnterpriseDB: http://www.enterpri

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Bruce Momjian
Simon Riggs wrote: > With a plugin option, I would not object to option 1. > > If option 1 was taken, without plugins, it's clear that would be against > consensus. > > Having said that, I'll confirm now there will not be an extreme reaction > from me if option (1) was forced, nor do I counsel th

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Simon Riggs
On Thu, 2010-05-06 at 12:03 -0700, Josh Berkus wrote: > So changing to a lock-based mechanism or designing a plugin interface > are really not at all realistic at this date. I agree that changing to a lock-based mechanism is too much at this stage of development. However, putting in a plugin is

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Bruce Momjian
Greg Smith wrote: > Bruce Momjian wrote: > > Remember, delaying wal application just delays making the standby a > > master and makes the slave data appear staler. We can just tell people > > that the larger their queries are, the larger this delay will be. If > > they want to control this, they

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Greg Smith
Bruce Momjian wrote: Remember, delaying wal application just delays making the standby a master and makes the slave data appear staler. We can just tell people that the larger their queries are, the larger this delay will be. If they want to control this, they can set 'statement_timeout' alread

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Bruce Momjian
Josh Berkus wrote: > All, > > We are in Beta1, now, and it's May. Original goal for 9.0 was > June-July. We cannot be introducing major design revisions to HS/SR at > this date, or we won't ship until December. > > There are at least 10 other major features in 9.0, some of which are > more impo

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Josh Berkus
All, We are in Beta1, now, and it's May. Original goal for 9.0 was June-July. We cannot be introducing major design revisions to HS/SR at this date, or we won't ship until December. There are at least 10 other major features in 9.0, some of which are more important to some of our users than HS/

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Robert Haas
On Thu, May 6, 2010 at 2:47 PM, Josh Berkus wrote: > >> Now that I've realized what the real problem is with max_standby_delay >> (namely, that inactivity on the master can use up the delay), I think >> we should do what Tom originally suggested here.  It's not as good as >> a really working max_s

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Josh Berkus
> Now that I've realized what the real problem is with max_standby_delay > (namely, that inactivity on the master can use up the delay), I think > we should do what Tom originally suggested here. It's not as good as > a really working max_standby_delay, but we're not going to have that > for 9.0,

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Robert Haas
On Mon, May 3, 2010 at 11:37 AM, Tom Lane wrote: > I'm inclined to think that we should throw away all this logic and just > have the slave cancel competing queries if the replay process waits > more than max_standby_delay seconds to acquire a lock.  This is simple, > understandable, and behaves t

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Simon Riggs
On Thu, 2010-05-06 at 17:39 +0300, Heikki Linnakangas wrote: > Not the same plugin. A hook for stop/resume would need to be called > before and/or after each record, the one for conflict resolution would > need to be called at each conflict. Designing a good interface for a > plugin is hard, you n

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Heikki Linnakangas
Simon Riggs wrote: > On Thu, 2010-05-06 at 16:09 +0200, Dimitri Fontaine wrote: >> Simon Riggs writes: >>> It would be easier to implement a conflict resolution plugin that is >>> called when a conflict occurs, allowing users to have a customisable >>> mechanism. Again, I have no objection to that

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Heikki Linnakangas
Simon Riggs wrote: > On Thu, 2010-05-06 at 16:09 +0200, Dimitri Fontaine wrote: >> Simon Riggs writes: >>> It would be easier to implement a conflict resolution plugin that is >>> called when a conflict occurs, allowing users to have a customisable >>> mechanism. Again, I have no objection to that

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Simon Riggs
On Thu, 2010-05-06 at 16:09 +0200, Dimitri Fontaine wrote: > Simon Riggs writes: > > It would be easier to implement a conflict resolution plugin that is > > called when a conflict occurs, allowing users to have a customisable > > mechanism. Again, I have no objection to that proposal. > > To imp

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Dimitri Fontaine
Simon Riggs writes: > It would be easier to implement a conflict resolution plugin that is > called when a conflict occurs, allowing users to have a customisable > mechanism. Again, I have no objection to that proposal. To implement, if you say so, no doubt. To use, that means you need to install

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Greg Smith
Yeb Havinga wrote: Rob Wultsch wrote: I can not imagine setting this value to anything other than a bool and most of the time that bool would be -1. That's funny because when I was reading this thread, I was thinking the exact opposite: having max_standby_delay always set to 0 so I know the st

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Greg Smith
Heikki Linnakangas wrote: Robert Haas wrote: I am not convinced it will be unpredictable. The only caveats that I've seen so far are: - You need to run ntpd. - Queries will get cancelled like crazy if you're not using steaming replication. And also in situations where the master is i

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Simon Riggs
On Thu, 2010-05-06 at 13:46 +0200, Florian Pflug wrote: > On May 6, 2010, at 12:48 , Simon Riggs wrote: > > On Thu, 2010-05-06 at 11:36 +0200, Florian Pflug wrote: > >> If there was an additional SQL-callable function that returned the > >> backends the recovery process is currently waiting for, p

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Florian Pflug
On May 6, 2010, at 12:48 , Simon Riggs wrote: > On Thu, 2010-05-06 at 11:36 +0200, Florian Pflug wrote: >> If there was an additional SQL-callable function that returned the backends >> the recovery process is currently waiting for, plus one that reported that >> last timestamp seen in the WAL, t

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Simon Riggs
On Thu, 2010-05-06 at 11:36 +0200, Florian Pflug wrote: > If there was an additional SQL-callable function that returned the backends > the recovery process is currently waiting for, plus one that reported that > last timestamp seen in the WAL, than all those different cancellation > policies c

  1   2   >