Re: [HACKERS] max_standby_delay considered harmful

2010-05-14 Thread Robert Haas
On Thu, May 13, 2010 at 1:12 PM, Josh Berkus j...@agliodbs.com wrote: On 5/12/10 8:07 PM, Robert Haas wrote: I think that would be a good thing to check (it'll confirm whether this is the same bug), but I'm not convinced we should actually fix it that way.  Prior to 8.4, we handled a smart

Re: [HACKERS] max_standby_delay considered harmful

2010-05-14 Thread Josh Berkus
This would be OK as long as we document it well. We patched the shutdown the way we did specifically because Fujii thought it would be an easy fix; if it's complicated, we should revert it and document the issue for DBAs. I don't understand this comment. In other words, I'm saying that

Re: [HACKERS] max_standby_delay considered harmful

2010-05-14 Thread Robert Haas
On Fri, May 14, 2010 at 5:51 PM, Josh Berkus j...@agliodbs.com wrote: This would be OK as long as we document it well.  We patched the shutdown the way we did specifically because Fujii thought it would be an easy fix; if it's complicated, we should revert it and document the issue for DBAs.

Re: [HACKERS] max_standby_delay considered harmful

2010-05-13 Thread Josh Berkus
On 5/12/10 8:07 PM, Robert Haas wrote: I think that would be a good thing to check (it'll confirm whether this is the same bug), but I'm not convinced we should actually fix it that way. Prior to 8.4, we handled a smart shutdown during recovery at the conclusion of recovery, just prior to

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Simon Riggs
On Tue, 2010-05-11 at 14:01 +0900, Fujii Masao wrote: On Mon, May 10, 2010 at 3:27 PM, Simon Riggs si...@2ndquadrant.com wrote: I already explained that killing the startup process first is a bad idea for many reasons when shutdown was discussed. Can't remember who added the new standby

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Robert Haas
On Wed, May 12, 2010 at 2:50 AM, Simon Riggs si...@2ndquadrant.com wrote: On Tue, 2010-05-11 at 14:01 +0900, Fujii Masao wrote: On Mon, May 10, 2010 at 3:27 PM, Simon Riggs si...@2ndquadrant.com wrote: I already explained that killing the startup process first is a bad idea for many reasons

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Simon Riggs
On Wed, 2010-05-12 at 07:10 -0400, Robert Haas wrote: I'm not sure what to make of this. Sometimes not shutting down doesn't sound like a feature to me. It acts exactly the same in recovery as in normal running. It is not a special feature of recovery at all, bug or otherwise. You may think

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Greg Stark
On Wed, May 12, 2010 at 12:26 PM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, 2010-05-12 at 07:10 -0400, Robert Haas wrote: I'm not sure what to make of this.  Sometimes not shutting down doesn't sound like a feature to me. It acts exactly the same in recovery as in normal running. It

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Robert Haas
On Wed, May 12, 2010 at 7:26 AM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, 2010-05-12 at 07:10 -0400, Robert Haas wrote: I'm not sure what to make of this.  Sometimes not shutting down doesn't sound like a feature to me. It acts exactly the same in recovery as in normal running. It is

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Simon Riggs
On Wed, 2010-05-12 at 08:52 -0400, Robert Haas wrote: On Wed, May 12, 2010 at 7:26 AM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, 2010-05-12 at 07:10 -0400, Robert Haas wrote: I'm not sure what to make of this. Sometimes not shutting down doesn't sound like a feature to me. It

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Stefan Kaltenbrunner
Simon Riggs wrote: On Wed, 2010-05-12 at 08:52 -0400, Robert Haas wrote: On Wed, May 12, 2010 at 7:26 AM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, 2010-05-12 at 07:10 -0400, Robert Haas wrote: I'm not sure what to make of this. Sometimes not shutting down doesn't sound like a

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Simon Riggs
On Wed, 2010-05-12 at 16:03 +0200, Stefan Kaltenbrunner wrote: Simon Riggs wrote: On Wed, 2010-05-12 at 08:52 -0400, Robert Haas wrote: On Wed, May 12, 2010 at 7:26 AM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, 2010-05-12 at 07:10 -0400, Robert Haas wrote: I'm not sure what to

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Simon Riggs
On Wed, 2010-05-12 at 14:18 +0100, Simon Riggs wrote: On Wed, 2010-05-12 at 08:52 -0400, Robert Haas wrote: On Wed, May 12, 2010 at 7:26 AM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, 2010-05-12 at 07:10 -0400, Robert Haas wrote: I'm not sure what to make of this. Sometimes not

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Robert Haas
On Wed, May 12, 2010 at 11:28 AM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, 2010-05-12 at 14:18 +0100, Simon Riggs wrote: On Wed, 2010-05-12 at 08:52 -0400, Robert Haas wrote: On Wed, May 12, 2010 at 7:26 AM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, 2010-05-12 at 07:10 -0400,

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Simon Riggs
On Wed, 2010-05-12 at 12:04 -0400, Robert Haas wrote: Huh?? The evidence that this bug is linked with HS is that it occurs on a server running in HS mode, and not otherwise. As for whether the bug is code I committed, that's certainly possible, but keep in mind it didn't work at all before

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Greg Stark
On Wed, May 12, 2010 at 5:49 PM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, 2010-05-12 at 12:04 -0400, Robert Haas wrote: Huh?? The evidence that this bug is linked with HS is that it occurs on a server running in HS mode, and not otherwise.  As for whether the bug is code I committed,

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Simon Riggs
On Wed, 2010-05-12 at 18:05 +0100, Greg Stark wrote: I'm not sure who to blame for the shouting match over whose commit introduced the bug -- it doesn't seem like a relevant or useful thing to argue about, please both stop. I haven't blamed Robert's code, merely asked him to consider that it

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Joshua D. Drake
On Wed, 2010-05-12 at 17:49 +0100, Simon Riggs wrote: On Wed, 2010-05-12 at 12:04 -0400, Robert Haas wrote: Normal shutdown didn't work on a standby before HS was committed and it didn't work afterwards either. Use all the capitals you like but if you use poor arguments and combine that with

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Robert Haas
On Wed, May 12, 2010 at 1:21 PM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, 2010-05-12 at 18:05 +0100, Greg Stark wrote: I'm not sure who to blame for the shouting match over whose commit introduced the bug -- it doesn't seem like a relevant or useful thing to argue about, please both

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Stefan Kaltenbrunner
On 05/12/2010 05:28 PM, Simon Riggs wrote: On Wed, 2010-05-12 at 14:18 +0100, Simon Riggs wrote: On Wed, 2010-05-12 at 08:52 -0400, Robert Haas wrote: On Wed, May 12, 2010 at 7:26 AM, Simon Riggssi...@2ndquadrant.com wrote: On Wed, 2010-05-12 at 07:10 -0400, Robert Haas wrote: I'm not sure

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Simon Riggs
On Wed, 2010-05-12 at 21:10 +0200, Stefan Kaltenbrunner wrote: There is no evidence to link this behaviour with HS, as yet, and you should be considering the possibility the problem lies elsewhere, especially since it could be code you committed that is at fault. Well I'm not sure why

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Alvaro Herrera
Excerpts from Stefan Kaltenbrunner's message of mié may 12 15:10:28 -0400 2010: the startup process has the following backtrace: (gdb) bt #0 0x7fbe24cb2c83 in select () from /lib/libc.so.6 #1 0x006e811a in pg_usleep () #2 0x0048c333 in XLogPageRead () #3

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Simon Riggs
On Wed, 2010-05-12 at 15:36 -0400, Alvaro Herrera wrote: I just noticed that we have some code assigning the return value of time() to a pg_time_t variable. Is this supposed to work reliably? (xlog.c lines 9267ff) Code's used that for a while now. Checkpoints and everywhere. -- Simon Riggs

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Robert Haas
On Wed, May 12, 2010 at 3:36 PM, Alvaro Herrera alvhe...@alvh.no-ip.org wrote: Excerpts from Stefan Kaltenbrunner's message of mié may 12 15:10:28 -0400 2010: the startup process has the following backtrace: (gdb) bt #0  0x7fbe24cb2c83 in select () from /lib/libc.so.6 #1  

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Robert Haas
On Wed, May 12, 2010 at 3:51 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, May 12, 2010 at 3:36 PM, Alvaro Herrera alvhe...@alvh.no-ip.org wrote: Excerpts from Stefan Kaltenbrunner's message of mié may 12 15:10:28 -0400 2010: the startup process has the following backtrace: (gdb)

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Simon Riggs
On Wed, 2010-05-12 at 14:43 -0400, Robert Haas wrote: I thought that it would be a good idea for Simon to look at it because, on the surface, it APPEARS to have something to do with Hot Standby, since that's what Stefan was testing when he found it. He was also testing SR, yet you haven't

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Josh Berkus
Simon, Robert, He was also testing SR, yet you haven't breathed a word about that for some strange reason. It didn't APPEAR like it was HS at all, not from basic logic or from technical knowledge. So you'll have to forgive me if I don't leap into action when you say something is an HS problem

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Joshua D. Drake
On Wed, 2010-05-12 at 22:34 +0100, Simon Riggs wrote: On Wed, 2010-05-12 at 14:43 -0400, Robert Haas wrote: I thought that it would be a good idea for Simon to look at it because, on the surface, it APPEARS to have something to do with Hot Standby, since that's what Stefan was testing

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Robert Haas
On Wed, May 12, 2010 at 5:34 PM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, 2010-05-12 at 14:43 -0400, Robert Haas wrote: I thought that it would be a good idea for Simon to look at it because, on the surface, it APPEARS to have something to do with Hot Standby, since that's what

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Fujii Masao
On Thu, May 13, 2010 at 4:55 AM, Robert Haas robertmh...@gmail.com wrote: I am wondering if we are not correctly handling the case where we get a shutdown request while we are still in the PM_STARTUP state.  It looks like we might go ahead and switch to PM_RECOVERY and then

Re: [HACKERS] max_standby_delay considered harmful

2010-05-12 Thread Robert Haas
On Wed, May 12, 2010 at 10:46 PM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, May 13, 2010 at 4:55 AM, Robert Haas robertmh...@gmail.com wrote: I am wondering if we are not correctly handling the case where we get a shutdown request while we are still in the PM_STARTUP state.  It looks

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Simon Riggs
On Sun, 2010-05-09 at 20:56 -0400, Robert Haas wrote: Seems like it could take FOREVER on a busy system. Surely that's not OK. The fact that Hot Standby has to take exclusive locks that can't be released until WAL replay has progressed to a certain point seems like a fairly serious

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: On Sun, May 9, 2010 at 6:58 PM, Andres Freund and...@anarazel.de wrote: The difference is that in HS you have to wait for a moment where *no exclusive lock at all* exist, possibly without contending for any of them, while on the master you might not

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Heikki Linnakangas
Robert Haas wrote: On Sun, May 9, 2010 at 6:58 PM, Andres Freund and...@anarazel.de wrote: On Monday 10 May 2010 00:25:44 Florian Pflug wrote: On May 9, 2010, at 22:01 , Robert Haas wrote: On Sun, May 9, 2010 at 3:09 PM, Dimitri Fontaine dfonta...@hi-media.com wrote: Seems like it could take

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Heikki Linnakangas
Robert Haas wrote: On Thu, May 6, 2010 at 2:47 PM, Josh Berkus j...@agliodbs.com wrote: Now that I've realized what the real problem is with max_standby_delay (namely, that inactivity on the master can use up the delay), I think we should do what Tom originally suggested here. It's not as

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Florian Pflug
On May 10, 2010, at 11:43 , Heikki Linnakangas wrote: If you're not going to apply any more WAL records before shutdown, you could also just release all the AccessExclusiveLocks held by the startup process. Whatever the transaction was doing with the locked relation, if we're not going to

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Aidan Van Dyk
* Heikki Linnakangas heikki.linnakan...@enterprisedb.com [100510 06:03]: A problem with using the name max_standby_delay for Tom's suggestion is that it sounds like a hard limit, which it isn't. But if we name it something like: I'ld still rather an if your killing something, make sure you

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Robert Haas
On Mon, May 10, 2010 at 2:27 AM, Simon Riggs si...@2ndquadrant.com wrote: I already explained that killing the startup process first is a bad idea for many reasons when shutdown was discussed. Can't remember who added the new standby shutdown code recently, but it sounds like their design was

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Robert Haas
On Mon, May 10, 2010 at 6:03 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Yeah, I could live with that. A problem with using the name max_standby_delay for Tom's suggestion is that it sounds like a hard limit, which it isn't. But if we name it something like: # -1 = no

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Robert Haas
On Mon, May 10, 2010 at 6:13 AM, Florian Pflug f...@phlo.org wrote: On May 10, 2010, at 11:43 , Heikki Linnakangas wrote: If you're not going to apply any more WAL records before shutdown, you could also just release all the AccessExclusiveLocks held by the startup process. Whatever the

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Heikki Linnakangas
Florian Pflug wrote: On May 10, 2010, at 11:43 , Heikki Linnakangas wrote: If you're not going to apply any more WAL records before shutdown, you could also just release all the AccessExclusiveLocks held by the startup process. Whatever the transaction was doing with the locked relation, if

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Andres Freund
On Monday 10 May 2010 14:00:45 Heikki Linnakangas wrote: Florian Pflug wrote: On May 10, 2010, at 11:43 , Heikki Linnakangas wrote: If you're not going to apply any more WAL records before shutdown, you could also just release all the AccessExclusiveLocks held by the startup process.

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Bruce Momjian
Simon Riggs wrote: Bruce has used the word crippleware for the current state. Raising a problem and then blocking solutions is the best way I know to cripple a release. It should be clear that I've done my best to avoid this FYI, it was Robert Haas who used the term crippleware to describe a

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Bruce Momjian
Robert Haas wrote: Wultsch (who doesn't ever want to kill queries and therefore would be happy with a boolean), Yeb Havinga (who never wants to stall recovery and therefore would also be happy with a boolean), and Florian Pflug (who points out that pause/resume is actually a nontrivial

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Mike Rylander
On Mon, May 10, 2010 at 6:03 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Robert Haas wrote: On Thu, May 6, 2010 at 2:47 PM, Josh Berkus j...@agliodbs.com wrote: Now that I've realized what the real problem is with max_standby_delay (namely, that inactivity on the master

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Kevin Grittner
Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: Overall I would say opinion is about evenly split between: - leave it as-is - make it a Boolean - change it in some way but to something more expressive than a Boolean I think a boolean would limit the environments in which HS

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Stephen Frost
* Aidan Van Dyk (ai...@highrise.ca) wrote: * Heikki Linnakangas heikki.linnakan...@enterprisedb.com [100510 06:03]: A problem with using the name max_standby_delay for Tom's suggestion is that it sounds like a hard limit, which it isn't. But if we name it something like: I'ld still

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Bruce Momjian
Kevin Grittner wrote: Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: Overall I would say opinion is about evenly split between: - leave it as-is - make it a Boolean - change it in some way but to something more expressive than a Boolean I think a boolean would

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Greg Stark
On Mon, May 10, 2010 at 5:20 PM, Bruce Momjian br...@momjian.us wrote: You are right that we are much more flexible about changing administrative configuration parameters (like this one) than SQL. In the past, we even renamed logging parameters to be more consistent, and I think that proves

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Josh Berkus
1) Replace max_standby_delay with a boolean as per heikki's suggestion 2) Add an explicitly experimental option like max_standby_delay or recovery_conflict_timeout which is only effective if you've chosen recovery_conflict=pause recovery option and is explicitly documented as being

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Florian Pflug
On May 10, 2010, at 17:39 , Kevin Grittner wrote: Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: Overall I would say opinion is about evenly split between: - leave it as-is - make it a Boolean - change it in some way but to something more expressive than a Boolean I think

Re: [HACKERS] max_standby_delay considered harmful

2010-05-10 Thread Fujii Masao
On Mon, May 10, 2010 at 3:27 PM, Simon Riggs si...@2ndquadrant.com wrote: I already explained that killing the startup process first is a bad idea for many reasons when shutdown was discussed. Can't remember who added the new standby shutdown code recently, but it sounds like their design was

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Simon Riggs
On Sat, 2010-05-08 at 14:48 -0400, Bruce Momjian wrote: I think the consensus is to change this setting to a boolean. If you don't want to do it, I am sure we can find someone who will. You expect others to act on consensus and follow rules, yet ignore them yourself when it suits your

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Kevin Grittner
Bruce Momjian wrote: I think everyone agrees the current code is unusable, per Heikki's comment about a WAL file arriving after a period of no WAL activity I don't. I am curious to hear how many complaints we've had from alpha and beta testers of HS regarding this issue. I know that if

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: I like the proposal of a boolean because it provides only the minimal feature set of two cases that are both clearly needed and easily implementable. Whatever we do later is certain to provide a superset of those two cases. If we do something else (and

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Florian Pflug
On May 9, 2010, at 13:59 , Dimitri Fontaine wrote: Tom Lane t...@sss.pgh.pa.us writes: I like the proposal of a boolean because it provides only the minimal feature set of two cases that are both clearly needed and easily implementable. Whatever we do later is certain to provide a superset

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Greg Stark
On Sun, May 9, 2010 at 4:00 AM, Greg Smith g...@2ndquadrant.com wrote:  The use cases are covered as best they can be without better support from expected future SR features like heartbeats and XID loopback. For what it's worth I think deferring these extra complications is a very useful

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Robert Haas
On Sun, May 9, 2010 at 12:47 PM, Greg Stark gsst...@mit.edu wrote: On Sun, May 9, 2010 at 4:00 AM, Greg Smith g...@2ndquadrant.com wrote:  The use cases are covered as best they can be without better support from expected future SR features like heartbeats and XID loopback. For what it's

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Simon Riggs
On Sat, 2010-05-08 at 20:57 -0400, Tom Lane wrote: Andres Freund and...@anarazel.de writes: On Sunday 09 May 2010 01:34:18 Bruce Momjian wrote: I think everyone agrees the current code is unusable, per Heikki's comment about a WAL file arriving after a period of no WAL activity, and look

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Simon Riggs
On Sun, 2010-05-09 at 16:10 +0200, Florian Pflug wrote: Adding pause/resume seems to introduce some non-trivial locking problems, though. How would you handle a pause request if the recovery process currently held a lock? (We are only talking about AccessExclusiveLocks here. No LWlocks are

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Dimitri Fontaine
Florian Pflug f...@phlo.org writes: The only remaining option is to continue applying WAL until you reach a point where no locks are held, then pause. But from a user's POV that is nearly indistinguishable from simply setting hot_standby_conflict_winner to in the first place I think. Not

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Florian Pflug
On May 9, 2010, at 21:04 , Simon Riggs wrote: On Sun, 2010-05-09 at 16:10 +0200, Florian Pflug wrote: Adding pause/resume seems to introduce some non-trivial locking problems, though. How would you handle a pause request if the recovery process currently held a lock? (We are only talking

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Robert Haas
On Sun, May 9, 2010 at 3:09 PM, Dimitri Fontaine dfonta...@hi-media.com wrote: Florian Pflug f...@phlo.org writes: The only remaining option is to continue applying WAL until you reach a point where no locks are held, then pause. But from a user's POV that is nearly indistinguishable from

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Simon Riggs
On Sun, 2010-05-09 at 16:01 -0400, Robert Haas wrote: The fact that Hot Standby has to take exclusive locks that can't be released until WAL replay has progressed to a certain point seems like a fairly serious wart. LOL And people lecture me about design. -- Simon Riggs

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Florian Pflug
On May 9, 2010, at 22:01 , Robert Haas wrote: On Sun, May 9, 2010 at 3:09 PM, Dimitri Fontaine dfonta...@hi-media.com wrote: Florian Pflug f...@phlo.org writes: The only remaining option is to continue applying WAL until you reach a point where no locks are held, then pause. But from a

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Andres Freund
On Monday 10 May 2010 00:25:44 Florian Pflug wrote: On May 9, 2010, at 22:01 , Robert Haas wrote: On Sun, May 9, 2010 at 3:09 PM, Dimitri Fontaine dfonta...@hi-media.com wrote: Florian Pflug f...@phlo.org writes: The only remaining option is to continue applying WAL until you reach a

Re: [HACKERS] max_standby_delay considered harmful

2010-05-09 Thread Robert Haas
On Sun, May 9, 2010 at 6:58 PM, Andres Freund and...@anarazel.de wrote: On Monday 10 May 2010 00:25:44 Florian Pflug wrote: On May 9, 2010, at 22:01 , Robert Haas wrote: On Sun, May 9, 2010 at 3:09 PM, Dimitri Fontaine dfonta...@hi-media.com wrote: Florian Pflug f...@phlo.org writes: The

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Simon Riggs
On Thu, 2010-05-06 at 12:03 -0700, Josh Berkus wrote: So changing to a lock-based mechanism or designing a plugin interface are really not at all realistic at this date. I agree that changing to a lock-based mechanism is too much at this stage of development. However, putting in a plugin is

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Bruce Momjian
Simon Riggs wrote: With a plugin option, I would not object to option 1. If option 1 was taken, without plugins, it's clear that would be against consensus. Having said that, I'll confirm now there will not be an extreme reaction from me if option (1) was forced, nor do I counsel that

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Robert Haas
On Sat, May 8, 2010 at 2:48 PM, Bruce Momjian br...@momjian.us wrote: I think the concensus is to change this setting to a boolean.  If you don't want to do it, I am sure we can find someone who will. I still think we should revert to Tom's original proposal. -- Robert Haas EnterpriseDB:

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Bruce Momjian
Robert Haas wrote: On Sat, May 8, 2010 at 2:48 PM, Bruce Momjian br...@momjian.us wrote: I think the concensus is to change this setting to a boolean. ?If you don't want to do it, I am sure we can find someone who will. I still think we should revert to Tom's original proposal. And Tom's

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Robert Haas
On Sat, May 8, 2010 at 3:40 PM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: On Sat, May 8, 2010 at 2:48 PM, Bruce Momjian br...@momjian.us wrote: I think the concensus is to change this setting to a boolean. ?If you don't want to do it, I am sure we can find someone who will.

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Dimitri Fontaine
Bruce Momjian br...@momjian.us writes: I have no idea why an objection from you should mean more than an objection from anyone else in the community, and I have no idea what an extreme reaction means, or why anyone should care. Maybe I shouldn't say anything here. But clearly while you're spot

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Bruce Momjian
Robert Haas wrote: On Sat, May 8, 2010 at 3:40 PM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: On Sat, May 8, 2010 at 2:48 PM, Bruce Momjian br...@momjian.us wrote: I think the concensus is to change this setting to a boolean. ?If you don't want to do it, I am sure we can

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Greg Smith
Bruce Momjian wrote: I think the big question is whether this issue is significant enough that we should ignore our policy of no feature design during beta The idea that you're considering removal of a feature that we already have people using in beta and making plans around is a policy

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Bruce Momjian
Greg Smith wrote: Bruce Momjian wrote: I think the big question is whether this issue is significant enough that we should ignore our policy of no feature design during beta The idea that you're considering removal of a feature that we already have people using in beta and making plans

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Andres Freund
On Sunday 09 May 2010 01:34:18 Bruce Momjian wrote: I think everyone agrees the current code is unusable, per Heikki's comment about a WAL file arriving after a period of no WAL activity, and look how long it took our group to even understand why that fails so badly. To be honest its not

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Tom Lane
Andres Freund and...@anarazel.de writes: On Sunday 09 May 2010 01:34:18 Bruce Momjian wrote: I think everyone agrees the current code is unusable, per Heikki's comment about a WAL file arriving after a period of no WAL activity, and look how long it took our group to even understand why that

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Greg Smith
Tom Lane wrote: Taking out features after they've been in a release is very hard, even if we realize they're badly designed. It doesn't have to be; that's the problem the release often part takes care of. If a release has only been out a year, and a new one comes out saying oh, that

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Bruce Momjian
Greg Smith wrote: Tom Lane wrote: Taking out features after they've been in a release is very hard, even if we realize they're badly designed. It doesn't have to be; that's the problem the release often part takes care of. If a release has only been out a year, and a new one

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Robert Haas
On Sat, May 8, 2010 at 6:51 PM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: On Sat, May 8, 2010 at 3:40 PM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: On Sat, May 8, 2010 at 2:48 PM, Bruce Momjian br...@momjian.us wrote: I think the concensus is to change this

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Bruce Momjian
Robert Haas wrote: Clearly, anything is more feature-full than boolean --- the big question is whether Tom's proposal is significantly better than boolean that we should spend the time designing and implementing it, with the possibility it will all be changed in 9.1. I doubt it's likely

Re: [HACKERS] max_standby_delay considered harmful

2010-05-08 Thread Robert Haas
On Sun, May 9, 2010 at 12:08 AM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: Clearly, anything is more feature-full than boolean --- the big question is whether Tom's proposal is significantly better than boolean that we should spend the time designing and implementing it, with

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Rob Wultsch
On Wed, May 5, 2010 at 9:32 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, May 5, 2010 at 11:50 PM, Bruce Momjian br...@momjian.us wrote: If someone wants to suggest that HS is useless if max_standby_delay supports only boolean values, I am ready to suggest we remove HS as well and head

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Simon Riggs
On Thu, 2010-05-06 at 00:47 -0400, Robert Haas wrote: That just doesn't sound that bad to me, especially since the proposed alternative is: - Queries will get cancelled like crazy, period. Or else: - Replication can fall infinitely far behind and you can write a tedious and

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Simon Riggs
On Wed, 2010-05-05 at 23:15 -0700, Rob Wultsch wrote: I manage a bunch of different environments and I am pretty sure that in any of them if the db started seemingly randomly killing queries I would have application teams followed quickly by executives coming after me with torches and

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Dimitri Fontaine
Greg Smith g...@2ndquadrant.com writes: If you need a script that involves changing a server setting to do something, that translates into you can't do that for a typical DBA. The idea of a program regularly changing a server configuration setting on a production system is one you just can't

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Florian Pflug
On May 6, 2010, at 11:26 , Dimitri Fontaine wrote: Greg Smith g...@2ndquadrant.com writes: If you need a script that involves changing a server setting to do something, that translates into you can't do that for a typical DBA. The idea of a program regularly changing a server configuration

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Yeb Havinga
Rob Wultsch wrote: I manage a bunch of different environments and I am pretty sure that in any of them if the db started seemingly randomly killing queries I would have application teams followed quickly by executives coming after me with torches and pitchforks. I can not imagine setting this

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Robert Haas
On Thu, May 6, 2010 at 1:35 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Robert Haas wrote: On Wed, May 5, 2010 at 11:52 PM, Bruce Momjian br...@momjian.us wrote: I am afraid the current setting is tempting for users to enable, but will be so unpredictable that it will

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Andres Freund
Hi, On Thursday 06 May 2010 07:35:49 Heikki Linnakangas wrote: Robert Haas wrote: On Wed, May 5, 2010 at 11:52 PM, Bruce Momjian br...@momjian.us wrote: I am afraid the current setting is tempting for users to enable, but will be so unpredictable that it will tarnish the repuation of HS

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Simon Riggs
On Thu, 2010-05-06 at 11:36 +0200, Florian Pflug wrote: If there was an additional SQL-callable function that returned the backends the recovery process is currently waiting for, plus one that reported that last timestamp seen in the WAL, than all those different cancellation policies

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Florian Pflug
On May 6, 2010, at 12:48 , Simon Riggs wrote: On Thu, 2010-05-06 at 11:36 +0200, Florian Pflug wrote: If there was an additional SQL-callable function that returned the backends the recovery process is currently waiting for, plus one that reported that last timestamp seen in the WAL, than

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Simon Riggs
On Thu, 2010-05-06 at 13:46 +0200, Florian Pflug wrote: On May 6, 2010, at 12:48 , Simon Riggs wrote: On Thu, 2010-05-06 at 11:36 +0200, Florian Pflug wrote: If there was an additional SQL-callable function that returned the backends the recovery process is currently waiting for, plus one

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Greg Smith
Heikki Linnakangas wrote: Robert Haas wrote: I am not convinced it will be unpredictable. The only caveats that I've seen so far are: - You need to run ntpd. - Queries will get cancelled like crazy if you're not using steaming replication. And also in situations where the master is

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Greg Smith
Yeb Havinga wrote: Rob Wultsch wrote: I can not imagine setting this value to anything other than a bool and most of the time that bool would be -1. That's funny because when I was reading this thread, I was thinking the exact opposite: having max_standby_delay always set to 0 so I know the

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Dimitri Fontaine
Simon Riggs si...@2ndquadrant.com writes: It would be easier to implement a conflict resolution plugin that is called when a conflict occurs, allowing users to have a customisable mechanism. Again, I have no objection to that proposal. To implement, if you say so, no doubt. To use, that means

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Simon Riggs
On Thu, 2010-05-06 at 16:09 +0200, Dimitri Fontaine wrote: Simon Riggs si...@2ndquadrant.com writes: It would be easier to implement a conflict resolution plugin that is called when a conflict occurs, allowing users to have a customisable mechanism. Again, I have no objection to that

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Heikki Linnakangas
Simon Riggs wrote: On Thu, 2010-05-06 at 16:09 +0200, Dimitri Fontaine wrote: Simon Riggs si...@2ndquadrant.com writes: It would be easier to implement a conflict resolution plugin that is called when a conflict occurs, allowing users to have a customisable mechanism. Again, I have no

Re: [HACKERS] max_standby_delay considered harmful

2010-05-06 Thread Heikki Linnakangas
Simon Riggs wrote: On Thu, 2010-05-06 at 16:09 +0200, Dimitri Fontaine wrote: Simon Riggs si...@2ndquadrant.com writes: It would be easier to implement a conflict resolution plugin that is called when a conflict occurs, allowing users to have a customisable mechanism. Again, I have no

  1   2   >