Re: [HACKERS] Standalone synchronous master

2014-01-27 Thread Robert Haas
On Sun, Jan 26, 2014 at 10:56 PM, Rajeev rastogi rajeev.rast...@huawei.com wrote: On 01/25/2014, Josh Berkus wrote: ISTM the consensus is that we need better monitoring/administration interfaces so that people can script the behavior they want in external tools. Also, a new synchronous

Re: [HACKERS] Standalone synchronous master

2014-01-27 Thread Josh Berkus
On 01/26/2014 07:56 PM, Rajeev rastogi wrote: I shall rework to improve this patch. Below are the summarization of all discussions, which will be used as input for improving the patch: 1. Method of degrading the synchronous mode: a. Expose the configuration variable to a new

Re: [HACKERS] Standalone synchronous master

2014-01-26 Thread Hannu Krosing
On 01/24/2014 10:29 PM, Josh Berkus wrote: On 01/24/2014 12:47 PM, Heikki Linnakangas wrote: ISTM the consensus is that we need better monitoring/administration interfaces so that people can script the behavior they want in external tools. Also, a new synchronous apply replication mode would

Re: [HACKERS] Standalone synchronous master

2014-01-26 Thread Rajeev rastogi
On 01/25/2014, Josh Berkus wrote: ISTM the consensus is that we need better monitoring/administration interfaces so that people can script the behavior they want in external tools. Also, a new synchronous apply replication mode would be handy, but that'd be a whole different patch. We

Re: [HACKERS] Standalone synchronous master

2014-01-24 Thread Heikki Linnakangas
ISTM the consensus is that we need better monitoring/administration interfaces so that people can script the behavior they want in external tools. Also, a new synchronous apply replication mode would be handy, but that'd be a whole different patch. We don't have a patch on the table that we

Re: [HACKERS] Standalone synchronous master

2014-01-24 Thread Josh Berkus
On 01/24/2014 12:47 PM, Heikki Linnakangas wrote: ISTM the consensus is that we need better monitoring/administration interfaces so that people can script the behavior they want in external tools. Also, a new synchronous apply replication mode would be handy, but that'd be a whole different

Re: [HACKERS] Standalone synchronous master

2014-01-24 Thread Florian Pflug
On Jan24, 2014, at 22:29 , Josh Berkus j...@agliodbs.com wrote: On 01/24/2014 12:47 PM, Heikki Linnakangas wrote: ISTM the consensus is that we need better monitoring/administration interfaces so that people can script the behavior they want in external tools. Also, a new synchronous apply

Re: [HACKERS] Standalone synchronous master

2014-01-13 Thread Rajeev rastogi
On Sun, Jan 12, Amit Kapila wrote: How would that work? Would it be a tool in contrib? There already is a timeout, so if a tool checked more frequently than the timeout, it should work. The durable notification of the admin would happen in the tool, right? Well, you know what

Re: [HACKERS] Standalone synchronous master

2014-01-13 Thread Florian Pflug
On Jan12, 2014, at 04:18 , Josh Berkus j...@agliodbs.com wrote: Thing is, when we talk about auto-degrade, we need to determine things like Is the replica down or is this just a network blip? and take action according to the user's desired configuration. This is not something, realistically,

Re: [HACKERS] Standalone synchronous master

2014-01-13 Thread Hannu Krosing
On 01/13/2014 04:12 PM, Florian Pflug wrote: On Jan12, 2014, at 04:18 , Josh Berkus j...@agliodbs.com wrote: Thing is, when we talk about auto-degrade, we need to determine things like Is the replica down or is this just a network blip? and take action according to the user's desired

Re: [HACKERS] Standalone synchronous master

2014-01-13 Thread Joshua D. Drake
On 01/13/2014 10:12 AM, Hannu Krosing wrote: In other words, if we're going to have auto-degrade, the most intelligent place for it is in RepMgr/HandyRep/OmniPITR/pgPoolII/whatever. It's also the *easiest* place. Anything we do *inside* Postgres is going to have a really, really hard time

Re: [HACKERS] Standalone synchronous master

2014-01-13 Thread Jim Nasby
On 1/13/14, 12:21 PM, Joshua D. Drake wrote: On 01/13/2014 10:12 AM, Hannu Krosing wrote: In other words, if we're going to have auto-degrade, the most intelligent place for it is in RepMgr/HandyRep/OmniPITR/pgPoolII/whatever. It's also the *easiest* place. Anything we do *inside* Postgres

Re: [HACKERS] Standalone synchronous master

2014-01-13 Thread Andres Freund
On 2014-01-13 15:14:21 -0600, Jim Nasby wrote: On 1/13/14, 12:21 PM, Joshua D. Drake wrote: On 01/13/2014 10:12 AM, Hannu Krosing wrote: In other words, if we're going to have auto-degrade, the most intelligent place for it is in RepMgr/HandyRep/OmniPITR/pgPoolII/whatever. It's also the

Re: [HACKERS] Standalone synchronous master

2014-01-13 Thread Joshua D. Drake
On 01/13/2014 01:14 PM, Jim Nasby wrote: On 1/13/14, 12:21 PM, Joshua D. Drake wrote: On 01/13/2014 10:12 AM, Hannu Krosing wrote: In other words, if we're going to have auto-degrade, the most intelligent place for it is in RepMgr/HandyRep/OmniPITR/pgPoolII/whatever. It's also the

Re: [HACKERS] Standalone synchronous master

2014-01-13 Thread Florian Pflug
On Jan13, 2014, at 22:30 , Joshua D. Drake j...@commandprompt.com wrote: On 01/13/2014 01:14 PM, Jim Nasby wrote: On 1/13/14, 12:21 PM, Joshua D. Drake wrote: On 01/13/2014 10:12 AM, Hannu Krosing wrote: In other words, if we're going to have auto-degrade, the most intelligent place for

Re: [HACKERS] Standalone synchronous master

2014-01-12 Thread Florian Pflug
On Jan11, 2014, at 18:53 , Andres Freund and...@2ndquadrant.com wrote: On 2014-01-11 18:28:31 +0100, Florian Pflug wrote: Hm, I was about to suggest that you can set statement_timeout before doing COMMIT to limit the amount of time you want to wait for the standby to respond. Interestingly,

Re: [HACKERS] Standalone synchronous master

2014-01-12 Thread Josh Berkus
All, I'm leading this off with a review of the features offered by the actual patch submitted. My general discussion of the issues of Sync Degrade, which justifies my specific suggestions below, follows that. Rajeev, please be aware that other hackers may have different opinions than me on what

Re: [HACKERS] Standalone synchronous master

2014-01-12 Thread Stephen Frost
* Josh Berkus (j...@agliodbs.com) wrote: On 01/11/2014 08:52 PM, Amit Kapila wrote: It is better than async mode in a way such that in async mode it never waits for commits to be written to standby, but in this new mode it will do so unless it is not possible (all sync standby's goes down).

Re: [HACKERS] Standalone synchronous master

2014-01-12 Thread Kevin Grittner
Josh Berkus j...@agliodbs.com wrote: Add a new parameter : synchronous_standalone_master = on | off I think this is a TERRIBLE name for any such parameter.  What does synchronous standalone even mean?  A better name for the parameter would be auto_degrade_sync_replication or

Re: [HACKERS] Standalone synchronous master

2014-01-12 Thread Josh Berkus
On 01/12/2014 12:35 PM, Stephen Frost wrote: * Josh Berkus (j...@agliodbs.com) wrote: You don't want to handle all of those issues the same way as far as sync rep is concerned. For example, if the standby is restaring, you probably want to wait instead of degrading. *What*?! Certainly not

Re: [HACKERS] Standalone synchronous master

2014-01-12 Thread Stephen Frost
* Josh Berkus (j...@agliodbs.com) wrote: Well, then that becomes a reason to want better/more configurability. I agree with this- the challenge is figuring out what those options should be and how we should document them. In the couple of sync rep sites I admin, I *would* want to wait. That's

Re: [HACKERS] Standalone synchronous master

2014-01-12 Thread Amit Kapila
On 01/11/2014 08:52 PM, Amit Kapila wrote: It is better than async mode in a way such that in async mode it never waits for commits to be written to standby, but in this new mode it will do so unless it is not possible (all sync standby's goes down). Can't we use existing wal_sender_timeout,

Re: [HACKERS] Standalone synchronous master

2014-01-12 Thread Rajeev rastogi
On 13th January 2013, Josh Berkus Wrote: I'm leading this off with a review of the features offered by the actual patch submitted. My general discussion of the issues of Sync Degrade, which justifies my specific suggestions below, follows that. Rajeev, please be aware that other hackers may

Re: [HACKERS] Standalone synchronous master

2014-01-11 Thread Amit Kapila
On Fri, Jan 10, 2014 at 9:17 PM, Bruce Momjian br...@momjian.us wrote: On Fri, Jan 10, 2014 at 10:21:42AM +0530, Amit Kapila wrote: Here I think if user is aware from beginning that this is the behaviour, then may be the importance of message is not very high. What I want to say is that if we

Re: [HACKERS] Standalone synchronous master

2014-01-11 Thread Bruce Momjian
On Sat, Jan 11, 2014 at 01:29:23PM +0530, Amit Kapila wrote: Okay, this is one way of providing this new mode, others could be: a. Have just one GUC sync_standalone_mode = true|false and make this as PGC_POSTMASTER parameter, so that user is only allowed to set this mode at startup. Even if

Re: [HACKERS] Standalone synchronous master

2014-01-11 Thread Florian Pflug
On Jan11, 2014, at 01:48 , Joshua D. Drake j...@commandprompt.com wrote: On 01/10/2014 04:38 PM, Stephen Frost wrote: Adrian, * Adrian Klaver (adrian.kla...@gmail.com) wrote: On 01/10/2014 04:25 PM, Stephen Frost wrote: * Adrian Klaver (adrian.kla...@gmail.com) wrote: A) Change the

Re: [HACKERS] Standalone synchronous master

2014-01-11 Thread Tom Lane
Florian Pflug f...@phlo.org writes: Hm, I was about to suggest that you can set statement_timeout before doing COMMIT to limit the amount of time you want to wait for the standby to respond. Interestingly, however, that doesn't seem to work, which is weird, since AFAICS statement_timeout

Re: [HACKERS] Standalone synchronous master

2014-01-11 Thread Andres Freund
On 2014-01-11 18:28:31 +0100, Florian Pflug wrote: Hm, I was about to suggest that you can set statement_timeout before doing COMMIT to limit the amount of time you want to wait for the standby to respond. Interestingly, however, that doesn't seem to work, which is weird, since AFAICS

Re: [HACKERS] Standalone synchronous master

2014-01-11 Thread Mark Kirkwood
On 11/01/14 13:25, Stephen Frost wrote: Adrian, * Adrian Klaver (adrian.kla...@gmail.com) wrote: A) Change the existing sync mode to allow the master and standby fall out of sync should a standby fall over. I'm not sure that anyone is argueing for this.. B) Create a new mode that does

Re: [HACKERS] Standalone synchronous master

2014-01-11 Thread Tom Lane
Mark Kirkwood mark.kirkw...@catalyst.net.nz writes [slightly rearranged] My 2c is: The current behavior in CAP theorem speak is 'Cap' - i.e focused on consistency at the expense of availability. A reasonable thing to want. The other behavior being asked for is 'cAp' - i.e focused on

Re: [HACKERS] Standalone synchronous master

2014-01-11 Thread Josh Berkus
On 01/10/2014 06:27 PM, Bruce Momjian wrote: How would that work? Would it be a tool in contrib? There already is a timeout, so if a tool checked more frequently than the timeout, it should work. The durable notification of the admin would happen in the tool, right? Well, you know what

Re: [HACKERS] Standalone synchronous master

2014-01-11 Thread Amit Kapila
On Sun, Jan 12, 2014 at 8:48 AM, Josh Berkus j...@agliodbs.com wrote: On 01/10/2014 06:27 PM, Bruce Momjian wrote: How would that work? Would it be a tool in contrib? There already is a timeout, so if a tool checked more frequently than the timeout, it should work. The durable notification

Re: [HACKERS] Standalone synchronous master

2014-01-11 Thread Bruce Momjian
On Sat, Jan 11, 2014 at 07:18:02PM -0800, Josh Berkus wrote: In other words, if we're going to have auto-degrade, the most intelligent place for it is in RepMgr/HandyRep/OmniPITR/pgPoolII/whatever. It's also the *easiest* place. Anything we do *inside* Postgres is going to have a really,

Re: [HACKERS] Standalone synchronous master

2014-01-11 Thread Amit Kapila
On Sat, Jan 11, 2014 at 9:41 PM, Bruce Momjian br...@momjian.us wrote: On Sat, Jan 11, 2014 at 01:29:23PM +0530, Amit Kapila wrote: Okay, this is one way of providing this new mode, others could be: a. Have just one GUC sync_standalone_mode = true|false and make this as PGC_POSTMASTER

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Bruce Momjian
On Fri, Jan 10, 2014 at 10:21:42AM +0530, Amit Kapila wrote: On Thu, Jan 9, 2014 at 10:45 PM, Bruce Momjian br...@momjian.us wrote: I think RAID-1 is a very good comparison because it is successful technology and has similar issues. RAID-1 is like Postgres synchronous_standby_names mode

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Simon Riggs
On 10 January 2014 15:47, Bruce Momjian br...@momjian.us wrote: I know there was a desire to remove this TODO item, but I think we have brought up enough new issues that we can keep it to see if we can come up with a solution. Can you summarise what you think the new issues are? All I see is

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Hannu Krosing
On 01/10/2014 05:09 PM, Simon Riggs wrote: On 10 January 2014 15:47, Bruce Momjian br...@momjian.us wrote: I know there was a desire to remove this TODO item, but I think we have brought up enough new issues that we can keep it to see if we can come up with a solution. Can you summarise what

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Joshua D. Drake
On 01/10/2014 07:47 AM, Bruce Momjian wrote: I know there was a desire to remove this TODO item, but I think we have brought up enough new issues that we can keep it to see if we can come up with a solution. I have added a link to this discussion on the TODO item. I think we will need at

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Jim Nasby
On 1/10/14, 12:59 PM, Joshua D. Drake wrote: I know I am the one that instigated all of this so I want to be very clear on what I and what I am confident that my customers would expect. If a synchronous slave goes down, the master continues to operate. That is all. I don't care if it is

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Andres Freund
On 2014-01-10 10:59:23 -0800, Joshua D. Drake wrote: On 01/10/2014 07:47 AM, Bruce Momjian wrote: I know there was a desire to remove this TODO item, but I think we have brought up enough new issues that we can keep it to see if we can come up with a solution. I have added a link to this

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: On 2014-01-10 10:59:23 -0800, Joshua D. Drake wrote: If a synchronous slave goes down, the master continues to operate. That is all. I don't care if it is configurable (I would be fine with that). I don't care if it is not automatic (e.g;

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Andres Freund
On 2014-01-10 17:02:08 -0500, Stephen Frost wrote: * Andres Freund (and...@2ndquadrant.com) wrote: On 2014-01-10 10:59:23 -0800, Joshua D. Drake wrote: If a synchronous slave goes down, the master continues to operate. That is all. I don't care if it is configurable (I would be fine with

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Stephen Frost
Andres, On Friday, January 10, 2014, Andres Freund wrote: On 2014-01-10 17:02:08 -0500, Stephen Frost wrote: * Andres Freund (and...@2ndquadrant.com javascript:;) wrote: On 2014-01-10 10:59:23 -0800, Joshua D. Drake wrote: If a synchronous slave goes down, the master continues to

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Joshua D. Drake
On 01/10/2014 01:49 PM, Andres Freund wrote: I know I am the one that instigated all of this so I want to be very clear on what I and what I am confident that my customers would expect. If a synchronous slave goes down, the master continues to operate. That is all. I don't care if it is

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Andres Freund
On 2014-01-10 14:29:58 -0800, Joshua D. Drake wrote: db02 goes down. It doesn't matter why. It is down. db01 continues to accept orders, allow people to log into the website and we can still service accounts. The continuity of service continues. Why is that configuration advantageous over a

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Adrian Klaver
On 01/10/2014 02:33 PM, Andres Freund wrote: On 2014-01-10 14:29:58 -0800, Joshua D. Drake wrote: db02 goes down. It doesn't matter why. It is down. db01 continues to accept orders, allow people to log into the website and we can still service accounts. The continuity of service continues.

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Joshua D. Drake
On 01/10/2014 02:33 PM, Andres Freund wrote: On 2014-01-10 14:29:58 -0800, Joshua D. Drake wrote: db02 goes down. It doesn't matter why. It is down. db01 continues to accept orders, allow people to log into the website and we can still service accounts. The continuity of service continues.

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Andres Freund
Hi, On 2014-01-10 17:28:55 -0500, Stephen Frost wrote: Why do you know that you didn't loose any transactions? Trivial network hiccups, a restart of a standby, IO overload on the standby all can cause a very short interruptions in the walsender connection - leading to degradation. You

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Jeff Janes
On Fri, Jan 10, 2014 at 2:33 PM, Andres Freund and...@2ndquadrant.comwrote: On 2014-01-10 14:29:58 -0800, Joshua D. Drake wrote: db02 goes down. It doesn't matter why. It is down. db01 continues to accept orders, allow people to log into the website and we can still service accounts. The

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Andres Freund
On 2014-01-10 14:44:28 -0800, Joshua D. Drake wrote: On 01/10/2014 02:33 PM, Andres Freund wrote: On 2014-01-10 14:29:58 -0800, Joshua D. Drake wrote: db02 goes down. It doesn't matter why. It is down. db01 continues to accept orders, allow people to log into the website and we can still

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Stephen Frost
Greetings, On Friday, January 10, 2014, Andres Freund wrote: Hi, On 2014-01-10 17:28:55 -0500, Stephen Frost wrote: Why do you know that you didn't loose any transactions? Trivial network hiccups, a restart of a standby, IO overload on the standby all can cause a very short

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Joshua D. Drake
On 01/10/2014 02:47 PM, Andres Freund wrote: Really, the commits themselves are sent to the server at exactly the same speed independent of sync/async. The only thing that's delayed is the *notificiation* of the client that sent the commit. Not the commit itself. Which is irrelevant to the

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Joshua D. Drake
On 01/10/2014 02:57 PM, Stephen Frost wrote: Yes, if you have a BBU that memory is authoritative in most cases. But in that case the argument of having two disks is pretty much pointless, the SPOF suddenly became the battery + ram. If that is a concern then use multiple

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Hannu Krosing
On 01/10/2014 11:59 PM, Joshua D. Drake wrote: On 01/10/2014 02:57 PM, Stephen Frost wrote: Yes, if you have a BBU that memory is authoritative in most cases. But in that case the argument of having two disks is pretty much pointless, the SPOF suddenly became the battery + ram.

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Josh Berkus
On 01/10/2014 02:59 PM, Joshua D. Drake wrote: On 01/10/2014 02:47 PM, Andres Freund wrote: Really, the commits themselves are sent to the server at exactly the same speed independent of sync/async. The only thing that's delayed is the *notificiation* of the client that sent the commit.

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Josh Berkus
On 01/10/2014 01:49 PM, Andres Freund wrote: On 2014-01-10 10:59:23 -0800, Joshua D. Drake wrote: On 01/10/2014 07:47 AM, Bruce Momjian wrote: I know there was a desire to remove this TODO item, but I think we have brought up enough new issues that we can keep it to see if we can come up

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Joshua D. Drake
On 01/10/2014 03:17 PM, Josh Berkus wrote: Any continuous replication should not be a SPOF. The current behavior guarantees that a two node sync cluster is a SPOF. The proposed behavior removes that. Again, if that's your goal, then use async replication. I think I have gone about this the

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Adrian Klaver
On 01/10/2014 03:38 PM, Joshua D. Drake wrote: On 01/10/2014 03:17 PM, Josh Berkus wrote: Any continuous replication should not be a SPOF. The current behavior guarantees that a two node sync cluster is a SPOF. The proposed behavior removes that. Again, if that's your goal, then use async

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Stephen Frost
Adrian, * Adrian Klaver (adrian.kla...@gmail.com) wrote: A) Change the existing sync mode to allow the master and standby fall out of sync should a standby fall over. I'm not sure that anyone is argueing for this.. B) Create a new mode that does this without changing the existing sync mode.

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Adrian Klaver
On 01/10/2014 04:25 PM, Stephen Frost wrote: Adrian, * Adrian Klaver (adrian.kla...@gmail.com) wrote: A) Change the existing sync mode to allow the master and standby fall out of sync should a standby fall over. I'm not sure that anyone is argueing for this.. Looks like here, unless I am

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Stephen Frost
Adrian, * Adrian Klaver (adrian.kla...@gmail.com) wrote: On 01/10/2014 04:25 PM, Stephen Frost wrote: * Adrian Klaver (adrian.kla...@gmail.com) wrote: A) Change the existing sync mode to allow the master and standby fall out of sync should a standby fall over. I'm not sure that anyone is

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Joshua D. Drake
On 01/10/2014 04:38 PM, Stephen Frost wrote: Adrian, * Adrian Klaver (adrian.kla...@gmail.com) wrote: On 01/10/2014 04:25 PM, Stephen Frost wrote: * Adrian Klaver (adrian.kla...@gmail.com) wrote: A) Change the existing sync mode to allow the master and standby fall out of sync should a

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Jim Nasby
On 1/10/14, 6:19 PM, Adrian Klaver wrote: 1) Async. Runs at the speed of the master as it does not have to wait on the standby to signal a successful commit. There is some degree of offset between master and standby(s) due to latency. 2) Sync. Runs at the speed of the standby + latency

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Adrian Klaver
On 01/10/2014 04:48 PM, Joshua D. Drake wrote: On 01/10/2014 04:38 PM, Stephen Frost wrote: Adrian, * Adrian Klaver (adrian.kla...@gmail.com) wrote: On 01/10/2014 04:25 PM, Stephen Frost wrote: * Adrian Klaver (adrian.kla...@gmail.com) wrote: A) Change the existing sync mode to allow the

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Peter Eisentraut
On Wed, 2014-01-08 at 17:56 -0500, Stephen Frost wrote: * Andres Freund (and...@2ndquadrant.com) wrote: That's why you should configure a second standby as another (candidate) synchronous replica, also listed in synchronous_standby_names. Perhaps we should stress in the docs that this is,

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Bruce Momjian
On Fri, Jan 10, 2014 at 03:17:34PM -0800, Josh Berkus wrote: The purpose of sync rep is to know determinatively whether or not you have lost data when disaster strikes. If knowing for certain isn't important to you, then use async. BTW, people are using RAID1 as an analogy to 2-node sync

Re: [HACKERS] Standalone synchronous master

2014-01-10 Thread Bruce Momjian
On Fri, Jan 10, 2014 at 03:27:10PM -0800, Josh Berkus wrote: On 01/10/2014 01:49 PM, Andres Freund wrote: On 2014-01-10 10:59:23 -0800, Joshua D. Drake wrote: On 01/10/2014 07:47 AM, Bruce Momjian wrote: I know there was a desire to remove this TODO item, but I think we have brought

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread MauMau
From: Andres Freund and...@2ndquadrant.com On 2014-01-08 14:42:37 -0800, Joshua D. Drake wrote: If we have the following: db0-db1:down Using the model (as I understand it) that is being discussed we have increased our failure rate because the moment db1:down we also lose db0. The node db0

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread Hannu Krosing
On 01/09/2014 05:09 AM, Robert Treat wrote: On Wed, Jan 8, 2014 at 6:15 PM, Josh Berkus j...@agliodbs.com wrote: Stephen, I'm aware, my point was simply that we should state, up-front in 25.2.7.3 *and* where we document synchronous_standby_names, that it requires at least three servers to

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread Hannu Krosing
On 01/09/2014 12:05 AM, Stephen Frost wrote: * Andres Freund (and...@2ndquadrant.com) wrote: On 2014-01-08 17:56:37 -0500, Stephen Frost wrote: * Andres Freund (and...@2ndquadrant.com) wrote: That's why you should configure a second standby as another (candidate) synchronous replica, also

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread Hannu Krosing
On 01/08/2014 11:49 PM, Tom Lane wrote: Joshua D. Drake j...@commandprompt.com writes: On 01/08/2014 01:55 PM, Tom Lane wrote: Sync mode is about providing a guarantee that the data exists on more than one server *before* we tell the client it's committed. If you don't need that guarantee,

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread Hannu Krosing
On 01/09/2014 01:57 PM, MauMau wrote: From: Andres Freund and...@2ndquadrant.com On 2014-01-08 14:42:37 -0800, Joshua D. Drake wrote: If we have the following: db0-db1:down Using the model (as I understand it) that is being discussed we have increased our failure rate because the moment

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread Hannu Krosing
On 01/09/2014 02:01 AM, Jim Nasby wrote: On 1/8/14, 6:05 PM, Tom Lane wrote: Josh Berkusj...@agliodbs.com writes: On 01/08/2014 03:27 PM, Tom Lane wrote: What we lack, and should work on, is a way for sync mode to have M larger than one. AFAICS, right now we'll report commit as soon as

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread MauMau
From: Hannu Krosing ha...@2ndquadrant.com On 01/09/2014 01:57 PM, MauMau wrote: Let me ask a (probably) stupid question. How is the sync rep different from RAID-1? When I first saw sync rep, I expected that it would provide the same guarantees as RAID-1 in terms of durability (data is always

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread Hannu Krosing
On 01/09/2014 04:15 PM, MauMau wrote: From: Hannu Krosing ha...@2ndquadrant.com On 01/09/2014 01:57 PM, MauMau wrote: Let me ask a (probably) stupid question. How is the sync rep different from RAID-1? When I first saw sync rep, I expected that it would provide the same guarantees as

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread Bruce Momjian
On Thu, Jan 9, 2014 at 04:55:22PM +0100, Hannu Krosing wrote: On 01/09/2014 04:15 PM, MauMau wrote: From: Hannu Krosing ha...@2ndquadrant.com On 01/09/2014 01:57 PM, MauMau wrote: Let me ask a (probably) stupid question. How is the sync rep different from RAID-1? When I first saw

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread Jeff Janes
On Wed, Jan 8, 2014 at 3:00 PM, Josh Berkus j...@agliodbs.com wrote: On 01/08/2014 01:49 PM, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: If we really want auto-degrading sync rep, then we'd (at a minimum) need a way to determine *from the replica* whether or not it was in

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread Bruce Momjian
On Thu, Jan 9, 2014 at 09:36:47AM -0800, Jeff Janes wrote: Oh, right. Because the main reason for a sync replica degrading is that it's down. In which case it isn't going to record anything. This would still be useful for sync rep candidates, though, and I'll document why

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread Josh Berkus
Robert, I think the problem here is that we tend to have a limited view of the right way to use synch rep. If I have 5 nodes, and I set 1 synchronous and the other 3 asynchronous, I've set up a known successor in the event that the leader fails. In this scenario though, if the successor

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread Simon Riggs
On 8 January 2014 21:40, Tom Lane t...@sss.pgh.pa.us wrote: Kevin Grittner kgri...@ymail.com writes: I'm torn on whether we should cave to popular demand on this; but if we do, we sure need to be very clear in the documentation about what a successful return from a commit request means.

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread Jim Nasby
On 1/9/14, 9:01 AM, Hannu Krosing wrote: Yeah, and I think that the logging command that was suggested allows for that*if configured correctly*. *But* for relying on this, we would also need to make logging *synchronous*, which would probably not go down well with many people, as it makes

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread Amit Kapila
On Thu, Jan 9, 2014 at 10:45 PM, Bruce Momjian br...@momjian.us wrote: I think RAID-1 is a very good comparison because it is successful technology and has similar issues. RAID-1 is like Postgres synchronous_standby_names mode in the sense that the RAID-1 controller will not return success

Re: [HACKERS] Standalone synchronous master

2014-01-09 Thread Michael Paquier
On Fri, Jan 10, 2014 at 3:23 AM, Simon Riggs si...@2ndquadrant.com wrote: On 8 January 2014 21:40, Tom Lane t...@sss.pgh.pa.us wrote: Kevin Grittner kgri...@ymail.com writes: I'm torn on whether we should cave to popular demand on this; but if we do, we sure need to be very clear in the

Re: [HACKERS] Standalone synchronous master

2014-01-08 Thread Heikki Linnakangas
On 11/13/2013 03:09 PM, Rajeev rastogi wrote: This patch implements the following TODO item: Add a new eager synchronous mode that starts out synchronous but reverts to asynchronous after a failure timeout period This would require some type of command to be executed to alert administrators

Re: [HACKERS] Standalone synchronous master

2014-01-08 Thread Andres Freund
On 2014-01-08 11:07:48 +0200, Heikki Linnakangas wrote: I'm going to say right off the bat that I think the whole notion to automatically disable synchronous replication when the standby goes down is completely bonkers. If you don't need the strong guarantee that your transaction is safe in at

Re: [HACKERS] Standalone synchronous master

2014-01-08 Thread Simon Riggs
On 8 January 2014 09:07, Heikki Linnakangas hlinnakan...@vmware.com wrote: I'm going to say right off the bat that I think the whole notion to automatically disable synchronous replication when the standby goes down is completely bonkers. Agreed We had this discussion across 3 months and we

Re: [HACKERS] Standalone synchronous master

2014-01-08 Thread Bruce Momjian
On Wed, Jan 8, 2014 at 05:39:23PM +, Simon Riggs wrote: On 8 January 2014 09:07, Heikki Linnakangas hlinnakan...@vmware.com wrote: I'm going to say right off the bat that I think the whole notion to automatically disable synchronous replication when the standby goes down is

Re: [HACKERS] Standalone synchronous master

2014-01-08 Thread Hans-Jürgen Schönig
On Jan 8, 2014, at 9:27 PM, Bruce Momjian wrote: On Wed, Jan 8, 2014 at 05:39:23PM +, Simon Riggs wrote: On 8 January 2014 09:07, Heikki Linnakangas hlinnakan...@vmware.com wrote: I'm going to say right off the bat that I think the whole notion to automatically disable synchronous

Re: [HACKERS] Standalone synchronous master

2014-01-08 Thread Heikki Linnakangas
On 01/08/2014 10:27 PM, Bruce Momjian wrote: On Wed, Jan 8, 2014 at 05:39:23PM +, Simon Riggs wrote: On 8 January 2014 09:07, Heikki Linnakangas hlinnakan...@vmware.com wrote: I'm going to say right off the bat that I think the whole notion to automatically disable synchronous

Re: [HACKERS] Standalone synchronous master

2014-01-08 Thread Bruce Momjian
On Wed, Jan 8, 2014 at 10:46:51PM +0200, Heikki Linnakangas wrote: On 01/08/2014 10:27 PM, Bruce Momjian wrote: On Wed, Jan 8, 2014 at 05:39:23PM +, Simon Riggs wrote: On 8 January 2014 09:07, Heikki Linnakangas hlinnakan...@vmware.com wrote: I'm going to say right off the bat that I

Re: [HACKERS] Standalone synchronous master

2014-01-08 Thread Kevin Grittner
Bruce Momjian br...@momjian.us wrote: Heikki Linnakangas wrote: They want to have the cake and eat it too. But they're not actually getting that. What they actually get is extra latency when things work, with no gain in durability. They are getting guaranteed durability until they get a

Re: [HACKERS] Standalone synchronous master

2014-01-08 Thread Tom Lane
Kevin Grittner kgri...@ymail.com writes: I'm torn on whether we should cave to popular demand on this; but if we do, we sure need to be very clear in the documentation about what a successful return from a commit request means.  Sooner or later, Murphy's Law being what it is, if we do this

Re: [HACKERS] Standalone synchronous master

2014-01-08 Thread Andres Freund
On 2014-01-08 13:34:08 -0800, Kevin Grittner wrote: On the other hand, we keep getting people saying they want the database to make the promise of synchronous replication, and tell applications that it has been successful even when it hasn't been, as long as there's a line in the server log to

Re: [HACKERS] Standalone synchronous master

2014-01-08 Thread Josh Berkus
On 01/08/2014 12:27 PM, Bruce Momjian wrote: I am glad Heikki and Simon agree, but I don't. ;-) The way that I understand it is that you might want durability, but might not want to sacrifice availability. Phrased that way, it makes sense, and notifying the administrator seems the

Re: [HACKERS] Standalone synchronous master

2014-01-08 Thread Joshua D. Drake
On 01/08/2014 01:34 PM, Kevin Grittner wrote: I'm torn on whether we should cave to popular demand on this; but if we do, we sure need to be very clear in the documentation about what a successful return from a commit request means. Sooner or later, Murphy's Law being what it is, if we do

Re: [HACKERS] Standalone synchronous master

2014-01-08 Thread Heikki Linnakangas
On 01/08/2014 11:37 PM, Andres Freund wrote: On 2014-01-08 13:34:08 -0800, Kevin Grittner wrote: On the other hand, we keep getting people saying they want the database to make the promise of synchronous replication, and tell applications that it has been successful even when it hasn't been, as

Re: [HACKERS] Standalone synchronous master

2014-01-08 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: If we really want auto-degrading sync rep, then we'd (at a minimum) need a way to determine *from the replica* whether or not it was in degraded mode when the master died. What good do messages to the master log do you if the master no longer exists? How

Re: [HACKERS] Standalone synchronous master

2014-01-08 Thread Tom Lane
Joshua D. Drake j...@commandprompt.com writes: However, if the subscriber is down, the origin should NEVER wait. That is just silly behavior and makes synchronous replication pretty much useless. Machines go down, that is the nature of things. Yes, we should log and log loudly if the

Re: [HACKERS] Standalone synchronous master

2014-01-08 Thread Kevin Grittner
Andres Freund and...@2ndquadrant.com wrote: On 2014-01-08 13:34:08 -0800, Kevin Grittner wrote: On the other hand, we keep getting people saying they want the database to make the promise of synchronous replication, and tell applications that it has been successful even when it hasn't been,

Re: [HACKERS] Standalone synchronous master

2014-01-08 Thread Joshua D. Drake
On 01/08/2014 01:55 PM, Tom Lane wrote: Sync mode is about providing a guarantee that the data exists on more than one server *before* we tell the client it's committed. If you don't need that guarantee, you shouldn't be using sync mode. If you do need it, it's not clear to me why you'd

  1   2   >