Re: [HACKERS] Sync Rep for 2011CF1

2011-02-08 Thread Robert Haas
On Mon, Feb 7, 2011 at 5:16 PM, Simon Riggs si...@2ndquadrant.com wrote: On Mon, 2011-02-07 at 11:50 -0800, Josh Berkus wrote: I just spoke to my manager at EnterpriseDB and he cleared my schedule for the next two days to work on this.  So I'll go hack on this now. I haven't read the patch

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-08 Thread Robert Haas
On Mon, Feb 7, 2011 at 1:20 PM, Robert Haas robertmh...@gmail.com wrote: On Sat, Jan 15, 2011 at 4:40 PM, Simon Riggs si...@2ndquadrant.com wrote: Here's the latest patch for sync rep. Here is a rebased version of this patch which applies to head of the master branch.  I haven't tested it yet

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-08 Thread Magnus Hagander
On Tue, Feb 8, 2011 at 19:53, Robert Haas robertmh...@gmail.com wrote: On Mon, Feb 7, 2011 at 1:20 PM, Robert Haas robertmh...@gmail.com wrote: On Sat, Jan 15, 2011 at 4:40 PM, Simon Riggs si...@2ndquadrant.com wrote: Here's the latest patch for sync rep. Here is a rebased version of this

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-08 Thread Robert Haas
On Tue, Feb 8, 2011 at 2:34 PM, Magnus Hagander mag...@hagander.net wrote: I would usually not worry about the bandwidth, really, I'd be more worried about potentially increasing latency somewhere. The time to read and write the socket doesn't seem like it should be significant, unless the

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-08 Thread Joshua D. Drake
On Tue, 2011-02-08 at 13:53 -0500, Robert Haas wrote: That having been said, there is at least one part of this patch which looks to be in pretty good shape and seems independently useful regardless of what happens to the rest of it, and that is the code that sends replies from the standby

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-07 Thread Robert Haas
On Sun, Jan 30, 2011 at 11:44 AM, Robert Haas robertmh...@gmail.com wrote: Time's running short - do you have an updated patch? This patch hasn't been updated in more than three weeks. I assume this should now be marked Returned with Feedback, and we'll revisit synchronous replication for 9.2?

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-07 Thread Bruce Momjian
Robert Haas wrote: On Sun, Jan 30, 2011 at 11:44 AM, Robert Haas robertmh...@gmail.com wrote: Time's running short - do you have an updated patch? This patch hasn't been updated in more than three weeks. I assume this should now be marked Returned with Feedback, and we'll revisit

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-07 Thread Simon Riggs
On Mon, 2011-02-07 at 09:55 -0500, Robert Haas wrote: On Sun, Jan 30, 2011 at 11:44 AM, Robert Haas robertmh...@gmail.com wrote: Time's running short - do you have an updated patch? This patch hasn't been updated in more than three weeks. I assume this should now be marked Returned with

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-07 Thread Robert Haas
On Mon, Feb 7, 2011 at 11:33 AM, Simon Riggs si...@2ndquadrant.com wrote: On Mon, 2011-02-07 at 09:55 -0500, Robert Haas wrote: On Sun, Jan 30, 2011 at 11:44 AM, Robert Haas robertmh...@gmail.com wrote: Time's running short - do you have an updated patch? This patch hasn't been updated in

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-07 Thread Robert Haas
On Mon, Feb 7, 2011 at 12:28 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, Feb 7, 2011 at 11:33 AM, Simon Riggs si...@2ndquadrant.com wrote: On Mon, 2011-02-07 at 09:55 -0500, Robert Haas wrote: On Sun, Jan 30, 2011 at 11:44 AM, Robert Haas robertmh...@gmail.com wrote: Time's running

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-07 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: ... Well, the current CommitFest ends in one week, ... Really? I thought the idea for the last CF of a development cycle was that it kept going till we'd dealt with everything. Arbitrarily rejecting stuff we haven't dealt with doesn't seem fair.

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-07 Thread Simon Riggs
On Mon, 2011-02-07 at 12:39 -0500, Robert Haas wrote: I just spoke to my manager at EnterpriseDB and he cleared my schedule for the next two days to work on this. So I'll go hack on this now. I haven't read the patch yet so I don't know for sure how quite I'll be able to get up to speed on

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-07 Thread Robert Haas
On Mon, Feb 7, 2011 at 12:43 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: ... Well, the current CommitFest ends in one week, ... Really?  I thought the idea for the last CF of a development cycle was that it kept going till we'd dealt with everything.  

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-07 Thread Robert Haas
On Mon, Feb 7, 2011 at 12:59 PM, Simon Riggs si...@2ndquadrant.com wrote: On Mon, 2011-02-07 at 12:39 -0500, Robert Haas wrote: I just spoke to my manager at EnterpriseDB and he cleared my schedule for the next two days to work on this.  So I'll go hack on this now. I haven't read the patch

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-07 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: done in the time available is another thing entirely. I do NOT want to still be working on the items for this CommitFest in June - that's about when I'd like to be releasing beta3. Except that's not how we work here. You want to change that with

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-07 Thread Joshua D. Drake
On Mon, 2011-02-07 at 17:59 +, Simon Riggs wrote: On Mon, 2011-02-07 at 12:39 -0500, Robert Haas wrote: I just spoke to my manager at EnterpriseDB and he cleared my schedule for the next two days to work on this. So I'll go hack on this now. I haven't read the patch yet so I don't

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-07 Thread Robert Haas
On Mon, Feb 7, 2011 at 2:09 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Robert Haas robertmh...@gmail.com writes: done in the time available is another thing entirely.  I do NOT want to still be working on the items for this CommitFest in June - that's about when I'd like to be

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-07 Thread Josh Berkus
I just spoke to my manager at EnterpriseDB and he cleared my schedule for the next two days to work on this. So I'll go hack on this now. I haven't read the patch yet so I don't know for sure how quite I'll be able to get up to speed on it, so if someone who is more familiar with this code

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-07 Thread Robert Haas
On Mon, Feb 7, 2011 at 2:56 PM, Dave Page dp...@pgadmin.org wrote: On Mon, Feb 7, 2011 at 6:55 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, Feb 7, 2011 at 12:43 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: ... Well, the current CommitFest ends in one

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-07 Thread Dave Page
On Mon, Feb 7, 2011 at 6:55 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, Feb 7, 2011 at 12:43 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: ... Well, the current CommitFest ends in one week, ... Really?  I thought the idea for the last CF of a

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-07 Thread Kevin Grittner
Robert Haas robertmh...@gmail.com wrote: Dave Page dp...@pgadmin.org wrote: Robert Haas robertmh...@gmail.com wrote: Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: ... Well, the current CommitFest ends in one week, ... Really? I thought the idea for the last

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-07 Thread Dave Page
On Mon, Feb 7, 2011 at 8:59 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, Feb 7, 2011 at 2:56 PM, Dave Page dp...@pgadmin.org wrote: On Mon, Feb 7, 2011 at 6:55 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, Feb 7, 2011 at 12:43 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-07 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: I'm not trying to bypass compromising, and I don't know what makes you think otherwise. I am trying to ensure that the CommitFest wraps up Well, I'm too tired to allow myself posting such comments, sorry to have left the previous mail through. More

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-07 Thread Josh Berkus
On 2/7/11 11:41 AM, Robert Haas wrote: However, I don't want to prolong the CommitFest indefinitely in the face of patches that the authors are not actively working on or can't finish in the time available, or where there is no consensus that the proposed change is what we want. I believe

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-07 Thread Robert Haas
On Mon, Feb 7, 2011 at 3:06 PM, Dave Page dp...@pgadmin.org wrote: Rejecting stuff because we haven't gotten round to dealing with it in such a short period of time is a damn good way to limit the number of contributions we get. I don't believe we've agreed at any point that the last

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-07 Thread Joshua D. Drake
On Mon, 2011-02-07 at 12:24 -0800, Josh Berkus wrote: +1. I, for one, would vote against extending beta if Sync Rep isn't ready yet. There's plenty of other big features in 9.1, and Sync Rep will benefit from having additional development time given the number of major spec points we only

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-07 Thread Dave Page
On Mon, Feb 7, 2011 at 9:25 PM, Robert Haas robertmh...@gmail.com wrote: You're moving the bar.  It DOES say that the CommitFest will end on February 15th.  Now, if we want to have a discussion about changing that, let's have that discussion (perhaps on a thread where the subject has something

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-07 Thread Robert Haas
On Mon, Feb 7, 2011 at 3:14 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Robert Haas robertmh...@gmail.com writes: I'm not trying to bypass compromising, and I don't know what makes you think otherwise.  I am trying to ensure that the CommitFest wraps up Well, I'm too tired to allow

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-07 Thread Robert Haas
On Mon, Feb 7, 2011 at 3:34 PM, Dave Page dp...@pgadmin.org wrote: On Mon, Feb 7, 2011 at 9:25 PM, Robert Haas robertmh...@gmail.com wrote: You're moving the bar.  It DOES say that the CommitFest will end on February 15th.  Now, if we want to have a discussion about changing that, let's have

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-07 Thread Dave Page
On Mon, Feb 7, 2011 at 9:46 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, Feb 7, 2011 at 3:34 PM, Dave Page dp...@pgadmin.org wrote: On Mon, Feb 7, 2011 at 9:25 PM, Robert Haas robertmh...@gmail.com wrote: You're moving the bar.  It DOES say that the CommitFest will end on February

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-07 Thread Chris Browne
dp...@pgadmin.org (Dave Page) writes: On Mon, Feb 7, 2011 at 6:55 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, Feb 7, 2011 at 12:43 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: ... Well, the current CommitFest ends in one week, ... Really?  I

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-07 Thread Simon Riggs
On Mon, 2011-02-07 at 15:25 -0500, Robert Haas wrote: I would certainly appreciate it if everyone could at least credit me with acting in good faith. I think you are, if that helps. -- Simon Riggs http://www.2ndQuadrant.com/books/ PostgreSQL Development, 24x7 Support, Training

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-07 Thread Simon Riggs
On Mon, 2011-02-07 at 11:50 -0800, Josh Berkus wrote: I just spoke to my manager at EnterpriseDB and he cleared my schedule for the next two days to work on this. So I'll go hack on this now. I haven't read the patch yet so I don't know for sure how quite I'll be able to get up to speed

Re: [HACKERS] Sync Rep for 2011CF1

2011-01-30 Thread Robert Haas
On Sat, Jan 22, 2011 at 8:31 AM, Simon Riggs si...@2ndquadrant.com wrote: On Fri, 2011-01-21 at 13:32 -0500, Robert Haas wrote: One idea might be to wait both before and after commit.  If allow_standalone_primary is off, and a commit is attempted, we check whether there's a slave connected,

Re: [HACKERS] Sync Rep for 2011CF1

2011-01-22 Thread Simon Riggs
On Fri, 2011-01-21 at 13:32 -0500, Robert Haas wrote: One idea might be to wait both before and after commit. If allow_standalone_primary is off, and a commit is attempted, we check whether there's a slave connected, and if not, wait for one to connect. Then, we write and sync the commit

Re: [HACKERS] Sync Rep for 2011CF1

2011-01-21 Thread Heikki Linnakangas
(grr, I wrote this on Monday already, but just found it in my drafts folder, unsent) On 15.01.2011 23:40, Simon Riggs wrote: Here's the latest patch for sync rep. From here, I will be developing the patch further on public git repository towards commit. My expectation is that commit is at

Re: [HACKERS] Sync Rep for 2011CF1

2011-01-21 Thread Simon Riggs
On Fri, 2011-01-21 at 14:45 +0200, Heikki Linnakangas wrote: (grr, I wrote this on Monday already, but just found it in my drafts folder, unsent) No worries, thanks for commenting. Thanks! Some quick observations after first read-through: * The docs for synchronous_replication still claim

Re: [HACKERS] Sync Rep for 2011CF1

2011-01-21 Thread Robert Haas
On Fri, Jan 21, 2011 at 7:45 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: * it seems like overkill to not let clients to even connect when allow_standalone_primary=off and no synchronous standbys are available. What if you just want to run a read-only query? For what it's

Re: [HACKERS] Sync Rep for 2011CF1

2011-01-21 Thread Magnus Hagander
On Fri, Jan 21, 2011 at 14:24, Simon Riggs si...@2ndquadrant.com wrote: On Fri, 2011-01-21 at 14:45 +0200, Heikki Linnakangas wrote: * it seems like overkill to not let clients to even connect when allow_standalone_primary=off and no synchronous standbys are available. What if you just want to

Re: [HACKERS] Sync Rep for 2011CF1

2011-01-21 Thread Heikki Linnakangas
On 21.01.2011 15:24, Simon Riggs wrote: On Fri, 2011-01-21 at 14:45 +0200, Heikki Linnakangas wrote: * it seems like overkill to not let clients to even connect when allow_standalone_primary=off and no synchronous standbys are available. What if you just want to run a read-only query? That's

Re: [HACKERS] Sync Rep for 2011CF1

2011-01-21 Thread Robert Haas
On Fri, Jan 21, 2011 at 10:33 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: It's also possible that most of your transactions in fact do set synchronous_replication=off, and only a few actually do synchronous replication. It would be pretty bad to not allow connections in

Re: [HACKERS] Sync Rep for 2011CF1

2011-01-21 Thread Simon Riggs
On Fri, 2011-01-21 at 17:33 +0200, Heikki Linnakangas wrote: On 21.01.2011 15:24, Simon Riggs wrote: On Fri, 2011-01-21 at 14:45 +0200, Heikki Linnakangas wrote: * it seems like overkill to not let clients to even connect when allow_standalone_primary=off and no synchronous standbys are

Re: [HACKERS] Sync Rep for 2011CF1

2011-01-21 Thread Simon Riggs
On Fri, 2011-01-21 at 14:34 +0100, Magnus Hagander wrote: On Fri, Jan 21, 2011 at 14:24, Simon Riggs si...@2ndquadrant.com wrote: On Fri, 2011-01-21 at 14:45 +0200, Heikki Linnakangas wrote: * it seems like overkill to not let clients to even connect when allow_standalone_primary=off and no

Re: [HACKERS] Sync Rep for 2011CF1

2011-01-21 Thread Aidan Van Dyk
On Fri, Jan 21, 2011 at 11:59 AM, Simon Riggs si...@2ndquadrant.com wrote: We all think our own proposed options are the only reasonable thing, but that helps us not at all in moving forwards. I've put much time into delivering options many other people want, so there is a range of function.

Re: [HACKERS] Sync Rep for 2011CF1

2011-01-21 Thread Robert Haas
On Fri, Jan 21, 2011 at 12:23 PM, Aidan Van Dyk ai...@highrise.ca wrote: On Fri, Jan 21, 2011 at 11:59 AM, Simon Riggs si...@2ndquadrant.com wrote: We all think our own proposed options are the only reasonable thing, but that helps us not at all in moving forwards. I've put much time into

Re: [HACKERS] Sync Rep for 2011CF1

2011-01-21 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Fri, Jan 21, 2011 at 12:23 PM, Aidan Van Dyk ai...@highrise.ca wrote: When no sync slave is connected, yes, I want to stop things hard. What you're proposing is to fail things earlier than absolutely necessary (when they try to XLOG, rather than at

Re: [HACKERS] Sync Rep for 2011CF1

2011-01-21 Thread Aidan Van Dyk
On Fri, Jan 21, 2011 at 1:03 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Fri, Jan 21, 2011 at 12:23 PM, Aidan Van Dyk ai...@highrise.ca wrote: When no sync slave is connected, yes, I want to stop things hard. What you're proposing is to fail things

Re: [HACKERS] Sync Rep for 2011CF1

2011-01-21 Thread Robert Haas
On Fri, Jan 21, 2011 at 1:09 PM, Aidan Van Dyk ai...@highrise.ca wrote: On Fri, Jan 21, 2011 at 1:03 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Fri, Jan 21, 2011 at 12:23 PM, Aidan Van Dyk ai...@highrise.ca wrote: When no sync slave is connected, yes, I

Re: [HACKERS] Sync Rep for 2011CF1

2011-01-21 Thread Aidan Van Dyk
On Fri, Jan 21, 2011 at 1:32 PM, Robert Haas robertmh...@gmail.com wrote: Again, I'm trying to stop forward progress as soon as possible when a sync slave isn't replicating.  And I'ld like clients to fail with errors sooner (hopefully they get to the commit point) rather than accumulate the

Re: [HACKERS] Sync Rep for 2011CF1

2011-01-21 Thread Robert Haas
On Fri, Jan 21, 2011 at 1:59 PM, Aidan Van Dyk ai...@highrise.ca wrote: Yup.  And I'm OK with that.  In my case, it would be much better to have a few quick failures, which can complete automatically a few seconds later then to have a big buildup of transactions to re-verify by hand upon

Re: [HACKERS] Sync Rep for 2011CF1

2011-01-15 Thread Magnus Hagander
On Sat, Jan 15, 2011 at 22:40, Simon Riggs si...@2ndquadrant.com wrote: Here's the latest patch for sync rep. From here, I will be developing the patch further on public git repository towards commit. My expectation is that commit is at least 2 That's great. Just one tiny detail - which

Re: [HACKERS] Sync Rep Design

2011-01-04 Thread Josh Berkus
All, This is a pointless argument. Eventually, we will be implementing all possible sync rep configurations, because different users *need* different configurations. Some users care more about durability, some more about availability, and some more about response time. And you can't have all

Re: [HACKERS] Sync Rep Design

2011-01-04 Thread Josh Berkus
On 1/2/11 12:35 AM, Heikki Linnakangas wrote: Very likely. A synchronous standby can bring the master to a halt, while an asynchronous one is rather harmless. If I were a DBA, and the data wasn't very sensitive, I would liberally hand out async privileges to my colleagues to set up reporting

Re: [HACKERS] Sync Rep Design

2011-01-04 Thread Simon Riggs
On Tue, 2011-01-04 at 10:28 -0800, Josh Berkus wrote: The relevant question is: which configuration(s) can we have ready for the next CommitFest and alpha release? Based upon that series of conversations, I've reworked the design so that there is (currently) only a single standby offering sync

Re: [HACKERS] Sync Rep Design

2011-01-04 Thread Robert Haas
On Sun, Jan 2, 2011 at 4:19 PM, Simon Riggs si...@2ndquadrant.com wrote: On Sun, 2011-01-02 at 18:54 +0200, Heikki Linnakangas wrote: I believe we all agree that there's different use cases that require different setups. Both first-past-the-post and wait-for-all-to-ack have their uses.

Re: [HACKERS] Sync Rep Design

2011-01-04 Thread Joshua D. Drake
I'm not feeling well now, so I'm going to go to bed, not just to avoid snapping at people. Even given that short interlude, I see no problem about delivery. Cool! Thanks Simon. Feel better. Joshua D. Drake -- PostgreSQL.org Major Contributor Command Prompt, Inc:

Re: [HACKERS] Sync Rep Design

2011-01-04 Thread Stefan Kaltenbrunner
On 01/04/2011 07:51 PM, Simon Riggs wrote: On Tue, 2011-01-04 at 10:28 -0800, Josh Berkus wrote: The relevant question is: which configuration(s) can we have ready for the next CommitFest and alpha release? Based upon that series of conversations, I've reworked the design so that there is

Re: [HACKERS] Sync Rep Design

2011-01-04 Thread Robert Haas
On Tue, Jan 4, 2011 at 2:50 PM, Stefan Kaltenbrunner ste...@kaltenbrunner.cc wrote: On 01/04/2011 07:51 PM, Simon Riggs wrote: On Tue, 2011-01-04 at 10:28 -0800, Josh Berkus wrote: The relevant question is: which configuration(s) can we have ready for the next CommitFest and alpha release?

Re: [HACKERS] Sync Rep Design

2011-01-04 Thread Dimitri Fontaine
Josh Berkus j...@postgresql.org writes: So, again, I don't agree that a separate synchrep permission is useful, or warranted. +1 However, your arguments *do* make me backpedal on the issue of having a list of synch rep roles on the master. I can easily imagine a DBA needing to rapidly

Re: [HACKERS] Sync Rep Design

2011-01-04 Thread Josh Berkus
What about the HBA here? Hmmm. That's tempting; an synchronous HBA instead of a GUC? But that doesn't solve the problem of standby #6 is failing, I want to kick it off synch rep. I'd be opposed to having a GUC *and* an HBA. making DBAs set things independantly in two places just frustrates

Re: [HACKERS] Sync Rep Design

2011-01-04 Thread Dimitri Fontaine
Josh Berkus j...@postgresql.org writes: What about the HBA here? Hmmm. That's tempting; an synchronous HBA instead of a GUC? But that doesn't solve the problem of standby #6 is failing, I want to kick it off synch rep. I'd be opposed to having a GUC *and* an HBA. making DBAs set things

Re: [HACKERS] Sync Rep Design

2011-01-04 Thread Josh Berkus
I was just thinking that you could prepend a reject line at the right place in the file. Hmmm, that's worth thinking about. How do others feel about this? -- -- Josh Berkus PostgreSQL Experts Inc.

Re: [HACKERS] Sync Rep Design

2011-01-04 Thread Greg Smith
Simon Riggs wrote: Based upon that series of conversations, I've reworked the design so that there is (currently) only a single standby offering sync rep at any one time. Other standbys can request to be sync standbys but they only become the sync standby if the first one fails. Which was simple

Re: [HACKERS] Sync Rep Design

2011-01-03 Thread Simon Riggs
On Sun, 2011-01-02 at 20:53 +, Simon Riggs wrote: On Sun, 2011-01-02 at 12:13 -0800, MARK CALLAGHAN wrote: On Thu, Dec 30, 2010 at 9:02 PM, Robert Haas robertmh...@gmail.com wrote: reads MySQL documentation I see now that you've tried to design this feature in a way that is

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Heikki Linnakangas
On 02.01.2011 00:40, Josh Berkus wrote: On 1/1/11 5:59 AM, Stefan Kaltenbrunner wrote: well you keep saying that but to be honest I cannot really even see a usecase for me - what is only a random one of a set of servers is sync at any time and I don't really know which one. My usecases would al

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Stefan Kaltenbrunner
On 01/02/2011 09:35 AM, Heikki Linnakangas wrote: On 02.01.2011 00:40, Josh Berkus wrote: On 1/1/11 5:59 AM, Stefan Kaltenbrunner wrote: well you keep saying that but to be honest I cannot really even see a usecase for me - what is only a random one of a set of servers is sync at any time and

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Simon Riggs
On Sun, 2011-01-02 at 10:35 +0200, Heikki Linnakangas wrote: BTW, there's a bunch of replication related stuff that we should work to close, that are IMHO more important than synchronous replication. Like making the standby follow timeline changes, to make failovers smoother, and the facility

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Simon Riggs
On Sat, 2011-01-01 at 22:11 -0500, Aidan Van Dyk wrote: On Sat, Jan 1, 2011 at 6:08 PM, Simon Riggs si...@2ndquadrant.com wrote: On Sat, 2011-01-01 at 14:40 -0800, Josh Berkus wrote: Standby in general deals with the A,D,R triangle (Availability, Durability, Response time). Any one

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Simon Riggs
On Sun, 2011-01-02 at 10:35 +0200, Heikki Linnakangas wrote: Frankly, if Simon hadn't already submitted code, I'd be pushing for single-standby-only for 9.1, instead of any one. Yes, we are awfully late, but let's not panic. Yes, we're about a year late. Getting a simple feature like

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Hannu Krosing
On 2.1.2011 5:36, Robert Haas wrote: On Sat, Jan 1, 2011 at 6:54 AM, Simon Riggssi...@2ndquadrant.com wrote: Yes, working out the math is a good idea. Things are much clearer if we do that. Let's assume we have 98% availability on any single server. 1. Having one primary and 2 standbys,

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Simon Riggs
On Sat, 2011-01-01 at 23:36 -0500, Robert Haas wrote: On Sat, Jan 1, 2011 at 6:54 AM, Simon Riggs si...@2ndquadrant.com wrote: Yes, working out the math is a good idea. Things are much clearer if we do that. Let's assume we have 98% availability on any single server. 1. Having one

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Kevin Grittner
Simon Riggs wrote: On Sat, 2011-01-01 at 23:36 -0500, Robert Haas wrote: On Sat, Jan 1, 2011 at 6:54 AM, Simon Riggs wrote: Yes, working out the math is a good idea. Things are much clearer if we do that. Let's assume we have 98% availability on any single server. 1. Having one primary

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Simon Riggs
On Sun, 2011-01-02 at 08:08 -0600, Kevin Grittner wrote: I think you're talking about different metrics, and you're both right. With two servers configured in sync rep your chance of having an available (running) server is 99.9992%. The chance that you know that you have one that is totally

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Heikki Linnakangas
On 02.01.2011 15:41, Simon Riggs wrote: On Sat, 2011-01-01 at 23:36 -0500, Robert Haas wrote: On Sat, Jan 1, 2011 at 6:54 AM, Simon Riggssi...@2ndquadrant.com wrote: Yes, working out the math is a good idea. Things are much clearer if we do that. Let's assume we have 98% availability on any

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Kevin Grittner
Simon Riggs wrote: Do you agree that requiring response from 2 sync standbys, or locking up, gives us 94% server availability, but 99.9992% data durability? I'm not sure how to answer that. The calculations so far have been based around up-time and the probabilities that you have a

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Simon Riggs
On Sun, 2011-01-02 at 11:11 -0600, Kevin Grittner wrote: Simon Riggs wrote: Do you agree that requiring response from 2 sync standbys, or locking up, gives us 94% server availability, but 99.9992% data durability? I'm not sure how to answer that. The calculations so far have been

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread MARK CALLAGHAN
On Thu, Dec 30, 2010 at 9:02 PM, Robert Haas robertmh...@gmail.com wrote: reads MySQL documentation I see now that you've tried to design this feature in a way that is similar to MySQL's offering, which does have some value.  But it appears to me that the documentation you've written here is

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Jeff Janes
On Sat, Jan 1, 2011 at 8:35 AM, Simon Riggs si...@2ndquadrant.com wrote: On Sat, 2011-01-01 at 05:13 -0800, Jeff Janes wrote: On 12/31/10, Simon Riggs si...@2ndquadrant.com wrote: 2. sync does not guarantee that the updates to the standbys are in any way coordinated. You can run a query on

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Simon Riggs
On Sun, 2011-01-02 at 12:13 -0800, MARK CALLAGHAN wrote: On Thu, Dec 30, 2010 at 9:02 PM, Robert Haas robertmh...@gmail.com wrote: reads MySQL documentation I see now that you've tried to design this feature in a way that is similar to MySQL's offering, which does have some value. But it

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Simon Riggs
On Sun, 2011-01-02 at 18:54 +0200, Heikki Linnakangas wrote: I believe we all agree that there's different use cases that require different setups. Both first-past-the-post and wait-for-all-to-ack have their uses. Robert's analysis is that first-past-the-post doesn't actually improve the

Re: [HACKERS] Sync Rep Design

2011-01-02 Thread Hannu Krosing
On 2.1.2011 5:36, Robert Haas wrote: On Sat, Jan 1, 2011 at 6:54 AM, Simon Riggssi...@2ndquadrant.com wrote: Yes, working out the math is a good idea. Things are much clearer if we do that. Let's assume we have 98% availability on any single server. 1. Having one primary and 2 standbys,

Re: [HACKERS] Sync Rep Design

2011-01-01 Thread Simon Riggs
On Fri, 2010-12-31 at 22:18 +0100, Hannu Krosing wrote: On 31.12.2010 13:40, Heikki Linnakangas wrote: Sounds good. I still don't like the synchronous_standbys='' and synchronous_replication=on combination, though. IMHO that still amounts to letting the standby control the behavior

Re: [HACKERS] Sync Rep Design

2011-01-01 Thread Jeff Janes
On 12/31/10, Simon Riggs si...@2ndquadrant.com wrote: On Fri, 2010-12-31 at 09:27 +0100, Stefan Kaltenbrunner wrote: Maybe it has been discussed but I still don't see way it makes any sense. If I declare a standby a sync standby I better want it sync - not maybe sync. consider the case of a 1

Re: [HACKERS] Sync Rep Design

2011-01-01 Thread Stefan Kaltenbrunner
On 12/31/2010 08:15 PM, Simon Riggs wrote: On Fri, 2010-12-31 at 14:40 +0200, Heikki Linnakangas wrote: On 31.12.2010 13:48, Simon Riggs wrote: I see significant real-world issues with configuring replication using multiple named servers, as described in the link above: All of these points

Re: [HACKERS] Sync Rep Design

2011-01-01 Thread Stefan Kaltenbrunner
On 01/01/2011 02:13 PM, Jeff Janes wrote: On 12/31/10, Simon Riggssi...@2ndquadrant.com wrote: On Fri, 2010-12-31 at 09:27 +0100, Stefan Kaltenbrunner wrote: Maybe it has been discussed but I still don't see way it makes any sense. If I declare a standby a sync standby I better want it sync

Re: [HACKERS] Sync Rep Design

2011-01-01 Thread Robert Haas
On Sat, Jan 1, 2011 at 9:03 AM, Stefan Kaltenbrunner ste...@kaltenbrunner.cc wrote: that is exactly my point - if have no guarantee that your SYNC standby is actually sync there is no use for it being used in business cases that require sync replication. If we cannot support that usecase I

Re: [HACKERS] Sync Rep Design

2011-01-01 Thread Stefan Kaltenbrunner
On 01/01/2011 03:15 PM, Robert Haas wrote: On Sat, Jan 1, 2011 at 9:03 AM, Stefan Kaltenbrunner ste...@kaltenbrunner.cc wrote: that is exactly my point - if have no guarantee that your SYNC standby is actually sync there is no use for it being used in business cases that require sync

Re: [HACKERS] Sync Rep Design

2011-01-01 Thread Dimitri Fontaine
Stefan Kaltenbrunner ste...@kaltenbrunner.cc writes: well you keep saying that but to be honest I cannot really even see a usecase for me - what is only a random one of a set of servers is sync at any time and I don't really know which one. It looks easy enough to get to know which one it is.

Re: [HACKERS] Sync Rep Design

2011-01-01 Thread Simon Riggs
On Sat, 2011-01-01 at 05:13 -0800, Jeff Janes wrote: On 12/31/10, Simon Riggs si...@2ndquadrant.com wrote: On Fri, 2010-12-31 at 09:27 +0100, Stefan Kaltenbrunner wrote: Maybe it has been discussed but I still don't see way it makes any sense. If I declare a standby a sync standby I

Re: [HACKERS] Sync Rep Design

2011-01-01 Thread Stefan Kaltenbrunner
On 01/01/2011 05:28 PM, Dimitri Fontaine wrote: Stefan Kaltenbrunnerste...@kaltenbrunner.cc writes: well you keep saying that but to be honest I cannot really even see a usecase for me - what is only a random one of a set of servers is sync at any time and I don't really know which one. It

Re: [HACKERS] Sync Rep Design

2011-01-01 Thread Simon Riggs
On Sat, 2011-01-01 at 16:12 +0100, Stefan Kaltenbrunner wrote: I still would like to get a statement on why simon thinks that the design heikki and others have proposed I've explained in huge detail why I think what I think, nor avoided any technical issue. It appears to me there has been

Re: [HACKERS] Sync Rep Design

2011-01-01 Thread Simon Riggs
On Sat, 2011-01-01 at 17:37 +0100, Stefan Kaltenbrunner wrote: On 01/01/2011 05:28 PM, Dimitri Fontaine wrote: Stefan Kaltenbrunnerste...@kaltenbrunner.cc writes: well you keep saying that but to be honest I cannot really even see a usecase for me - what is only a random one of a set of

Re: [HACKERS] Sync Rep Design

2011-01-01 Thread Stefan Kaltenbrunner
On 01/01/2011 05:55 PM, Simon Riggs wrote: On Sat, 2011-01-01 at 16:12 +0100, Stefan Kaltenbrunner wrote: I still would like to get a statement on why simon thinks that the design heikki and others have proposed I've explained in huge detail why I think what I think, nor avoided any

Re: [HACKERS] Sync Rep Design

2011-01-01 Thread Simon Riggs
On Sat, 2011-01-01 at 18:13 +0100, Stefan Kaltenbrunner wrote: On 01/01/2011 05:55 PM, Simon Riggs wrote: It appears to me there has been substantial confusion over alternatives, because of a misunderstanding about how synchronisation works. Requiring confirmation that standbys are in

Re: [HACKERS] Sync Rep Design

2011-01-01 Thread Simon Riggs
On Sat, 2011-01-01 at 17:28 +0100, Dimitri Fontaine wrote: something visible through a system view for users? This as been asked for before and I was thinking there was a consensus on this. Yes, it will be there. -- Simon Riggs http://www.2ndQuadrant.com/books/ PostgreSQL

Re: [HACKERS] Sync Rep Design

2011-01-01 Thread Stefan Kaltenbrunner
On 01/01/2011 06:29 PM, Simon Riggs wrote: On Sat, 2011-01-01 at 18:13 +0100, Stefan Kaltenbrunner wrote: On 01/01/2011 05:55 PM, Simon Riggs wrote: It appears to me there has been substantial confusion over alternatives, because of a misunderstanding about how synchronisation works.

Re: [HACKERS] Sync Rep Design

2011-01-01 Thread Simon Riggs
On Sat, 2011-01-01 at 18:49 +0100, Stefan Kaltenbrunner wrote: hmm maybe my surviving standbys(the case I'm wondering about is whole datacenter failures which might take out more than just the master) was not clear - consider three boxes, one master and two standby and semisync

Re: [HACKERS] Sync Rep Design

2011-01-01 Thread Heikki Linnakangas
On 01.01.2011 19:03, Simon Riggs wrote: On Sat, 2011-01-01 at 17:37 +0100, Stefan Kaltenbrunner wrote: On 01/01/2011 05:28 PM, Dimitri Fontaine wrote: Stefan Kaltenbrunnerste...@kaltenbrunner.cc writes: well you keep saying that but to be honest I cannot really even see a usecase for me -

Re: [HACKERS] Sync Rep Design

2011-01-01 Thread Heikki Linnakangas
On 31.12.2010 23:18, Hannu Krosing wrote: On 31.12.2010 13:40, Heikki Linnakangas wrote: That thread makes no mention of how to specify which standbys are synchronous and which are not. The simplest way would be to have separate database users for sync and async standbys ? That would allow

Re: [HACKERS] Sync Rep Design

2011-01-01 Thread Josh Berkus
On 1/1/11 5:59 AM, Stefan Kaltenbrunner wrote: well you keep saying that but to be honest I cannot really even see a usecase for me - what is only a random one of a set of servers is sync at any time and I don't really know which one. My usecases would al involved 2 sync standbys and 1 or more

<    1   2   3   4   5   6   7   >