Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Markus Wanner
On 09/23/2010 10:09 PM, Robert Haas wrote: I think maybe you missed Tom's point, or else you just didn't respond to it. If the master is wedged because it is waiting for a standby, then you cannot commit transactions on the master. Therefore you cannot update the system catalog which you

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Markus Wanner
Simon, On 09/24/2010 12:11 AM, Simon Riggs wrote: As I keep pointing out, waiting for an acknowledgement from something that isn't there might just take a while. The only guarantee that provides is that you will wait a long time. Is my data more safe? No. By now I agree that waiting for

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Heikki Linnakangas
On 24/09/10 01:11, Simon Riggs wrote: But that's not what I call synchronous replication, it doesn't give you the guarantees that textbook synchronous replication does. Which textbook? I was using that word metaphorically, but for example: Wikipedia

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Heikki Linnakangas
On 24/09/10 01:11, Simon Riggs wrote: On Thu, 2010-09-23 at 20:42 +0300, Heikki Linnakangas wrote: If you want the behavior where the master doesn't acknowledge a commit to the client until the standby (or all standbys, or one of them etc.) acknowledges it, even if the standby is not currently

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Simon Riggs
On Thu, 2010-09-23 at 14:26 +0200, Csaba Nagy wrote: Unfortunately it was quite long time ago we last tried, and I don't remember exactly what was bottlenecked. Our application is quite write-intensive, the ratio of writes to reads which actually reaches the disk is about 50-200% (according to

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Simon Riggs
On Thu, 2010-09-23 at 16:09 -0400, Robert Haas wrote: On Thu, Sep 23, 2010 at 3:46 PM, Simon Riggs si...@2ndquadrant.com wrote: Well, its not at all hard to see how that could be configured, because I already proposed a simple way of implementing parameters that doesn't suffer from those

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: Oh, I thought part of the objective here was to try to centralize that stuff. If we're assuming that slaves will still have local replication configuration files, then I think we should just add any necessary info to those files and drop this entire

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Dimitri Fontaine
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: If you want the behavior where the master doesn't acknowledge a commit to the client until the standby (or all standbys, or one of them etc.) acknowledges it, even if the standby is not currently connected, the master needs to know

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Simon Riggs
On Fri, 2010-09-24 at 11:08 +0300, Heikki Linnakangas wrote: On 24/09/10 01:11, Simon Riggs wrote: But that's not what I call synchronous replication, it doesn't give you the guarantees that textbook synchronous replication does. Which textbook? I was using that word metaphorically,

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Simon Riggs
On Fri, 2010-09-24 at 11:43 +0300, Heikki Linnakangas wrote: To get zero data loss *and* continuous availability, you need two standbys offering sync rep and reply-to-first behaviour. Yes, that is a good point. I'm starting to understand what your proposal was all about. It makes sense

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: I think maybe you missed Tom's point, or else you just didn't respond to it. If the master is wedged because it is waiting for a standby, then you cannot commit transactions on the master. Therefore you cannot update the system catalog which you must

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Heikki Linnakangas
On 24/09/10 13:57, Simon Riggs wrote: If you want high availability you need N+1 redundancy. If you want a standby server that is N=1. If you want a highly available standby configuration then N+1 = 2. Yep. Synchronous replication with one standby gives you zero data loss. When you add a 2nd

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Simon Riggs
On Fri, 2010-09-24 at 14:12 +0300, Heikki Linnakangas wrote: What I'm saying is that in a two standby situation, if you're willing to continue operation as usual in the master even if the standby is down, you're not doing synchronous replication. Oracle and I disagree with you on that point,

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Robert Haas
On Fri, Sep 24, 2010 at 6:37 AM, Simon Riggs si...@2ndquadrant.com wrote: Earlier you argued that centralizing parameters would make this nice and simple. Now you're pointing out that we aren't centralizing this at all, and it won't be simple. We'll have to have a standby.conf set up that is

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Aidan Van Dyk
On Fri, Sep 24, 2010 at 7:47 AM, Simon Riggs si...@2ndquadrant.com wrote: On Fri, 2010-09-24 at 14:12 +0300, Heikki Linnakangas wrote: What I'm saying is that in a two standby situation, if you're willing to continue operation as usual in the master even if the standby is down, you're not

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Heikki Linnakangas
On 24/09/10 14:47, Simon Riggs wrote: On Fri, 2010-09-24 at 14:12 +0300, Heikki Linnakangas wrote: What I'm saying is that in a two standby situation, if you're willing to continue operation as usual in the master even if the standby is down, you're not doing synchronous replication. Oracle

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Dimitri Fontaine
Hi, Defending my ideas as not to be put in the bag you're wanting to put away. We have more than 2 proposals lying around here. I'm one of the guys with a proposal and no code, but still trying to be clear. Robert Haas robertmh...@gmail.com writes: The reason I think that we should centralize

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Simon Riggs
On Fri, 2010-09-24 at 16:01 +0200, Dimitri Fontaine wrote: I'd like that we now follow Josh Berkus (and some other) advice now, and start a new thread to decide what we mean by synchronous replication, what kind of normal behaviour we want and what responses to errors we expect to be able to

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Heikki Linnakangas
On 24/09/10 17:13, Simon Riggs wrote: On Fri, 2010-09-24 at 16:01 +0200, Dimitri Fontaine wrote: I'd like that we now follow Josh Berkus (and some other) advice now, and start a new thread to decide what we mean by synchronous replication, what kind of normal behaviour we want and what

Re: [HACKERS] Configuring synchronous replication

2010-09-24 Thread Robert Haas
On Fri, Sep 24, 2010 at 10:01 AM, Dimitri Fontaine dfonta...@hi-media.com wrote:  Configuring whether the master will retain WAL for a disconnected slave on the slave is outright byzantine. Again, I can't remember having proposed such a thing. No one has, but I keep hearing we don't need the

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Simon Riggs
On Mon, 2010-09-20 at 18:24 -0400, Robert Haas wrote: I feel like that's really nice and simple. There are already 5 separate places to configure to make streaming rep work in a 2 node cluster (master.pg_hba.conf, master.postgresql.conf, standby.postgresql.conf, standby.recovery.conf, password

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Heikki Linnakangas
On 23/09/10 11:34, Csaba Nagy wrote: In the meantime our DBs are not able to keep in sync via WAL replication, that would need some kind of parallel WAL restore on the slave I guess, or I'm not able to configure it properly - in any case now we use slony which is working. It would be

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Csaba Nagy
Hi all, Some time ago I was also interested in this feature, and that time I also thought about complete setup possibility via postgres connections, meaning the transfer of the files and all configuration/slave registration to be done through normal backend connections. In the meantime our DBs

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Csaba Nagy
On Thu, 2010-09-23 at 12:02 +0300, Heikki Linnakangas wrote: On 23/09/10 11:34, Csaba Nagy wrote: In the meantime our DBs are not able to keep in sync via WAL replication, that would need some kind of parallel WAL restore on the slave I guess, or I'm not able to configure it properly - in

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Heikki Linnakangas
On 23/09/10 15:26, Csaba Nagy wrote: Unfortunately it was quite long time ago we last tried, and I don't remember exactly what was bottlenecked. Our application is quite write-intensive, the ratio of writes to reads which actually reaches the disk is about 50-200% (according to the disk stats -

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Simon Riggs
On Wed, 2010-09-22 at 13:00 -0400, Robert Haas wrote: I think it should be a separate config file, and I think it should be a config file that can be edited using DDL commands as you propose. But it CAN'T be a system catalog, because, among other problems, that rules out cascading slaves,

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: ISTM that we can have a system catalog and still have cascading slaves. If we administer the catalog via the master, why can't we administer all slaves, however they cascade, via the master too? What other problems are there that mean we *must* have a

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Simon Riggs
On Thu, 2010-09-23 at 11:43 -0400, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: ISTM that we can have a system catalog and still have cascading slaves. If we administer the catalog via the master, why can't we administer all slaves, however they cascade, via the master too?

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Robert Haas
On Thu, Sep 23, 2010 at 11:32 AM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, 2010-09-22 at 13:00 -0400, Robert Haas wrote: I think it should be a separate config file, and I think it should be a config file that can be edited using DDL commands as you propose. But it CAN'T be a system

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On Thu, 2010-09-23 at 11:43 -0400, Tom Lane wrote: Well, for one thing, how do you add a new slave? If its configuration comes from a system catalog, it seems that it has to already be replicating before it knows what its configuration is. At the

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Robert Haas
On Thu, Sep 23, 2010 at 12:52 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On Thu, 2010-09-23 at 11:43 -0400, Tom Lane wrote: Well, for one thing, how do you add a new slave?  If its configuration comes from a system catalog, it seems that it has to already

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Now, admittedly, in more complex topologies, and especially if you're using configuration options that pertain to the behavior of disconnected standbys (e.g. wait for them, or retain WAL for them), you're going to need to adjust the configs. But I

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Sep 23, 2010 at 12:52 PM, Tom Lane t...@sss.pgh.pa.us wrote: Um ... so how does this standby know what master to connect to, what password to offer, etc?  I don't think that pass down parameters after connecting is likely to cover anything but

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Robert Haas
On Thu, Sep 23, 2010 at 1:03 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Thu, Sep 23, 2010 at 12:52 PM, Tom Lane t...@sss.pgh.pa.us wrote: Um ... so how does this standby know what master to connect to, what password to offer, etc?  I don't think that

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Csaba Nagy
On Thu, 2010-09-23 at 16:18 +0300, Heikki Linnakangas wrote: There's a program called pg_readahead somewhere on pgfoundry by NTT that will help if it's the single-threadedness of I/O. Before handing the WAL file to the server, it scans it through and calls posix_fadvise for all the blocks

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Csaba Nagy
On Thu, 2010-09-23 at 11:43 -0400, Tom Lane wrote: What other problems are there that mean we *must* have a file? Well, for one thing, how do you add a new slave? If its configuration comes from a system catalog, it seems that it has to already be replicating before it knows what its

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Heikki Linnakangas
On 23/09/10 20:03, Tom Lane wrote: Robert Haasrobertmh...@gmail.com writes: On Thu, Sep 23, 2010 at 12:52 PM, Tom Lanet...@sss.pgh.pa.us wrote: Um ... so how does this standby know what master to connect to, what password to offer, etc? I don't think that pass down parameters after

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Simon Riggs
On Thu, 2010-09-23 at 13:07 -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: Now, admittedly, in more complex topologies, and especially if you're using configuration options that pertain to the behavior of disconnected standbys (e.g. wait for them, or retain WAL for them),

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Robert Haas
On Thu, Sep 23, 2010 at 3:46 PM, Simon Riggs si...@2ndquadrant.com wrote: Well, its not at all hard to see how that could be configured, because I already proposed a simple way of implementing parameters that doesn't suffer from those problems. My proposal did not give roles to named standbys

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Simon Riggs
On Wed, 2010-09-22 at 15:31 -0700, Josh Berkus wrote: The above case is one where I can see your point and it does sound easier in that case. But I then think: What happens after failover?. We would then need to have 12 different standby.conf files, one on each standby that describes what

Re: [HACKERS] Configuring synchronous replication

2010-09-23 Thread Simon Riggs
On Thu, 2010-09-23 at 20:42 +0300, Heikki Linnakangas wrote: If you want the behavior where the master doesn't acknowledge a commit to the client until the standby (or all standbys, or one of them etc.) acknowledges it, even if the standby is not currently connected, the master needs to

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Heikki Linnakangas
On 22/09/10 03:25, Joshua D. Drake wrote: Why is this in the config file at all. It should be: synchronous_replication = TRUE/FALSE Umm, what does this do? then ALTER CLUSTER ENABLE REPLICATION FOR FOO; ALTER CLUSTER SET keep_connect ON FOO TO TRUE; Or some such thing. I like a

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Heikki Linnakangas
On 21/09/10 18:12, Tom Lane wrote: Heikki Linnakangasheikki.linnakan...@enterprisedb.com writes: On 21/09/10 11:52, Thom Brown wrote: My fear would be standby.conf would be edited by users who don't really know XML and then we'd have 3 different styles of config to tell the user to edit.

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Markus Wanner
Hi, On 09/21/2010 08:05 PM, Simon Riggs wrote: Hmm, no reason? The reason is that the alternative is that the session would hang until a standby arrived that offered that level of service. Why would you want that behaviour? Would you really request that option? I think I now agree with Simon

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Andrew Dunstan
On 09/22/2010 04:18 AM, Heikki Linnakangas wrote: On 21/09/10 18:12, Tom Lane wrote: Heikki Linnakangasheikki.linnakan...@enterprisedb.com writes: On 21/09/10 11:52, Thom Brown wrote: My fear would be standby.conf would be edited by users who don't really know XML and then we'd have 3

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Dave Page
On Wed, Sep 22, 2010 at 9:47 AM, Andrew Dunstan and...@dunslane.net wrote: The ini file format is not flexible enough, IMNSHO. If we're going to adopt a new config file format it should have these characteristics, among others: well known (let's not invent a new one) supports hierarchical

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Andrew Dunstan
On 09/22/2010 04:54 AM, Dave Page wrote: On Wed, Sep 22, 2010 at 9:47 AM, Andrew Dunstanand...@dunslane.net wrote: The ini file format is not flexible enough, IMNSHO. If we're going to adopt a new config file format it should have these characteristics, among others: well known (let's not

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Dave Page
On Wed, Sep 22, 2010 at 12:07 PM, Andrew Dunstan and...@dunslane.net wrote: On 09/22/2010 04:54 AM, Dave Page wrote: On Wed, Sep 22, 2010 at 9:47 AM, Andrew Dunstanand...@dunslane.net  wrote: The ini file format is not flexible enough, IMNSHO. If we're going to adopt a new config file

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Peter Eisentraut
On ons, 2010-09-22 at 12:20 +0100, Dave Page wrote: No, it's really not hierarchical. It only has goes one level deep. I guess pgAdmin/wxWidgets are broken then :-) [Servers] Count=5 [Servers/1] Server=localhost Well, by that logic, even what we have now for postgresql.conf is

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Andrew Dunstan
On 09/22/2010 07:20 AM, Dave Page wrote: On Wed, Sep 22, 2010 at 12:07 PM, Andrew Dunstanand...@dunslane.net wrote: On 09/22/2010 04:54 AM, Dave Page wrote: On Wed, Sep 22, 2010 at 9:47 AM, Andrew Dunstanand...@dunslane.net wrote: The ini file format is not flexible enough, IMNSHO. If

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Dave Page
On Wed, Sep 22, 2010 at 12:50 PM, Peter Eisentraut pete...@gmx.net wrote: On ons, 2010-09-22 at 12:20 +0100, Dave Page wrote: No, it's really not hierarchical. It only has goes one level deep. I guess pgAdmin/wxWidgets are broken then :-) [Servers] Count=5 [Servers/1] Server=localhost

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Andrew Dunstan
On 09/22/2010 07:57 AM, Dave Page wrote: On Wed, Sep 22, 2010 at 12:50 PM, Peter Eisentrautpete...@gmx.net wrote: On ons, 2010-09-22 at 12:20 +0100, Dave Page wrote: No, it's really not hierarchical. It only has goes one level deep. I guess pgAdmin/wxWidgets are broken then :-) [Servers]

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Dave Page
On Wed, Sep 22, 2010 at 1:25 PM, Andrew Dunstan and...@dunslane.net wrote: XML is not the only alternative - please don't use it as a straw man. For example, here is a fragment from the Bacula docs using their hierarchical format: FileSet { Name = Test Include { File =

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Andrew Dunstan
On 09/22/2010 08:32 AM, Dave Page wrote: On Wed, Sep 22, 2010 at 1:25 PM, Andrew Dunstanand...@dunslane.net wrote: XML is not the only alternative - please don't use it as a straw man. For example, here is a fragment from the Bacula docs using their hierarchical format: FileSet { Name =

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Robert Haas
On Wed, Sep 22, 2010 at 9:01 AM, Andrew Dunstan and...@dunslane.net wrote: I can't imagine trying to configure Bacula using ini file format - the mind just boggles. Frankly, I'd rather stick with our current config format than change to something as inadequate as ini file format. Perhaps we

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Simon Riggs
On Tue, 2010-09-21 at 17:04 -0700, Josh Berkus wrote: That said, the timeout option also feels a bit wishy-washy to me. With a timeout, acknowledgment of a commit means your transaction is safely committed in the master and slave. Or not, if there was some glitch with the slave. That

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Bruce Momjian
Robert Haas wrote: [server] guc=value or server.guc=value Yes, this was my idea too. It uses our existing config file format. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Thom Brown
On 22 September 2010 17:23, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: [server] guc=value or server.guc=value   Yes, this was my idea too.  It uses our existing config file format. So... sync_rep_services = {critical: recv=2, fsync=2, replay=1;

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Bruce Momjian
Thom Brown wrote: On 22 September 2010 17:23, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: [server] guc=value or server.guc=value ? Yes, this was my idea too. ?It uses our existing config file format. So... sync_rep_services = {critical:

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Joshua D. Drake
On Wed, 2010-09-22 at 17:43 +0100, Thom Brown wrote: So... sync_rep_services = {critical: recv=2, fsync=2, replay=1; important: fsync=3; reporting: recv=2, apply=1} becomes ... sync_rep_services.critical.recv = 2

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Robert Haas
On Wed, Sep 22, 2010 at 12:51 PM, Joshua D. Drake j...@commandprompt.com wrote: On Wed, 2010-09-22 at 17:43 +0100, Thom Brown wrote: So... sync_rep_services = {critical: recv=2, fsync=2, replay=1;                      important: fsync=3;                      reporting: recv=2, apply=1}

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Heikki Linnakangas
On 22/09/10 20:00, Robert Haas wrote: But it CAN'T be a system catalog, because, among other problems, that rules out cascading slaves, which are a feature a lot of people probably want to eventually have. FWIW it could be a system catalog backed by a flat file. But I'm not in favor of that

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Aidan Van Dyk
On Wed, Sep 22, 2010 at 8:12 AM, Simon Riggs si...@2ndquadrant.com wrote: Not speaking to the necessity of standby registration, but... Thinking of this as a sysadmin, what I want is to have *one place* I can go an troubleshoot my standby setup.  If I have 12 synch standbys and they're

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Joshua D. Drake
On Wed, 2010-09-22 at 13:00 -0400, Robert Haas wrote: I mean really? ALTER CLUSTER ENABLE [SYNC] REPLICATION ON db.foobar.com PORT 5432 ALIAS CRITICAL; ALTER CLUSTER SET REPLICATION CRITICAL RECEIVE FOR 2; ALTER CLUSTER SET REPLICATION CRITICAL FSYNC FOR 2; ALTER CLUSTER SET

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Bruce Momjian
Heikki Linnakangas wrote: On 22/09/10 20:00, Robert Haas wrote: But it CAN'T be a system catalog, because, among other problems, that rules out cascading slaves, which are a feature a lot of people probably want to eventually have. FWIW it could be a system catalog backed by a flat file.

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Heikki Linnakangas
On 22/09/10 20:02, Heikki Linnakangas wrote: On 22/09/10 20:00, Robert Haas wrote: But it CAN'T be a system catalog, because, among other problems, that rules out cascading slaves, which are a feature a lot of people probably want to eventually have. FWIW it could be a system catalog backed

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Robert Haas
On Wed, Sep 22, 2010 at 1:09 PM, Joshua D. Drake j...@commandprompt.com wrote: On Wed, 2010-09-22 at 13:00 -0400, Robert Haas wrote: I mean really? ALTER CLUSTER ENABLE [SYNC] REPLICATION ON db.foobar.com PORT 5432 ALIAS CRITICAL; ALTER CLUSTER SET REPLICATION CRITICAL RECEIVE FOR 2;

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Sep 22, 2010 at 1:09 PM, Joshua D. Drake j...@commandprompt.com wrote: On Wed, 2010-09-22 at 13:00 -0400, Robert Haas wrote: But it CAN'T be a system catalog, because, among other problems, that rules out cascading slaves, which are a feature

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Joshua D. Drake
On Wed, 2010-09-22 at 13:26 -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Sep 22, 2010 at 1:09 PM, Joshua D. Drake j...@commandprompt.com wrote: On Wed, 2010-09-22 at 13:00 -0400, Robert Haas wrote: But it CAN'T be a system catalog, because, among other

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Dimitri Fontaine
Joshua D. Drake j...@commandprompt.com writes: Unless I am missing something the catalog only needs information for its specific cluster. E.g; My Master is, I am master for. I think the cluster here is composed of all and any server partaking into the replication network, whatever its role and

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Yeb Havinga
Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Sep 22, 2010 at 1:09 PM, Joshua D. Drake j...@commandprompt.com wrote: On Wed, 2010-09-22 at 13:00 -0400, Robert Haas wrote: But it CAN'T be a system catalog, because, among other problems, that rules out

Re: [HACKERS] Configuring synchronous replication

2010-09-22 Thread Josh Berkus
The above case is one where I can see your point and it does sound easier in that case. But I then think: What happens after failover?. We would then need to have 12 different standby.conf files, one on each standby that describes what the setup would look like if that standby became the

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Simon Riggs
On Mon, 2010-09-20 at 22:42 +0100, Thom Brown wrote: On 20 September 2010 22:14, Robert Haas robertmh...@gmail.com wrote: Well, if you need to talk to all the other standbys and see who has the furtherst-advanced xlog pointer, it seems like you have to have a list somewhere of who they all

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Fujii Masao
On Sat, Sep 18, 2010 at 4:36 AM, Dimitri Fontaine dfonta...@hi-media.com wrote: Simon Riggs si...@2ndquadrant.com writes: On Fri, 2010-09-17 at 21:20 +0900, Fujii Masao wrote: What synchronization level does each combination of sync_replication and sync_replication_service lead to? There are

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: So, here, we have two quite different things to be concerned about. First is the configuration, and I say that managing a distributed setup will be easier for the DBA. Yeah, I disagree with that, but I suppose it's a question of opinion. I'd be

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Fujii Masao
On Sun, Sep 19, 2010 at 7:20 AM, Robert Haas robertmh...@gmail.com wrote: On Sat, Sep 18, 2010 at 5:42 PM, Josh Berkus j...@agliodbs.com wrote: There are considerable benefits to having a standby registry with a table-like interface.  Particularly, one where we could change replication via

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Thom Brown
On 21 September 2010 09:29, Fujii Masao masao.fu...@gmail.com wrote: On Sun, Sep 19, 2010 at 7:20 AM, Robert Haas robertmh...@gmail.com wrote: On Sat, Sep 18, 2010 at 5:42 PM, Josh Berkus j...@agliodbs.com wrote: There are considerable benefits to having a standby registry with a table-like

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Dave Page
On Tue, Sep 21, 2010 at 9:34 AM, Thom Brown t...@linux.com wrote: I really don't think an XML config would improve anything.  In fact it would just introduce more ways to break the config by the mere fact it has to be well-formed.  I'd be in favour of one similar to pg_hba.conf, because then,

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Thom Brown
On 21 September 2010 09:37, Dave Page dp...@pgadmin.org wrote: On Tue, Sep 21, 2010 at 9:34 AM, Thom Brown t...@linux.com wrote: I really don't think an XML config would improve anything.  In fact it would just introduce more ways to break the config by the mere fact it has to be well-formed.  

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Fujii Masao
On Mon, Sep 20, 2010 at 3:27 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: However, the wait forever behavior becomes useful if you have a monitoring application outside the DB that decides when enough is enough and tells the DB that the slave can be considered dead. So wait

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Heikki Linnakangas
On 21/09/10 11:52, Thom Brown wrote: My fear would be standby.conf would be edited by users who don't really know XML and then we'd have 3 different styles of config to tell the user to edit. I'm not a big fan of XML either. That said, the format could use some hierarchy. If we add many more

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: On 21/09/10 11:52, Thom Brown wrote: My fear would be standby.conf would be edited by users who don't really know XML and then we'd have 3 different styles of config to tell the user to edit. I'm not a big fan of XML either. ...

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Robert Haas
On Tue, Sep 21, 2010 at 11:12 AM, Tom Lane t...@sss.pgh.pa.us wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: On 21/09/10 11:52, Thom Brown wrote: My fear would be standby.conf would be edited by users who don't really know XML and then we'd have 3 different styles of

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Simon Riggs
On Tue, 2010-09-21 at 16:58 +0900, Fujii Masao wrote: On Sat, Sep 18, 2010 at 4:36 AM, Dimitri Fontaine dfonta...@hi-media.com wrote: Simon Riggs si...@2ndquadrant.com writes: On Fri, 2010-09-17 at 21:20 +0900, Fujii Masao wrote: What synchronization level does each combination of

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Bruce Momjian
Robert Haas wrote: On Tue, Sep 21, 2010 at 11:12 AM, Tom Lane t...@sss.pgh.pa.us wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: On 21/09/10 11:52, Thom Brown wrote: My fear would be standby.conf would be edited by users who don't really know XML and then we'd have 3

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Josh Berkus
That said, the timeout option also feels a bit wishy-washy to me. With a timeout, acknowledgment of a commit means your transaction is safely committed in the master and slave. Or not, if there was some glitch with the slave. That doesn't seem like a very useful guarantee; if you're happy

Re: [HACKERS] Configuring synchronous replication

2010-09-21 Thread Joshua D. Drake
Crazy idea, but could we use format like postgresql.conf by extending postgresql.conf syntax, e.g.: server1.failover = false server1.keep_connect = true Why is this in the config file at all. It should be: synchronous_replication = TRUE/FALSE then ALTER CLUSTER ENABLE

Re: [HACKERS] Configuring synchronous replication

2010-09-20 Thread Heikki Linnakangas
On 19/09/10 01:20, Robert Haas wrote: On Sat, Sep 18, 2010 at 5:42 PM, Josh Berkusj...@agliodbs.com wrote: There are considerable benefits to having a standby registry with a table-like interface. Particularly, one where we could change replication via UPDATE (or ALTER STANDBY) statements.

Re: [HACKERS] Configuring synchronous replication

2010-09-20 Thread Markus Wanner
Hi, On 09/17/2010 01:56 PM, Fujii Masao wrote: And standby registration is required when we support wait forever when synchronous standby isn't connected at the moment option that Heikki explained upthread. That requirement can be reduced to say that the master only needs to known how many

Re: [HACKERS] Configuring synchronous replication

2010-09-20 Thread Simon Riggs
On Mon, 2010-09-20 at 09:27 +0300, Heikki Linnakangas wrote: On 18/09/10 22:59, Robert Haas wrote: On Sat, Sep 18, 2010 at 4:50 AM, Simon Riggssi...@2ndquadrant.com wrote: Waiting might sound attractive. In practice, waiting will make all of your connections lock up and it will look to

Re: [HACKERS] Configuring synchronous replication

2010-09-20 Thread Heikki Linnakangas
On 20/09/10 12:17, Simon Riggs wrote: err... what is the difference between a timeout and stonith? STONITH (Shoot The Other Node In The Head) means that the other node is somehow disabled so that it won't unexpectedly come back alive. A timeout means that the slave hasn't been seen for a

Re: [HACKERS] Configuring synchronous replication

2010-09-20 Thread Simon Riggs
On Mon, 2010-09-20 at 15:16 +0300, Heikki Linnakangas wrote: On 20/09/10 12:17, Simon Riggs wrote: err... what is the difference between a timeout and stonith? STONITH (Shoot The Other Node In The Head) means that the other node is somehow disabled so that it won't unexpectedly come back

Re: [HACKERS] Configuring synchronous replication

2010-09-20 Thread Heikki Linnakangas
On 20/09/10 15:50, Simon Riggs wrote: On Mon, 2010-09-20 at 15:16 +0300, Heikki Linnakangas wrote: On 20/09/10 12:17, Simon Riggs wrote: err... what is the difference between a timeout and stonith? STONITH (Shoot The Other Node In The Head) means that the other node is somehow disabled so

Re: [HACKERS] Configuring synchronous replication

2010-09-20 Thread Robert Haas
On Mon, Sep 20, 2010 at 8:50 AM, Simon Riggs si...@2ndquadrant.com wrote: Please respond to the main point: Following some thought and analysis, AFAICS there is no sensible use case that requires standby registration. I disagree. You keep analyzing away the cases that require standby

Re: [HACKERS] Configuring synchronous replication

2010-09-20 Thread Dimitri Fontaine
Hi, I'm somewhat sorry to have to play this game, as I sure don't feel smarter by composing this email. Quite the contrary. Robert Haas robertmh...@gmail.com writes: So the wait forever case is, in my opinion, sufficient to demonstrate that we need it, but it's not even my primary reason

Re: [HACKERS] Configuring synchronous replication

2010-09-20 Thread Robert Haas
On Mon, Sep 20, 2010 at 4:10 PM, Dimitri Fontaine dfonta...@hi-media.com wrote: Robert Haas robertmh...@gmail.com writes:   So the wait forever case is, in my opinion, sufficient to demonstrate that we need it, but it's not even my primary reason for wanting to have it. You're talking about

Re: [HACKERS] Configuring synchronous replication

2010-09-20 Thread Thom Brown
On 20 September 2010 22:14, Robert Haas robertmh...@gmail.com wrote: Well, if you need to talk to all the other standbys and see who has the furtherst-advanced xlog pointer, it seems like you have to have a list somewhere of who they all are. When they connect to the master to get the stream,

Re: [HACKERS] Configuring synchronous replication

2010-09-20 Thread Robert Haas
On Mon, Sep 20, 2010 at 5:42 PM, Thom Brown t...@linux.com wrote: On 20 September 2010 22:14, Robert Haas robertmh...@gmail.com wrote: Well, if you need to talk to all the other standbys and see who has the furtherst-advanced xlog pointer, it seems like you have to have a list somewhere of who

Re: [HACKERS] Configuring synchronous replication

2010-09-19 Thread Simon Riggs
On Sat, 2010-09-18 at 14:42 -0700, Josh Berkus wrote: * Per-transaction control. Some transactions are important, others are not. Low priority. I see this as a 9.2 feature. Nobody I know is asking for it yet, and I think we need to get the other stuff right first. I understand completely

Re: [HACKERS] Configuring synchronous replication

2010-09-19 Thread Josh Berkus
I've designed a way to tune sync rep so it is usable and useful. And putting that feature into 9.1 costs very little, if anything. My patch to do this is actually smaller than any other attempt to implement this and I claim faster too. You don't need to use the per-transaction controls, but

  1   2   >