Re: [HACKERS] proposal: multiple read-write masters in a cluster with wal-streaming synchronization

2014-01-02 Thread Andres Freund
On 2013-12-31 13:51:08 -0800, Mark Dilger wrote: The BDR documentation http://wiki.postgresql.org/images/7/75/BDR_Presentation_PGCon2012.pdf says,     Physical replication forces us to use just one node: multi-master required for write scalability     Physical replication provides

Re: [HACKERS] proposal: multiple read-write masters in a cluster with wal-streaming synchronization

2014-01-02 Thread Mark Dilger
My original email was mostly a question about whether WAL data could be merged from multiple servers, or whether I was overlooking some unsolvable difficulty.  I'm still mostly curious about that question. I anticipated that my proposal would require partitioning the catalogs. For instance,

Re: [HACKERS] proposal: multiple read-write masters in a cluster with wal-streaming synchronization

2014-01-02 Thread Andres Freund
On 2014-01-02 10:18:52 -0800, Mark Dilger wrote: I anticipated that my proposal would require partitioning the catalogs. For instance, autovacuum could only run on locally owned tables, and would need to store the analyze stats data in a catalog partition belonging to the local server, but

Re: [HACKERS] proposal: multiple read-write masters in a cluster with wal-streaming synchronization

2014-01-02 Thread Merlin Moncure
On Tue, Dec 31, 2013 at 3:51 PM, Mark Dilger markdil...@yahoo.com wrote: The BDR documentation http://wiki.postgresql.org/images/7/75/BDR_Presentation_PGCon2012.pdf says, Physical replication forces us to use just one node: multi-master required for write scalability Physical

Re: [HACKERS] proposal: multiple read-write masters in a cluster with wal-streaming synchronization

2014-01-02 Thread Mark Dilger
Thanks to both of you for all the feedback.  Your reasoning about why it is not worth implementing, what the problems with it would be, etc., are helpful. Sorry about using the word multimaster where it might have been better to say sharded. BTW, since the space shuttle has already left orbit,

Re: [HACKERS] proposal: multiple read-write masters in a cluster with wal-streaming synchronization

2014-01-02 Thread Andres Freund
On 2014-01-02 11:35:57 -0800, Mark Dilger wrote: BTW, since the space shuttle has already left orbit, as you metaphorically put it, maybe there should be more visibility to the wider world about this?  You can go to postgresql.org and find diddly squat about it.  I grant you that it is not a

[HACKERS] proposal: multiple read-write masters in a cluster with wal-streaming synchronization

2013-12-31 Thread Mark Dilger
This is not entirely pie in the sky, but feel free to tell me why this is crazy.  I have had this idea for several years, but have not seen anyone else suggest it, nor any arguments why it would not work. If we had 64-bit Oids, we could reserve the top 16 bits (for instance) to indicate a

Re: [HACKERS] proposal: multiple read-write masters in a cluster with wal-streaming synchronization

2013-12-31 Thread Alvaro Herrera
Mark Dilger wrote: This is not entirely pie in the sky, but feel free to tell me why this is crazy. Have you seen http://wiki.postgresql.org/wiki/BDR ? -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via

Re: [HACKERS] proposal: multiple read-write masters in a cluster with wal-streaming synchronization

2013-12-31 Thread Mark Dilger
The BDR documentation http://wiki.postgresql.org/images/7/75/BDR_Presentation_PGCon2012.pdf says,     Physical replication forces us to use just one node: multi-master required for write scalability     Physical replication provides best read scalability I am inclined to agree with the