Re: [HACKERS] [GENERAL] Proposal for a cascaded master-slave replication system
On Wed, Nov 12, 2003 at 07:46:11PM -0500, James Robinson wrote: Speaking from a non-profit whose enterprise data sits inside postgres, we would be willing to invest a few thousand dollars into the pot of synchronous multi-master replication. Postgres-r sounded absolutely marvelous to us back in the day that it was rumored to be one of the possible deliverables of 7.4. As far as I know, Postgres-R hackers are eager for funding. There is a foundation which was established to fund such activities, in an effort to pool the resources that people had to donate. You could ask in the Postgres-R mailing list about it. A -- Andrew Sullivan 204-4141 Yonge Street Afilias CanadaToronto, Ontario Canada [EMAIL PROTECTED] M2P 2A8 +1 416 646 3304 x110 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] [GENERAL] Proposal for a cascaded master-slave replication system
Speaking from a non-profit whose enterprise data sits inside postgres, we would be willing to invest a few thousand dollars into the pot of synchronous multi-master replication. Postgres-r sounded absolutely marvelous to us back in the day that it was rumored to be one of the possible deliverables of 7.4. Not so much for nine-nines of uptime, but for the case of being able to take a full hit on a DB box in production yet still remain running w/o any data loss. Our application servers are JBoss and will be high-available clustered / fully-mirrored, but even with RAID on the DB box one bad thing could take it down, and the data between the hourly backup would go down with it. We have experimented in-house with C-JDBC [ being 'lucky' enough to have all DB writes to go through JDBC ], but would feel more confident w/o involving another service in-between the application and the DB layers, especially since it is not yet fully high-available -- currently shifts the single-point of failure from the DB layer to the CJDBC controller single point. It is reported to have HA via group communication 'soon', but, you never can tell. Read up on it at http://c-jdbc.objectweb.org/ , but the end feel I got from it was not nearly so warm and cozy with the problem being solved at the right place -- the postgres-r way felt much more robust / speedy. We won't ever have parallel oracle dollars, but we would have dollars to bring higher-availability to postgres. 'Cause its our butt on the line hosting our client's data. James Robinson Socialserve.com ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] [GENERAL] Proposal for a cascaded master-slave replication system
Jan Wieck wrote: http://developer.postgresql.org/~wieck/slony1.html Very interesting read. Nice work! We want to build this system as a community project. The plan was from the beginning to release the product under the BSD license. And we think it is best to start it as such and to ask for suggestions during the design phase already. I couldn't quite tell from the design doc -- do you intend to support conditional replication at a row level? I'm also curious, with cascaded replication, how do you handle the case where a second level slave has a transaction failure for some reason, i.e.: M / \ / \ Sa Sb / \ / \ Sc Sd Se Sf What happens if data is successfully replicated to Sa, Sb, Sc, and Sd, and then an exception/rollback occurs on Se? Joe ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] [GENERAL] Proposal for a cascaded master-slave replication system
On Nov 11, 2003, at 12:11 PM, Joe Conway wrote: Jan Wieck wrote: http://developer.postgresql.org/~wieck/slony1.html Very interesting read. Nice work! Ditto. I'll read it a bit closer later, but after a quick read it seems quite complete and well thought out. I especially like that sequences are being dealt with. Thanks for putting the effort in, and making it a community project. We want to build this system as a community project. The plan was from the beginning to release the product under the BSD license. And we think it is best to start it as such and to ask for suggestions during the design phase already. I couldn't quite tell from the design doc -- do you intend to support conditional replication at a row level? I'm also curious, with cascaded replication, how do you handle the case where a second level slave has a transaction failure for some reason, i.e.: M / \ / \ Sa Sb / \ / \ Sc Sd Se Sf What happens if data is successfully replicated to Sa, Sb, Sc, and Sd, and then an exception/rollback occurs on Se? Joe ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED]) Andrew Rawnsley President The Ravensfield Digital Resource Group, Ltd. (740) 587-0114 www.ravensfield.com ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] [GENERAL] Proposal for a cascaded master-slave replication system
Joe Conway wrote: Jan Wieck wrote: http://developer.postgresql.org/~wieck/slony1.html Very interesting read. Nice work! We want to build this system as a community project. The plan was from the beginning to release the product under the BSD license. And we think it is best to start it as such and to ask for suggestions during the design phase already. I couldn't quite tell from the design doc -- do you intend to support conditional replication at a row level? If you mean to configure the system to replicate rows to different destinations (slaves) based on arbitrary qualifications, no. I had thought about it, but it does not really fit into the datacenter and failover picture, so it is not required to meet the goals and adds unnecessary complexity. This sort of feature is much more important for a replication system designed for hundreds or thousands of sporadic, asynchronous multi-master systems, the typical salesman on the street kind of replication. I'm also curious, with cascaded replication, how do you handle the case where a second level slave has a transaction failure for some reason, i.e.: M / \ / \ Sa Sb / \ / \ Sc Sd Se Sf What happens if data is successfully replicated to Sa, Sb, Sc, and Sd, and then an exception/rollback occurs on Se? First, it does not replicate single transactions. It replicates batches of them together. Since the transactions are already committed (and possibly some other depending on them too), there is no way - you loose Se. If this is only a temporary failure, like a power fail and the database recovers on restart fine including the last confirmed SYNC event (they get confirmed after they commit locally, but that's before the next checkpoint so there is actually a gap where the slave could loose a committed transaction and then it's lost for sure) ... so if it comes back up without loosing the last confirmed SYNC, it will catch up. Jan -- #==# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #== [EMAIL PROTECTED] # ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] [GENERAL] Proposal for a cascaded master-slave replication system
Jan Wieck wrote: If you mean to configure the system to replicate rows to different destinations (slaves) based on arbitrary qualifications, no. I had thought about it, but it does not really fit into the datacenter and failover picture, so it is not required to meet the goals and adds unnecessary complexity. This sort of feature is much more important for a replication system designed for hundreds or thousands of sporadic, asynchronous multi-master systems, the typical salesman on the street kind of replication. OK, thanks. This actually fits any kind of distributed application. We have one that lives in our datacenters, but needs to replicate across both fast LAN/MAN and slow WAN. It is multimaster in the sense that individual data rows can be originated anywhere, but they are read-only in nodes other than where they were originated. Anyway, I'm using a hacked copy of dbmirror at the moment. First, it does not replicate single transactions. It replicates batches of them together. Since the transactions are already committed (and possibly some other depending on them too), there is no way - you loose Se. OK, got it. Thanks. Joe ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
