Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-16 Thread Markus Wanner
Hi, Greg Stark wrote: I think my definition would be that a query against the replica will produce the same result as a query against the master -- and that that will be the case even after a system failure. That might not necessarily mean that the log entry is fsynced on the replica, only that

Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-16 Thread Greg Smith
Markus Wanner wrote: You will definitely find different definitions and requirements of what synchronous replication means there. To quote from the Wikipedia entry on Database Replication that Simon pointed to during the earlier discussion, http://en.wikipedia.org/wiki/Database_replication

Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-16 Thread Markus Wanner
Hi, Quoting Greg Smith g...@2ndquadrant.com: Synchronous replication - guarantees zero data loss by the means of atomic write operation, i.e. write either completes on both sides or not at all. Write is not considered complete until acknowledgement by both local and remote storage. Note

Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-13 Thread Fujii Masao
On Fri, Nov 13, 2009 at 3:17 PM, Greg Smith g...@2ndquadrant.com wrote: Yeah, that's the other parts of the industry I was referring to.  MySQL uses semi-synchronous to distinguish between its completely asynchronous default replication mode and one where it provides a somewhat safer

Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-12 Thread Fujii Masao
Hi, On Thu, Nov 12, 2009 at 4:32 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Fujii Masao wrote: The problem is that fsync needs to be issued too frequently, which would be harmless in asynchronous replication, but not in synchronous one. A transaction would have to wait

Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-12 Thread Simon Riggs
On Thu, 2009-11-12 at 17:03 +0900, Fujii Masao wrote: On Thu, Nov 12, 2009 at 4:32 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Fujii Masao wrote: The problem is that fsync needs to be issued too frequently, which would be harmless in asynchronous replication, but not

Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-12 Thread Fujii Masao
On Thu, Nov 12, 2009 at 6:27 PM, Simon Riggs si...@2ndquadrant.com wrote: I agree with you, though it has taken some time to understand what you said and at first my reaction was to disagree. I think the responses you got on this are because you dived straight in with a question before

Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-12 Thread Simon Riggs
On Thu, 2009-11-12 at 21:45 +0900, Fujii Masao wrote: But, as I said on my first post on this thread, even such low-frequent fsync-WAL-before-buffer-flush might cause a response time spike on the primary because the walreceiver must sleep during that fsync. I think that leaving the

Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-12 Thread Tom Lane
Fujii Masao masao.fu...@gmail.com writes: The problem is that fsync needs to be issued too frequently, which would be harmless in asynchronous replication, but not in synchronous one. A transaction would have to wait for the primary's and standby's fsync before returning a success to a client.

Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-12 Thread Greg Smith
Tom Lane wrote: Fujii Masao masao.fu...@gmail.com writes: The problem is that fsync needs to be issued too frequently, which would be harmless in asynchronous replication, but not in synchronous one. A transaction would have to wait for the primary's and standby's fsync before returning a

Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-12 Thread Fujii Masao
On Fri, Nov 13, 2009 at 1:49 AM, Greg Smith g...@2ndquadrant.com wrote: This a distressingly common thing people get wrong about replication.  You can either have synchronous replication, which as you say has to be slow: you must wait for an fsync ACK from the secondary and a return trip before

Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-12 Thread Aidan Van Dyk
* Fujii Masao masao.fu...@gmail.com [091112 20:52]: Personally, I think that semi-synchronous replication is sufficient for HA. Often, but that's not synchronous replication so don't call it such... -- Aidan Van Dyk

Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-12 Thread Greg Smith
Fujii Masao wrote: Personally, I think that semi-synchronous replication is sufficient for HA. Whether or not you think it's sufficient for what you have in mind, synchronous replication requires a return ACK from the secondary before you say things are committed on the primary. If you

Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-12 Thread Fujii Masao
On Fri, Nov 13, 2009 at 10:58 AM, Aidan Van Dyk ai...@highrise.ca wrote: * Fujii Masao masao.fu...@gmail.com [091112 20:52]:                                                        Personally, I think that semi-synchronous replication is sufficient for HA. Often, but that's not synchronous

Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-12 Thread Fujii Masao
On Fri, Nov 13, 2009 at 11:15 AM, Greg Smith g...@2ndquadrant.com wrote: Whether or not you think it's sufficient for what you have in mind, synchronous replication requires a return ACK from the secondary before you say things are committed on the primary.  If you don't do that, it's not true

Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-12 Thread Greg Stark
On Fri, Nov 13, 2009 at 2:37 AM, Fujii Masao masao.fu...@gmail.com wrote: Umm... what is your definition of synchronous? I'm planning to provide four synchronization modes as follows, for v8.5. Does this fit in your I think my definition would be that a query against the replica will produce

Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-12 Thread Greg Smith
Fujii Masao wrote: Umm... what is your definition of synchronous? I'm planning to provide four synchronization modes as follows, for v8.5. Does this fit in your thought? The primary waits ... before returning success of a transaction; * nothing - asynchronous replication * recv ACK -

Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-12 Thread Fujii Masao
On Fri, Nov 13, 2009 at 11:54 AM, Greg Stark gsst...@mit.edu wrote: I think my definition would be that a query against the replica will produce the same result as a query against the master -- and that that will be the case even after a system failure. That might not necessarily mean that the

Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-12 Thread Fujii Masao
On Fri, Nov 13, 2009 at 1:49 PM, Greg Smith g...@2ndquadrant.com wrote: Right, those are the possibilities, all four of them have valid use cases in the field and are worth implementing.  I don't like the label semi-synchronous replication myself, but it's a valuable feature to implement, and

Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-12 Thread Robert Hodges
Hi Greg and Fujii, Just a point on terminology: there's a difference in the usage of semi-synchronous between DRBD and MySQL semi-synchronous replication, which was originally developed by Google. In the Google case semi-synchronous replication is a quorum algorithm where clients receive a

Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-12 Thread Greg Smith
Fujii Masao wrote: On Fri, Nov 13, 2009 at 1:49 PM, Greg Smith g...@2ndquadrant.com wrote: Right, those are the possibilities, all four of them have valid use cases in the field and are worth implementing. I don't like the label semi-synchronous replication myself, but it's a valuable

[HACKERS] write ahead logging in standby (streaming replication)

2009-11-11 Thread Fujii Masao
Hi, Should the standby also have to follow the WAL rule during recovery? The current patch doesn't care about the write order of the data page and WAL in the standby. So, after both servers fail, restarting the ex-standby by itself might corrupt the data. If the standby follows the WAL rule,

Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-11 Thread Tom Lane
Fujii Masao masao.fu...@gmail.com writes: Should the standby also have to follow the WAL rule during recovery? The current patch doesn't care about the write order of the data page and WAL in the standby. So, after both servers fail, restarting the ex-standby by itself might corrupt the data.

Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-11 Thread Fujii Masao
On Thu, Nov 12, 2009 at 12:03 PM, Tom Lane t...@sss.pgh.pa.us wrote: Fujii Masao masao.fu...@gmail.com writes: Should the standby also have to follow the WAL rule during recovery? The current patch doesn't care about the write order of the data page and WAL in the standby. So, after both

Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-11 Thread Heikki Linnakangas
Fujii Masao wrote: The problem is that fsync needs to be issued too frequently, which would be harmless in asynchronous replication, but not in synchronous one. A transaction would have to wait for the primary's and standby's fsync before returning a success to a client. So I'm inclined to