Re: [HACKERS] xlog.c: WALInsertLock vs. WALWriteLock

2010-10-27 Thread Markus Wanner
On 10/26/2010 05:52 PM, Alvaro Herrera wrote: And horrible for performance, I imagine. Those locks are highly trafficked. Note, however, that offloading this to the file-system just moves congestion there. So we are effectively saying that we expect filesystems to do a better job (in that

Re: [HACKERS] xlog.c: WALInsertLock vs. WALWriteLock

2010-10-27 Thread Alvaro Herrera
Excerpts from Markus Wanner's message of mié oct 27 11:44:20 -0300 2010: On 10/26/2010 05:52 PM, Alvaro Herrera wrote: And horrible for performance, I imagine. Those locks are highly trafficked. Note, however, that offloading this to the file-system just moves congestion there. So we are

Re: [HACKERS] xlog.c: WALInsertLock vs. WALWriteLock

2010-10-27 Thread Fujii Masao
On Wed, Oct 27, 2010 at 3:03 AM, fazool mein fazoolm...@gmail.com wrote: Might I suggest adopting the same technique walsender does, ie just read the data back from disk?  There's a reason why we gave up trying to have walsender read directly from the buffers. That is exactly what I do not

Re: [HACKERS] xlog.c: WALInsertLock vs. WALWriteLock

2010-10-26 Thread Jeff Janes
On Mon, Oct 25, 2010 at 6:31 PM, Robert Haas robertmh...@gmail.com wrote: On Fri, Oct 22, 2010 at 3:08 PM, fazool mein fazoolm...@gmail.com wrote: I'm writing a function that will read data from the buffer in xlog (i.e. from XLogCtl-pages and XLogCtl-xlblocks). I want to make sure that I am

Re: [HACKERS] xlog.c: WALInsertLock vs. WALWriteLock

2010-10-26 Thread Alvaro Herrera
Excerpts from Jeff Janes's message of mar oct 26 12:22:38 -0300 2010: I don't think that holding WALWriteLock accomplishes much. It prevents part of the buffer from being written out to OS/disk, and thus becoming eligible for being overwritten in the buffer, but the WALInsertLock prevents it

Re: [HACKERS] xlog.c: WALInsertLock vs. WALWriteLock

2010-10-26 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Jeff Janes's message of mar oct 26 12:22:38 -0300 2010: I don't think that holding WALWriteLock accomplishes much. It prevents part of the buffer from being written out to OS/disk, and thus becoming eligible for being overwritten

Re: [HACKERS] xlog.c: WALInsertLock vs. WALWriteLock

2010-10-26 Thread fazool mein
Might I suggest adopting the same technique walsender does, ie just read the data back from disk? There's a reason why we gave up trying to have walsender read directly from the buffers. That is exactly what I do not want to do, i.e. read from disk, as long as the piece of WAL is available

Re: [HACKERS] xlog.c: WALInsertLock vs. WALWriteLock

2010-10-26 Thread Heikki Linnakangas
On 26.10.2010 21:03, fazool mein wrote: Might I suggest adopting the same technique walsender does, ie just read the data back from disk? There's a reason why we gave up trying to have walsender read directly from the buffers. That is exactly what I do not want to do, i.e. read from disk, as

Re: [HACKERS] xlog.c: WALInsertLock vs. WALWriteLock

2010-10-26 Thread Robert Haas
On Tue, Oct 26, 2010 at 2:13 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 26.10.2010 21:03, fazool mein wrote: Might I suggest adopting the same technique walsender does, ie just read the data back from disk?  There's a reason why we gave up trying to have walsender

Re: [HACKERS] xlog.c: WALInsertLock vs. WALWriteLock

2010-10-26 Thread fazool mein
On Tue, Oct 26, 2010 at 11:23 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, Oct 26, 2010 at 2:13 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Can you please describe why walsender reading directly from the buffers was given up? To avoid a lot of locking?

Re: [HACKERS] xlog.c: WALInsertLock vs. WALWriteLock

2010-10-26 Thread Robert Haas
On Tue, Oct 26, 2010 at 2:57 PM, fazool mein fazoolm...@gmail.com wrote: On Tue, Oct 26, 2010 at 11:23 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, Oct 26, 2010 at 2:13 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Can you please describe why walsender

Re: [HACKERS] xlog.c: WALInsertLock vs. WALWriteLock

2010-10-26 Thread Josh Berkus
I agree that the standby might get ahead, but this doesn't necessarily lead to database corruption. Here, the interesting case is what happens when the primary fails, which can lead to *either* of the following two cases: 1) The standby, due to some triggering mechanism, becomes the new

Re: [HACKERS] xlog.c: WALInsertLock vs. WALWriteLock

2010-10-26 Thread Robert Haas
On Tue, Oct 26, 2010 at 3:00 PM, Josh Berkus j...@agliodbs.com wrote: I agree that the standby might get ahead, but this doesn't necessarily lead to database corruption. Here, the interesting case is what happens when the primary fails, which can lead to *either* of the following two cases:

Re: [HACKERS] xlog.c: WALInsertLock vs. WALWriteLock

2010-10-25 Thread Robert Haas
On Fri, Oct 22, 2010 at 3:08 PM, fazool mein fazoolm...@gmail.com wrote: I'm writing a function that will read data from the buffer in xlog (i.e. from XLogCtl-pages and XLogCtl-xlblocks). I want to make sure that I am doing it correctly. For reading from the buffer, do I need to lock

Re: [HACKERS] xlog.c: WALInsertLock vs. WALWriteLock

2010-10-24 Thread Tallat Mahmood
I'm writing a function that will read data from the buffer in xlog (i.e. from XLogCtl-pages and XLogCtl-xlblocks). I want to make sure that I am doing it correctly. Got an example of what the function might look like? Say something like this: bool ReadLogFromBuffer(char *buf, int len,

Re: [HACKERS] xlog.c: WALInsertLock vs. WALWriteLock

2010-10-23 Thread David Fetter
On Fri, Oct 22, 2010 at 12:08:54PM -0700, fazool mein wrote: Hello guys, I'm writing a function that will read data from the buffer in xlog (i.e. from XLogCtl-pages and XLogCtl-xlblocks). I want to make sure that I am doing it correctly. Got an example of what the function might look