Re: [HACKERS] walsender doesn't send keepalives when writes are pending

2014-03-06 Thread Heikki Linnakangas
On 03/05/2014 10:57 PM, Andres Freund wrote: On 2014-03-05 18:26:13 +0200, Heikki Linnakangas wrote: The logic was the same before the patch, but I added the XXX comment above. Why do we sleep in increments of 1/10 of wal_sender_timeout? Originally, the code calculated when the next wakeup

Re: [HACKERS] walsender doesn't send keepalives when writes are pending

2014-03-05 Thread Heikki Linnakangas
On 02/25/2014 06:41 PM, Robert Haas wrote: On Tue, Feb 25, 2014 at 11:23 AM, Andres Freund and...@2ndquadrant.com wrote: Usually that state will be reached very quickly because before that we're writing data to the network as fast as it can be read from disk. I'm unimpressed. Even if that is

Re: [HACKERS] walsender doesn't send keepalives when writes are pending

2014-03-05 Thread Andres Freund
On 2014-03-05 18:26:13 +0200, Heikki Linnakangas wrote: On 02/25/2014 06:41 PM, Robert Haas wrote: On Tue, Feb 25, 2014 at 11:23 AM, Andres Freund and...@2ndquadrant.com wrote: Usually that state will be reached very quickly because before that we're writing data to the network as fast as it

Re: [HACKERS] walsender doesn't send keepalives when writes are pending

2014-02-25 Thread Andres Freund
On 2014-02-25 10:45:55 -0500, Robert Haas wrote: On Fri, Feb 14, 2014 at 7:05 AM, Andres Freund and...@2ndquadrant.com wrote: In WalSndLoop() we do: wakeEvents = WL_LATCH_SET | WL_POSTMASTER_DEATH | WL_TIMEOUT | WL_SOCKET_READABLE; if (pq_is_send_pending())

Re: [HACKERS] walsender doesn't send keepalives when writes are pending

2014-02-25 Thread Robert Haas
On Fri, Feb 14, 2014 at 7:05 AM, Andres Freund and...@2ndquadrant.com wrote: In WalSndLoop() we do: wakeEvents = WL_LATCH_SET | WL_POSTMASTER_DEATH | WL_TIMEOUT | WL_SOCKET_READABLE; if (pq_is_send_pending()) wakeEvents |= WL_SOCKET_WRITEABLE; else if

Re: [HACKERS] walsender doesn't send keepalives when writes are pending

2014-02-25 Thread Robert Haas
On Tue, Feb 25, 2014 at 10:54 AM, Andres Freund and...@2ndquadrant.com wrote: I am not sure I can follow. Why doesn't it make sense to send out the keepalive (with replyRequested = true) when we're busy sending stuff (which will be the case most of the time on a busy server)? It may very well

Re: [HACKERS] walsender doesn't send keepalives when writes are pending

2014-02-25 Thread Andres Freund
On 2014-02-25 11:15:46 -0500, Robert Haas wrote: On Tue, Feb 25, 2014 at 10:54 AM, Andres Freund and...@2ndquadrant.com wrote: I am not sure I can follow. Why doesn't it make sense to send out the keepalive (with replyRequested = true) when we're busy sending stuff (which will be the case

Re: [HACKERS] walsender doesn't send keepalives when writes are pending

2014-02-25 Thread Robert Haas
On Tue, Feb 25, 2014 at 11:23 AM, Andres Freund and...@2ndquadrant.com wrote: On 2014-02-25 11:15:46 -0500, Robert Haas wrote: On Tue, Feb 25, 2014 at 10:54 AM, Andres Freund and...@2ndquadrant.com wrote: I am not sure I can follow. Why doesn't it make sense to send out the keepalive (with

Re: [HACKERS] walsender doesn't send keepalives when writes are pending

2014-02-22 Thread Andres Freund
On 2014-02-22 09:08:39 +0530, Amit Kapila wrote: The danger is rather that *no* keepalive is sent (with requestReply = true triggering a reply by the client) by the walsender. Try to run pg_receivexlog against a busy server with a low walsender timeout. Since we'll never get to sending a

Re: [HACKERS] walsender doesn't send keepalives when writes are pending

2014-02-21 Thread Andres Freund
On 2014-02-21 10:08:44 +0530, Amit Kapila wrote: On Fri, Feb 14, 2014 at 5:35 PM, Andres Freund and...@2ndquadrant.com wrote: Hi, In WalSndLoop() we do: wakeEvents = WL_LATCH_SET | WL_POSTMASTER_DEATH | WL_TIMEOUT | WL_SOCKET_READABLE; if (pq_is_send_pending())

Re: [HACKERS] walsender doesn't send keepalives when writes are pending

2014-02-21 Thread Amit Kapila
On Fri, Feb 21, 2014 at 2:36 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-02-21 10:08:44 +0530, Amit Kapila wrote: I think the main reason of ping is to detect n/w break sooner. On a busy server, wouldn't WALSender can detect it when next time it will try to send the remaining

Re: [HACKERS] walsender doesn't send keepalives when writes are pending

2014-02-21 Thread Andres Freund
On 2014-02-21 19:03:29 +0530, Amit Kapila wrote: On Fri, Feb 21, 2014 at 2:36 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-02-21 10:08:44 +0530, Amit Kapila wrote: I think the main reason of ping is to detect n/w break sooner. On a busy server, wouldn't WALSender can detect it

Re: [HACKERS] walsender doesn't send keepalives when writes are pending

2014-02-21 Thread Amit Kapila
On Fri, Feb 21, 2014 at 7:10 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-02-21 19:03:29 +0530, Amit Kapila wrote: On Fri, Feb 21, 2014 at 2:36 PM, Andres Freund and...@2ndquadrant.com wrote: Well, especially on a pipelined connection, that can take a fair bit. TCP timeouts

Re: [HACKERS] walsender doesn't send keepalives when writes are pending

2014-02-20 Thread Amit Kapila
On Fri, Feb 14, 2014 at 5:35 PM, Andres Freund and...@2ndquadrant.com wrote: Hi, In WalSndLoop() we do: wakeEvents = WL_LATCH_SET | WL_POSTMASTER_DEATH | WL_TIMEOUT | WL_SOCKET_READABLE; if (pq_is_send_pending()) wakeEvents |= WL_SOCKET_WRITEABLE; else if

[HACKERS] walsender doesn't send keepalives when writes are pending

2014-02-14 Thread Andres Freund
Hi, In WalSndLoop() we do: wakeEvents = WL_LATCH_SET | WL_POSTMASTER_DEATH | WL_TIMEOUT | WL_SOCKET_READABLE; if (pq_is_send_pending()) wakeEvents |= WL_SOCKET_WRITEABLE; else if (wal_sender_timeout 0 !ping_sent) { ... if (GetCurrentTimestamp() =

Re: [HACKERS] walsender doesn't send keepalives when writes are pending

2014-02-14 Thread Greg Stark
On Fri, Feb 14, 2014 at 12:05 PM, Andres Freund and...@2ndquadrant.com wrote: There's no reason not to ask for a ping when we're writing. Is there a reason to ask for a ping? The point of keepalives is to ensure there's some traffic on idle connections so that if the connection is dead it

Re: [HACKERS] walsender doesn't send keepalives when writes are pending

2014-02-14 Thread Andres Freund
On 2014-02-14 12:55:06 +, Greg Stark wrote: On Fri, Feb 14, 2014 at 12:05 PM, Andres Freund and...@2ndquadrant.com wrote: There's no reason not to ask for a ping when we're writing. Is there a reason to ask for a ping? The point of keepalives is to ensure there's some traffic on idle

Re: [HACKERS] walsender doesn't send keepalives when writes are pending

2014-02-14 Thread Andres Freund
On 2014-02-14 13:58:59 +0100, Andres Freund wrote: On 2014-02-14 12:55:06 +, Greg Stark wrote: On Fri, Feb 14, 2014 at 12:05 PM, Andres Freund and...@2ndquadrant.com wrote: There's no reason not to ask for a ping when we're writing. Is there a reason to ask for a ping? The