Re: [HACKERS] WALWriter active during recovery

2015-07-02 Thread Andres Freund
On 2015-07-02 14:34:48 +0100, Simon Riggs wrote: This was pushed back from last CF and I haven't worked on it at all, nor will I. Pushing back again. Let's return with feedback, not move, it then.. Moving a entries along which aren't expected to receive updates anytime soon isn't a good

Re: [HACKERS] WALWriter active during recovery

2015-07-02 Thread Simon Riggs
On 2 July 2015 at 14:38, Andres Freund and...@anarazel.de wrote: On 2015-07-02 14:34:48 +0100, Simon Riggs wrote: This was pushed back from last CF and I haven't worked on it at all, nor will I. Pushing back again. Let's return with feedback, not move, it then.. Moving a entries

Re: [HACKERS] WALWriter active during recovery

2015-07-02 Thread Simon Riggs
On 2 July 2015 at 14:31, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Mar 5, 2015 at 5:22 PM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Dec 18, 2014 at 6:43 PM, Fujii Masao masao.fu...@gmail.com wrote: On Tue, Dec 16, 2014 at 3:51 AM, Simon Riggs si...@2ndquadrant.com wrote:

Re: [HACKERS] WALWriter active during recovery

2015-07-02 Thread Fujii Masao
On Thu, Mar 5, 2015 at 5:22 PM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Dec 18, 2014 at 6:43 PM, Fujii Masao masao.fu...@gmail.com wrote: On Tue, Dec 16, 2014 at 3:51 AM, Simon Riggs si...@2ndquadrant.com wrote: Currently, WALReceiver writes and fsyncs data it receives. Clearly, while

Re: [HACKERS] WALWriter active during recovery

2015-03-05 Thread Fujii Masao
On Thu, Dec 18, 2014 at 6:43 PM, Fujii Masao masao.fu...@gmail.com wrote: On Tue, Dec 16, 2014 at 3:51 AM, Simon Riggs si...@2ndquadrant.com wrote: Currently, WALReceiver writes and fsyncs data it receives. Clearly, while we are waiting for an fsync we aren't doing any other useful work.

Re: [HACKERS] WALWriter active during recovery

2015-02-12 Thread Michael Paquier
On Thu, Dec 18, 2014 at 6:43 PM, Fujii Masao masao.fu...@gmail.com wrote: On Tue, Dec 16, 2014 at 3:51 AM, Simon Riggs si...@2ndquadrant.com wrote: Currently, WALReceiver writes and fsyncs data it receives. Clearly, while we are waiting for an fsync we aren't doing any other useful work.

Re: [HACKERS] WALWriter active during recovery

2014-12-18 Thread Fujii Masao
On Tue, Dec 16, 2014 at 3:51 AM, Simon Riggs si...@2ndquadrant.com wrote: Currently, WALReceiver writes and fsyncs data it receives. Clearly, while we are waiting for an fsync we aren't doing any other useful work. Following patch starts WALWriter during recovery and makes it responsible for

Re: [HACKERS] WALWriter active during recovery

2014-12-17 Thread didier
Hi, On Tue, Dec 16, 2014 at 6:07 PM, Simon Riggs si...@2ndquadrant.com wrote: On 16 December 2014 at 14:12, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 12/15/2014 08:51 PM, Simon Riggs wrote: Currently, WALReceiver writes and fsyncs data it receives. Clearly, while we are waiting

Re: [HACKERS] WALWriter active during recovery

2014-12-17 Thread Simon Riggs
On 17 December 2014 at 11:27, didier did...@gmail.com wrote: If there's a fsync in progress WALReceiver will: 1- slow the fsync because its writes to the same file are grabbed by the fsync 2- stall until the end of fsync. PostgreSQL already fsyncs files while they are being written to. Are

Re: [HACKERS] WALWriter active during recovery

2014-12-17 Thread Alvaro Herrera
didier wrote: On many Linux systems it may not do that much (2.6.32 and 3.2 are bad, 3.13 is better but still it slows the fsync). If there's a fsync in progress WALReceiver will: 1- slow the fsync because its writes to the same file are grabbed by the fsync 2- stall until the end of

Re: [HACKERS] WALWriter active during recovery

2014-12-17 Thread didier
Hi On Wed, Dec 17, 2014 at 2:39 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: didier wrote: On many Linux systems it may not do that much (2.6.32 and 3.2 are bad, 3.13 is better but still it slows the fsync). If there's a fsync in progress WALReceiver will: 1- slow the fsync because

Re: [HACKERS] WALWriter active during recovery

2014-12-16 Thread Heikki Linnakangas
On 12/15/2014 08:51 PM, Simon Riggs wrote: Currently, WALReceiver writes and fsyncs data it receives. Clearly, while we are waiting for an fsync we aren't doing any other useful work. Following patch starts WALWriter during recovery and makes it responsible for fsyncing data, allowing

Re: [HACKERS] WALWriter active during recovery

2014-12-16 Thread Andres Freund
On 2014-12-16 16:12:40 +0200, Heikki Linnakangas wrote: On 12/15/2014 08:51 PM, Simon Riggs wrote: Currently, WALReceiver writes and fsyncs data it receives. Clearly, while we are waiting for an fsync we aren't doing any other useful work. Following patch starts WALWriter during recovery

Re: [HACKERS] WALWriter active during recovery

2014-12-16 Thread Simon Riggs
On 16 December 2014 at 14:12, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 12/15/2014 08:51 PM, Simon Riggs wrote: Currently, WALReceiver writes and fsyncs data it receives. Clearly, while we are waiting for an fsync we aren't doing any other useful work. Following patch starts

Re: [HACKERS] WALWriter active during recovery

2014-12-15 Thread Andres Freund
Hi, On 2014-12-15 18:51:44 +, Simon Riggs wrote: Currently, WALReceiver writes and fsyncs data it receives. Clearly, while we are waiting for an fsync we aren't doing any other useful work. Well, it can still buffer data on the network level, but there's definitely limits to that. So I