Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-06-04 Thread Andres Freund
Hi Dave, Ted, All, On 2014-05-23 16:42:47 +1000, Dave Chinner wrote: > On Tue, Apr 29, 2014 at 01:57:14AM +0200, Andres Freund wrote: > > Hi Dave, > > > > On 2014-04-29 09:47:56 +1000, Dave Chinner wrote: > > > ping? > > > > I'd replied at http://marc.info/?l=linux-mm=139730910307321=2 > > I

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-06-04 Thread Andres Freund
Hi Dave, Ted, All, On 2014-05-23 16:42:47 +1000, Dave Chinner wrote: On Tue, Apr 29, 2014 at 01:57:14AM +0200, Andres Freund wrote: Hi Dave, On 2014-04-29 09:47:56 +1000, Dave Chinner wrote: ping? I'd replied at http://marc.info/?l=linux-mmm=139730910307321w=2 I missed it,

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-05-23 Thread Dave Chinner
On Tue, Apr 29, 2014 at 01:57:14AM +0200, Andres Freund wrote: > Hi Dave, > > On 2014-04-29 09:47:56 +1000, Dave Chinner wrote: > > ping? > > I'd replied at http://marc.info/?l=linux-mm=139730910307321=2 I missed it, sorry. I've had a bit more time to look at this behaviour now and tweaked it

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-05-23 Thread Dave Chinner
On Tue, Apr 29, 2014 at 01:57:14AM +0200, Andres Freund wrote: Hi Dave, On 2014-04-29 09:47:56 +1000, Dave Chinner wrote: ping? I'd replied at http://marc.info/?l=linux-mmm=139730910307321w=2 I missed it, sorry. I've had a bit more time to look at this behaviour now and tweaked it as

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-04-28 Thread Andres Freund
Hi Dave, On 2014-04-29 09:47:56 +1000, Dave Chinner wrote: > ping? I'd replied at http://marc.info/?l=linux-mm=139730910307321=2 As an additional note: > On Wed, Apr 09, 2014 at 07:20:09PM +1000, Dave Chinner wrote: > > I'm not sure how you were generating the behaviour you reported, but > >

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-04-28 Thread Dave Chinner
ping? On Wed, Apr 09, 2014 at 07:20:09PM +1000, Dave Chinner wrote: > On Wed, Mar 26, 2014 at 08:11:13PM +0100, Andres Freund wrote: > > Hi, > > > > At LSF/MM there was a slot about postgres' problems with the kernel. Our > > top#1 concern is frequent slow read()s that happen while another

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-04-28 Thread Dave Chinner
ping? On Wed, Apr 09, 2014 at 07:20:09PM +1000, Dave Chinner wrote: On Wed, Mar 26, 2014 at 08:11:13PM +0100, Andres Freund wrote: Hi, At LSF/MM there was a slot about postgres' problems with the kernel. Our top#1 concern is frequent slow read()s that happen while another process

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-04-28 Thread Andres Freund
Hi Dave, On 2014-04-29 09:47:56 +1000, Dave Chinner wrote: ping? I'd replied at http://marc.info/?l=linux-mmm=139730910307321w=2 As an additional note: On Wed, Apr 09, 2014 at 07:20:09PM +1000, Dave Chinner wrote: I'm not sure how you were generating the behaviour you reported, but the

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-03-27 Thread Fernando Luis Vazquez Cao
(2014/03/28 0:50), Jan Kara wrote: On Wed 26-03-14 22:55:18, Andres Freund wrote: On 2014-03-26 14:41:31 -0700, Andy Lutomirski wrote: On Wed, Mar 26, 2014 at 12:11 PM, Andres Freund wrote: Hi, At LSF/MM there was a slot about postgres' problems with the kernel. Our top#1 concern is

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-03-27 Thread Jan Kara
Hello, On Wed 26-03-14 20:11:13, Andres Freund wrote: > At LSF/MM there was a slot about postgres' problems with the kernel. Our > top#1 concern is frequent slow read()s that happen while another process > calls fsync(), even though we'd be perfectly fine if that fsync() took > ages. > The

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-03-27 Thread Jan Kara
On Wed 26-03-14 22:55:18, Andres Freund wrote: > On 2014-03-26 14:41:31 -0700, Andy Lutomirski wrote: > > On Wed, Mar 26, 2014 at 12:11 PM, Andres Freund wrote: > > > Hi, > > > > > > At LSF/MM there was a slot about postgres' problems with the kernel. Our > > > top#1 concern is frequent slow

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-03-27 Thread Jan Kara
On Wed 26-03-14 22:55:18, Andres Freund wrote: On 2014-03-26 14:41:31 -0700, Andy Lutomirski wrote: On Wed, Mar 26, 2014 at 12:11 PM, Andres Freund and...@anarazel.de wrote: Hi, At LSF/MM there was a slot about postgres' problems with the kernel. Our top#1 concern is frequent slow

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-03-27 Thread Jan Kara
Hello, On Wed 26-03-14 20:11:13, Andres Freund wrote: At LSF/MM there was a slot about postgres' problems with the kernel. Our top#1 concern is frequent slow read()s that happen while another process calls fsync(), even though we'd be perfectly fine if that fsync() took ages. The

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-03-27 Thread Fernando Luis Vazquez Cao
(2014/03/28 0:50), Jan Kara wrote: On Wed 26-03-14 22:55:18, Andres Freund wrote: On 2014-03-26 14:41:31 -0700, Andy Lutomirski wrote: On Wed, Mar 26, 2014 at 12:11 PM, Andres Freund and...@anarazel.de wrote: Hi, At LSF/MM there was a slot about postgres' problems with the kernel. Our top#1

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-03-26 Thread Andy Lutomirski
On Wed, Mar 26, 2014 at 4:11 PM, Andy Lutomirski wrote: > On Wed, Mar 26, 2014 at 3:35 PM, David Lang wrote: >> On Wed, 26 Mar 2014, Andy Lutomirski wrote: >> > I'm not sure I understand the request queue stuff, but here's an idea. > The block core contains this little bit of code:

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-03-26 Thread Andy Lutomirski
On Wed, Mar 26, 2014 at 3:35 PM, David Lang wrote: > On Wed, 26 Mar 2014, Andy Lutomirski wrote: > I'm not sure I understand the request queue stuff, but here's an idea. The block core contains this little bit of code: >>> >>> >>> I haven't read enough of the code yet, to comment

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-03-26 Thread David Lang
On Wed, 26 Mar 2014, Andy Lutomirski wrote: I'm not sure I understand the request queue stuff, but here's an idea. The block core contains this little bit of code: I haven't read enough of the code yet, to comment intelligently ;) My little patch doesn't seem to help. I'm either changing

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-03-26 Thread Andy Lutomirski
On Wed, Mar 26, 2014 at 2:55 PM, Andres Freund wrote: > On 2014-03-26 14:41:31 -0700, Andy Lutomirski wrote: >> On Wed, Mar 26, 2014 at 12:11 PM, Andres Freund wrote: >> > Hi, >> > >> > At LSF/MM there was a slot about postgres' problems with the kernel. Our >> > top#1 concern is frequent slow

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-03-26 Thread Andres Freund
On 2014-03-26 14:41:31 -0700, Andy Lutomirski wrote: > On Wed, Mar 26, 2014 at 12:11 PM, Andres Freund wrote: > > Hi, > > > > At LSF/MM there was a slot about postgres' problems with the kernel. Our > > top#1 concern is frequent slow read()s that happen while another process > > calls fsync(),

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-03-26 Thread Andy Lutomirski
On Wed, Mar 26, 2014 at 12:11 PM, Andres Freund wrote: > Hi, > > At LSF/MM there was a slot about postgres' problems with the kernel. Our > top#1 concern is frequent slow read()s that happen while another process > calls fsync(), even though we'd be perfectly fine if that fsync() took > ages. >

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-03-26 Thread Andy Lutomirski
On Wed, Mar 26, 2014 at 12:11 PM, Andres Freund and...@anarazel.de wrote: Hi, At LSF/MM there was a slot about postgres' problems with the kernel. Our top#1 concern is frequent slow read()s that happen while another process calls fsync(), even though we'd be perfectly fine if that fsync()

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-03-26 Thread Andres Freund
On 2014-03-26 14:41:31 -0700, Andy Lutomirski wrote: On Wed, Mar 26, 2014 at 12:11 PM, Andres Freund and...@anarazel.de wrote: Hi, At LSF/MM there was a slot about postgres' problems with the kernel. Our top#1 concern is frequent slow read()s that happen while another process calls

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-03-26 Thread Andy Lutomirski
On Wed, Mar 26, 2014 at 2:55 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-03-26 14:41:31 -0700, Andy Lutomirski wrote: On Wed, Mar 26, 2014 at 12:11 PM, Andres Freund and...@anarazel.de wrote: Hi, At LSF/MM there was a slot about postgres' problems with the kernel. Our top#1

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-03-26 Thread David Lang
On Wed, 26 Mar 2014, Andy Lutomirski wrote: I'm not sure I understand the request queue stuff, but here's an idea. The block core contains this little bit of code: I haven't read enough of the code yet, to comment intelligently ;) My little patch doesn't seem to help. I'm either changing

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-03-26 Thread Andy Lutomirski
On Wed, Mar 26, 2014 at 3:35 PM, David Lang da...@lang.hm wrote: On Wed, 26 Mar 2014, Andy Lutomirski wrote: I'm not sure I understand the request queue stuff, but here's an idea. The block core contains this little bit of code: I haven't read enough of the code yet, to comment

Re: [Lsf] Postgresql performance problems with IO latency, especially during fsync()

2014-03-26 Thread Andy Lutomirski
On Wed, Mar 26, 2014 at 4:11 PM, Andy Lutomirski l...@amacapital.net wrote: On Wed, Mar 26, 2014 at 3:35 PM, David Lang da...@lang.hm wrote: On Wed, 26 Mar 2014, Andy Lutomirski wrote: I'm not sure I understand the request queue stuff, but here's an idea. The block core contains this little