Re: [HACKERS] Do you know the reason for increased max latency due to xlog scaling?

2014-02-19 Thread MauMau
From: Jeff Janes jeff.ja...@gmail.com I thought that this was the point I was making, not the point I was missing. You have the same hard drives you had before, but now due to a software improvement you are cramming 5 times more stuff through them. Yeah, you will get bigger latency spikes. Why

Re: [HACKERS] Do you know the reason for increased max latency due to xlog scaling?

2014-02-18 Thread MauMau
From: Andres Freund and...@2ndquadrant.com On 2014-02-18 01:35:52 +0900, MauMau wrote: For example, please see the max latencies of test set 2 (PG 9.3) and test set 4 (xlog scaling with padding). They are 207.359 and 1219.422 respectively. The throughput is of course greatly improved, but I

Re: [HACKERS] Do you know the reason for increased max latency due to xlog scaling?

2014-02-18 Thread Andres Freund
On 2014-02-18 20:49:06 +0900, MauMau wrote: From: Andres Freund and...@2ndquadrant.com On 2014-02-18 01:35:52 +0900, MauMau wrote: For example, please see the max latencies of test set 2 (PG 9.3) and test set 4 (xlog scaling with padding). They are 207.359 and 1219.422 respectively. The

Re: [HACKERS] Do you know the reason for increased max latency due to xlog scaling?

2014-02-18 Thread Jeff Janes
On Tue, Feb 18, 2014 at 3:49 AM, MauMau maumau...@gmail.com wrote: From: Andres Freund and...@2ndquadrant.com On 2014-02-18 01:35:52 +0900, MauMau wrote: For example, please see the max latencies of test set 2 (PG 9.3) and test set 4 (xlog scaling with padding). They are 207.359 and

Re: [HACKERS] Do you know the reason for increased max latency due to xlog scaling?

2014-02-18 Thread Heikki Linnakangas
On 02/18/2014 06:27 PM, Jeff Janes wrote: On Tue, Feb 18, 2014 at 3:49 AM, MauMau maumau...@gmail.com wrote: --- or in other words, greater variance in response times. With my simple understanding, that sounds like a problem for response-sensitive users. If you need the throughput provided

Re: [HACKERS] Do you know the reason for increased max latency due to xlog scaling?

2014-02-18 Thread Andres Freund
On 2014-02-18 19:12:32 +0200, Heikki Linnakangas wrote: You're missing MauMau's point. In essence, he's comparing two systems with the same number of clients, issuing queries as fast as they can, and one can do 2000 TPS while the other one can do 1 TPS. You would expect the

Re: [HACKERS] Do you know the reason for increased max latency due to xlog scaling?

2014-02-18 Thread Heikki Linnakangas
On 02/18/2014 10:51 PM, Andres Freund wrote: On 2014-02-18 19:12:32 +0200, Heikki Linnakangas wrote: Yeah, I'm pretty sure that's because of the extra checkpoints. If you look at the individual test graphs, there are clear spikes in latency, but the latency is otherwise small. With a higher

Re: [HACKERS] Do you know the reason for increased max latency due to xlog scaling?

2014-02-18 Thread Andres Freund
On 2014-02-18 23:01:08 +0200, Heikki Linnakangas wrote: On 02/18/2014 10:51 PM, Andres Freund wrote: On 2014-02-18 19:12:32 +0200, Heikki Linnakangas wrote: Yeah, I'm pretty sure that's because of the extra checkpoints. If you look at the individual test graphs, there are clear spikes in

Re: [HACKERS] Do you know the reason for increased max latency due to xlog scaling?

2014-02-18 Thread Jeff Janes
On Tue, Feb 18, 2014 at 9:12 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 02/18/2014 06:27 PM, Jeff Janes wrote: On Tue, Feb 18, 2014 at 3:49 AM, MauMau maumau...@gmail.com wrote: --- or in other words, greater variance in response times. With my simple understanding, that

[HACKERS] Do you know the reason for increased max latency due to xlog scaling?

2014-02-17 Thread MauMau
Hello Heikki san, I'm excited about your great work, xlog scaling. I'm looking forward to the release of 9.4. Please let me ask you about your performance data on the page: http://hlinnaka.iki.fi/xloginsert-scaling/padding/ I'm worried about the big increase in max latency. Do you know

Re: [HACKERS] Do you know the reason for increased max latency due to xlog scaling?

2014-02-17 Thread Andres Freund
Hi, On 2014-02-18 00:43:54 +0900, MauMau wrote: Please let me ask you about your performance data on the page: http://hlinnaka.iki.fi/xloginsert-scaling/padding/ I'm worried about the big increase in max latency. Do you know the cause? More frequent checkpoints caused by increased WAL

Re: [HACKERS] Do you know the reason for increased max latency due to xlog scaling?

2014-02-17 Thread MauMau
From: Andres Freund and...@2ndquadrant.com On 2014-02-18 00:43:54 +0900, MauMau wrote: I'm worried about the big increase in max latency. Do you know the cause? More frequent checkpoints caused by increased WAL volume thanks to enhanced performance? I don't see much evidence of increased

Re: [HACKERS] Do you know the reason for increased max latency due to xlog scaling?

2014-02-17 Thread Andres Freund
On 2014-02-18 01:35:52 +0900, MauMau wrote: From: Andres Freund and...@2ndquadrant.com On 2014-02-18 00:43:54 +0900, MauMau wrote: I'm worried about the big increase in max latency. Do you know the cause? More frequent checkpoints caused by increased WAL volume thanks to enhanced