Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-04 Thread Bruce Momjian
On Fri, Oct 3, 2014 at 08:04:08PM -0400, Bruce Momjian wrote: On Sat, Oct 4, 2014 at 01:57:01AM +0200, Andres Freund wrote: On 2014-10-03 19:54:36 -0400, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: Agreeed. Also, reality check --- we can't change postgresql.conf easily

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-04 Thread Andres Freund
On 2014-10-04 11:07:13 -0400, Bruce Momjian wrote: On Fri, Oct 3, 2014 at 08:04:08PM -0400, Bruce Momjian wrote: On Sat, Oct 4, 2014 at 01:57:01AM +0200, Andres Freund wrote: On 2014-10-03 19:54:36 -0400, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: Agreeed. Also,

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-04 Thread Bruce Momjian
On Sat, Oct 4, 2014 at 05:08:17PM +0200, Andres Freund wrote: On 2014-10-04 11:07:13 -0400, Bruce Momjian wrote: On Fri, Oct 3, 2014 at 08:04:08PM -0400, Bruce Momjian wrote: On Sat, Oct 4, 2014 at 01:57:01AM +0200, Andres Freund wrote: On 2014-10-03 19:54:36 -0400, Tom Lane wrote:

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Andres Freund
On 2014-10-02 20:08:33 -0400, Greg Smith wrote: I did a fair dive into double-checking the decision to just leave xloginsert_locks fixed at 8 for 9.4. My conclusion: good call, move along. Further improvements beyond what the 8-way split gives sure are possible. But my guess from chasing

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Gregory Smith
On 10/3/14, 8:26 AM, Andres Freund wrote: #define NUM_XLOGINSERT_LOCKS 1 tps = 52.711939 (including connections establishing) #define NUM_XLOGINSERT_LOCKS 8 tps = 286.496054 (including connections establishing) #define NUM_XLOGINSERT_LOCKS 16 tps = 346.113313 (including connections

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Andres Freund
On 2014-10-03 10:07:39 -0400, Gregory Smith wrote: On 10/3/14, 8:26 AM, Andres Freund wrote: #define NUM_XLOGINSERT_LOCKS 1 tps = 52.711939 (including connections establishing) #define NUM_XLOGINSERT_LOCKS 8 tps = 286.496054 (including connections establishing) #define NUM_XLOGINSERT_LOCKS

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Bruce Momjian
On Fri, Oct 3, 2014 at 04:11:30PM +0200, Andres Freund wrote: On 2014-10-03 10:07:39 -0400, Gregory Smith wrote: On 10/3/14, 8:26 AM, Andres Freund wrote: #define NUM_XLOGINSERT_LOCKS 1 tps = 52.711939 (including connections establishing) #define NUM_XLOGINSERT_LOCKS 8 tps =

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Arthur Silva
On Fri, Oct 3, 2014 at 1:40 PM, Bruce Momjian br...@momjian.us wrote: On Fri, Oct 3, 2014 at 04:11:30PM +0200, Andres Freund wrote: On 2014-10-03 10:07:39 -0400, Gregory Smith wrote: On 10/3/14, 8:26 AM, Andres Freund wrote: #define NUM_XLOGINSERT_LOCKS 1 tps = 52.711939 (including

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Bruce Momjian
On Fri, Oct 3, 2014 at 03:00:56PM -0300, Arthur Silva wrote: I remember Informix had a setting that had no description except try different values to see if it helps performance --- we don't want to do that. What if we emit a server message if the setting is too low? That's

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Bruce Momjian
On Fri, Oct 3, 2014 at 02:07:45PM -0400, Bruce Momjian wrote: On Fri, Oct 3, 2014 at 03:00:56PM -0300, Arthur Silva wrote: I remember Informix had a setting that had no description except try different values to see if it helps performance --- we don't want to do that.

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Arthur Silva
On Fri, Oct 3, 2014 at 3:10 PM, Bruce Momjian br...@momjian.us wrote: On Fri, Oct 3, 2014 at 02:07:45PM -0400, Bruce Momjian wrote: On Fri, Oct 3, 2014 at 03:00:56PM -0300, Arthur Silva wrote: I remember Informix had a setting that had no description except try different

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: On Fri, Oct 3, 2014 at 03:00:56PM -0300, Arthur Silva wrote: Not all GUC need to be straight forward to tune. If the gains are worthy I don't see any reason not to have it. Every GUC add complexity to the system because people have to understand it to

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Bruce Momjian
On Fri, Oct 3, 2014 at 03:30:35PM -0300, Arthur Silva wrote: Every GUC add complexity to the system because people have to understand it to know if they should tune it. No GUC is zero-cost. Please see my blog post about the cost of adding GUCs:

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Heikki Linnakangas
On 10/03/2014 09:42 PM, Bruce Momjian wrote: On Fri, Oct 3, 2014 at 03:30:35PM -0300, Arthur Silva wrote: Every GUC add complexity to the system because people have to understand it to know if they should tune it. No GUC is zero-cost. Please see my blog post about the cost

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Robert Haas
On Fri, Oct 3, 2014 at 2:49 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: I stand by my decision to make it a #define, at least until someone voices their objection in the form of a documentation patch. I think that's exactly right. If we knew users should tune this, then we might be

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Merlin Moncure
On Fri, Oct 3, 2014 at 2:20 PM, Robert Haas robertmh...@gmail.com wrote: On Fri, Oct 3, 2014 at 2:49 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: I stand by my decision to make it a #define, at least until someone voices their objection in the form of a documentation patch. I think

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Andres Freund
On 2014-10-03 12:40:21 -0400, Bruce Momjian wrote: On Fri, Oct 3, 2014 at 04:11:30PM +0200, Andres Freund wrote: On 2014-10-03 10:07:39 -0400, Gregory Smith wrote: On 10/3/14, 8:26 AM, Andres Freund wrote: #define NUM_XLOGINSERT_LOCKS 1 tps = 52.711939 (including connections

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Andres Freund
On 2014-10-03 17:55:19 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-10-03 12:40:21 -0400, Bruce Momjian wrote: Well, I think the issue is that having a GUC that can't reasonably be tuned by 95% of our users is nearly useless. Few users are going to run

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-10-03 12:40:21 -0400, Bruce Momjian wrote: Well, I think the issue is that having a GUC that can't reasonably be tuned by 95% of our users is nearly useless. Few users are going to run benchmarks to see what the optimal value is. It's

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Bruce Momjian
On Fri, Oct 3, 2014 at 11:58:14PM +0200, Andres Freund wrote: On 2014-10-03 17:55:19 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-10-03 12:40:21 -0400, Bruce Momjian wrote: Well, I think the issue is that having a GUC that can't reasonably be tuned by

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Andres Freund
On 2014-10-03 18:08:56 -0400, Bruce Momjian wrote: On Fri, Oct 3, 2014 at 11:58:14PM +0200, Andres Freund wrote: On 2014-10-03 17:55:19 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-10-03 12:40:21 -0400, Bruce Momjian wrote: Well, I think the issue is

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Bruce Momjian
On Sat, Oct 4, 2014 at 12:13:00AM +0200, Andres Freund wrote: Do we really want to expose a setting a few of us _might_ ask customers to change? They also will try that themselves. Our customers aren't a horde of dumb people. Some of them are willing to try things if they hit scalability

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Andres Freund
On 2014-10-03 18:16:28 -0400, Bruce Momjian wrote: On Sat, Oct 4, 2014 at 12:13:00AM +0200, Andres Freund wrote: Do we really want to expose a setting a few of us _might_ ask customers to change? They also will try that themselves. Our customers aren't a horde of dumb people. Some

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Mark Kirkwood
On 04/10/14 11:21, Andres Freund wrote: On 2014-10-03 18:16:28 -0400, Bruce Momjian wrote: On Sat, Oct 4, 2014 at 12:13:00AM +0200, Andres Freund wrote: Do we really want to expose a setting a few of us _might_ ask customers to change? They also will try that themselves. Our customers

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Bruce Momjian
On Sat, Oct 4, 2014 at 12:00:36PM +1300, Mark Kirkwood wrote: I don't think we can offer absolutely accurate tuning advice, but I'm sure we can give some guidance. Let me try. +1 I think it is ok to document our reason for providing the new GUC - along with that fact that it is a new

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Gregory Smith
On 10/3/14, 10:11 AM, Andres Freund wrote: So 25% performance on a relatively small machine improvements aren't worth a GUC? That are likely to be larger on a bigger machine? I utterly fail to see why that's a service to our users. I didn't say that. I said I don't think they're worth a GUC

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Bruce Momjian
On Fri, Oct 3, 2014 at 07:39:25PM -0400, Greg Smith wrote: I do not disagree with you fundamentally here: this *is* worth refining further, for sure, and the gains might be even greater if that keeps going. My guess is just that the Postgres community would not get a net benefit chasing that

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: Agreeed. Also, reality check --- we can't change postgresql.conf easily without an initdb, and I think our last 9.4 initdb is going to be 9.4beta3, which is going to be packaged on Monday. Good point: independently of all else, it's a bit late to be

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Andres Freund
On 2014-10-03 19:54:36 -0400, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: Agreeed. Also, reality check --- we can't change postgresql.conf easily without an initdb, and I think our last 9.4 initdb is going to be 9.4beta3, which is going to be packaged on Monday. Good point:

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Bruce Momjian
On Sat, Oct 4, 2014 at 01:57:01AM +0200, Andres Freund wrote: On 2014-10-03 19:54:36 -0400, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: Agreeed. Also, reality check --- we can't change postgresql.conf easily without an initdb, and I think our last 9.4 initdb is going to be

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Josh Berkus
On 10/03/2014 05:04 PM, Bruce Momjian wrote: On Sat, Oct 4, 2014 at 01:57:01AM +0200, Andres Freund wrote: On 2014-10-03 19:54:36 -0400, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: Agreeed. Also, reality check --- we can't change postgresql.conf easily without an initdb, and I

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-10-03 19:54:36 -0400, Tom Lane wrote: Good point: independently of all else, it's a bit late to be adding new features to 9.4. This is getting absurd. The feature was there three days ago. Well, an undocumented feature isn't a feature.

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Gregory Smith
On 10/3/14, 5:23 PM, Andres Freund wrote: How are we ever going to be able to tune it further without feedback from actual production usage? With improved targeted synthetic test cases that isolate the bottleneck to where it's really obvious, way more obvious than it will ever be in

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Mark Kirkwood
On 04/10/14 12:10, Bruce Momjian wrote: On Sat, Oct 4, 2014 at 12:00:36PM +1300, Mark Kirkwood wrote: I don't think we can offer absolutely accurate tuning advice, but I'm sure we can give some guidance. Let me try. +1 I think it is ok to document our reason for providing the new GUC -

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-02 Thread Peter Geoghegan
On Thu, Oct 2, 2014 at 5:08 PM, Greg Smith greg.sm...@crunchydatasolutions.com wrote: When 9.4 is already giving a more than 100% gain on this targeted test case, I can't see that chasing after maybe an extra 10% is worth having yet another GUC around. Especially when it will probably take