Re: [HACKERS] Performance optimization of btree binary search

2013-12-09 Thread Peter Geoghegan
On Fri, Dec 6, 2013 at 4:53 PM, Peter Geoghegan p...@heroku.com wrote: I had considered that something like Intel Speedstep technology had a role here, but I'm pretty sure it steps up very aggressively when things are CPU bound - I tested that against a Core 2 Duo desktop a couple of years

Re: [HACKERS] Performance optimization of btree binary search

2013-12-06 Thread Peter Geoghegan
On Thu, Dec 5, 2013 at 2:19 AM, Peter Geoghegan p...@heroku.com wrote: I did a relatively short variant 'insert' pgbench-tools benchmark, with a serial primary index created on the pgbench_history table. Results: http://postgres-benchmarks.s3-website-us-east-1.amazonaws.com/insert/ I saw

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Peter Geoghegan
On Wed, Dec 4, 2013 at 5:28 PM, Peter Geoghegan p...@heroku.com wrote: I'm also curious about the impact on insertion into primary key indexes. Presently, we hold an exclusive buffer lock for the duration of a couple of operations when checkUnique != UNIQUE_CHECK_NO. _bt_binsrch() is one such

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Andres Freund
On 2013-12-04 18:48:44 -0500, Robert Haas wrote: * When a type narrower than Datum is stored in a Datum, we place it in the * low-order bits and are careful that the DatumGetXXX macro for it discards * the unused high-order bits (as opposed to, say, assuming they are zero). * This is

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Heikki Linnakangas
On 12/05/2013 07:30 AM, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: On Wed, 2013-12-04 at 20:27 -0500, Tom Lane wrote: Lazy people? I'm not in a hurry to drop it; it's not costing us much to just sit there, other than in this connection which we see how to fix. Actually, I

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-12-04 18:48:44 -0500, Robert Haas wrote: And record_image_eq does a rather elaborate dance around here, calling the appropriate GET_x_BYTES macro depending on the type-width. If we can really count on the high-order bits to be zero, that's

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Andres Freund
On 2013-12-05 08:58:55 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: I don't think we can get rid of that dance in record_image_eq - it very well could used on records originally generated when those bits haven't been guaranteed to be zeroed. No, you're failing to

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-12-05 08:58:55 -0500, Tom Lane wrote: I'm a bit worried that somebody, particularly third-party code, might've sloppily written return foo in a V1 function when return Int32GetDatum(foo) would be correct. In that case, the resultant Datum

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Andres Freund
On 2013-12-05 10:02:56 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-12-05 08:58:55 -0500, Tom Lane wrote: I'm a bit worried that somebody, particularly third-party code, might've sloppily written return foo in a V1 function when return Int32GetDatum(foo)

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: I was actually thinking about making Datum (and some other types we have) structs or unions. Currently it's far, far to easy to mix them. We throw away pretty much all of the little typesafety C has by typedef'ing them to integral types with lots of

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Andres Freund
On 2013-12-05 10:34:16 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: I was actually thinking about making Datum (and some other types we have) structs or unions. Currently it's far, far to easy to mix them. We throw away pretty much all of the little typesafety C

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-12-05 10:34:16 -0500, Tom Lane wrote: In any case, the number of bugs I can remember that such a thing would've prevented is negligible. Cases talked about upthread, where a plain datatype is returned as a Datum instead of using

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Peter Eisentraut
On Thu, 2013-12-05 at 15:29 +0200, Heikki Linnakangas wrote: It happens to me about 75% of the time when I write a new C function. These days, I usually realize it pretty quickly. I wonder how easy it would be to make the compiler produce a warning about it. Or issue a warning in

[HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Peter Geoghegan
Having nothing better to do over the holiday weekend, I decided to pursue a number of ideas for improving performance that I thought about a long time ago. These include: * Pre-fetching list node pointers. This looks to be moderately promising, but I'm certainly not going to be the one to land

Re: [HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Tom Lane
Peter Geoghegan p...@heroku.com writes: I guess I could write a proper patch to have code setting up a scankey also set a flag that indicated that it was acceptable to assume that the special built-in comparator would do fine. ... I'd be happy with a scheme with only one built-in comparator,

Re: [HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Robert Haas
On Wed, Dec 4, 2013 at 4:28 PM, Tom Lane t...@sss.pgh.pa.us wrote: Peter Geoghegan p...@heroku.com writes: I guess I could write a proper patch to have code setting up a scankey also set a flag that indicated that it was acceptable to assume that the special built-in comparator would do fine.

Re: [HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Peter Geoghegan
On Wed, Dec 4, 2013 at 1:59 PM, Robert Haas robertmh...@gmail.com wrote: Yeah, I think if we can make something like this work, it would be neat-o. Getting this working for int4 would be a good win, as Peter says, but getting it working for both int4 and int8 with the same code would be a

Re: [HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Tom Lane
Peter Geoghegan p...@heroku.com writes: On Wed, Dec 4, 2013 at 1:59 PM, Robert Haas robertmh...@gmail.com wrote: Yeah, I think if we can make something like this work, it would be neat-o. Getting this working for int4 would be a good win, as Peter says, but getting it working for both int4

Re: [HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Robert Haas
On Wed, Dec 4, 2013 at 6:33 PM, Peter Geoghegan p...@heroku.com wrote: On Wed, Dec 4, 2013 at 1:59 PM, Robert Haas robertmh...@gmail.com wrote: Yeah, I think if we can make something like this work, it would be neat-o. Getting this working for int4 would be a good win, as Peter says, but

Re: [HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Hmm. And yet, there's this: * When a type narrower than Datum is stored in a Datum, we place it in the * low-order bits and are careful that the DatumGetXXX macro for it discards * the unused high-order bits (as opposed to, say, assuming they are

Re: [HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Tom Lane
I wrote: Yeah, that's another thing we could simplify if we fixed this problem at the source. I think these decisions date from a time when we still cared about the speed of fmgr_oldstyle. BTW, the text you're quoting is from 2007, but it's just documenting behavior that's mostly a lot older.

Re: [HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Robert Haas
On Wed, Dec 4, 2013 at 6:56 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Hmm. And yet, there's this: * When a type narrower than Datum is stored in a Datum, we place it in the * low-order bits and are careful that the DatumGetXXX macro for it discards

Re: [HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Peter Eisentraut
On Wed, 2013-12-04 at 19:45 -0500, Robert Haas wrote: On Wed, Dec 4, 2013 at 6:56 PM, Tom Lane t...@sss.pgh.pa.us wrote: Yeah, that's another thing we could simplify if we fixed this problem at the source. I think these decisions date from a time when we still cared about the speed of

Re: [HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On Wed, 2013-12-04 at 19:45 -0500, Robert Haas wrote: On Wed, Dec 4, 2013 at 6:56 PM, Tom Lane t...@sss.pgh.pa.us wrote: Yeah, that's another thing we could simplify if we fixed this problem at the source. I think these decisions date from a time when

Re: [HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Peter Geoghegan
On Wed, Dec 4, 2013 at 12:58 PM, Peter Geoghegan p...@heroku.com wrote: I'm kind of curious as to what this benchmark would like like on a server with many more cores. I'm also curious about the impact on insertion into primary key indexes. Presently, we hold an exclusive buffer lock for the

Re: [HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Peter Geoghegan
On Wed, Dec 4, 2013 at 5:28 PM, Peter Geoghegan p...@heroku.com wrote: I'm also curious about the impact on insertion into primary key indexes. Presently, we hold an exclusive buffer lock for the duration of a couple of operations when checkUnique != UNIQUE_CHECK_NO. _bt_binsrch() is one such

Re: [HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Peter Eisentraut
On Wed, 2013-12-04 at 20:27 -0500, Tom Lane wrote: Lazy people? I'm not in a hurry to drop it; it's not costing us much to just sit there, other than in this connection which we see how to fix. Actually, I think it probably costs a fair portion of extension authors when their initial code

Re: [HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On Wed, 2013-12-04 at 20:27 -0500, Tom Lane wrote: Lazy people? I'm not in a hurry to drop it; it's not costing us much to just sit there, other than in this connection which we see how to fix. Actually, I think it probably costs a fair portion of