Re: [HACKERS] Should TIDs be typbyval = FLOAT8PASSBYVAL to speed up CREATE INDEX CONCURRENTLY?

2015-12-17 Thread Peter Geoghegan
On Thu, Dec 17, 2015 at 9:29 AM, Corey Huinker wrote: > My apologies to Peter and all the Roberts, I wasn't able to set up a test > fast enough. Glad it got committed. I don't use the term "slam-dunk" casually. :-) This was the first time I ever referred to a patch of mine that way. It doesn't h

Re: [HACKERS] Should TIDs be typbyval = FLOAT8PASSBYVAL to speed up CREATE INDEX CONCURRENTLY?

2015-12-17 Thread Corey Huinker
On Wed, Dec 16, 2015 at 4:24 PM, Peter Geoghegan wrote: > On Wed, Dec 16, 2015 at 12:28 PM, Robert Haas > wrote: > >> I seem to be able to produce these sorting patches at a much greater > >> rate than they can be committed, in part because Robert is the only > >> one that ever reviews them, and

Re: [HACKERS] Should TIDs be typbyval = FLOAT8PASSBYVAL to speed up CREATE INDEX CONCURRENTLY?

2015-12-16 Thread Peter Geoghegan
On Wed, Dec 16, 2015 at 12:28 PM, Robert Haas wrote: >> I seem to be able to produce these sorting patches at a much greater >> rate than they can be committed, in part because Robert is the only >> one that ever reviews them, and he is only one person. > > I object to that vicious slander. I am

Re: [HACKERS] Should TIDs be typbyval = FLOAT8PASSBYVAL to speed up CREATE INDEX CONCURRENTLY?

2015-12-16 Thread Robert Haas
On Tue, Nov 17, 2015 at 6:50 PM, Peter Geoghegan wrote: > On Tue, Nov 17, 2015 at 12:53 AM, Simon Riggs wrote: >> Short and sweet! Looks good. > > Thanks. > >> I would be inclined to add more comments to explain it, these things have a >> habit of being forgotten. > > I'm not sure what additional

Re: [HACKERS] Should TIDs be typbyval = FLOAT8PASSBYVAL to speed up CREATE INDEX CONCURRENTLY?

2015-12-12 Thread Jeff Janes
On Sat, Dec 12, 2015 at 9:48 AM, Corey Huinker wrote: > > > What, if any, other load should be placed on the underlying table during the > test? > > I ask because CIC statements that run in seconds on our staging machine can > take many hours on our production machine, when most of the access is j

Re: [HACKERS] Should TIDs be typbyval = FLOAT8PASSBYVAL to speed up CREATE INDEX CONCURRENTLY?

2015-12-12 Thread Peter Geoghegan
On Fri, Dec 11, 2015 at 9:13 AM, Robert Haas wrote: > Also, I'd be in favor of you updating the patch to reflect the > comments from Tom and Simon on November 17th. Attached revision: * Has more worked out comments on encoding, per Simon's request. * Uses Tom's preferred formulation for encodin

Re: [HACKERS] Should TIDs be typbyval = FLOAT8PASSBYVAL to speed up CREATE INDEX CONCURRENTLY?

2015-12-12 Thread Peter Geoghegan
On Sat, Dec 12, 2015 at 9:48 AM, Corey Huinker wrote: > I ask because CIC statements that run in seconds on our staging machine can > take many hours on our production machine, when most of the access is just > reads, though those reads may have been part of a larger transaction that > did updates

Re: [HACKERS] Should TIDs be typbyval = FLOAT8PASSBYVAL to speed up CREATE INDEX CONCURRENTLY?

2015-12-12 Thread Corey Huinker
On Fri, Dec 11, 2015 at 5:35 PM, Peter Geoghegan wrote: > On Fri, Dec 11, 2015 at 2:26 PM, Corey Huinker > wrote: > > Sure, the machine we called "ninefivealpha", which incidentally, failed > to > > find a single bug in alpha2 thru beta2, is currently idle, and concurrent > > index creation time

Re: [HACKERS] Should TIDs be typbyval = FLOAT8PASSBYVAL to speed up CREATE INDEX CONCURRENTLY?

2015-12-11 Thread Peter Geoghegan
On Fri, Dec 11, 2015 at 2:26 PM, Corey Huinker wrote: > Sure, the machine we called "ninefivealpha", which incidentally, failed to > find a single bug in alpha2 thru beta2, is currently idle, and concurrent > index creation times are a bugbear around these parts. Can somebody, either > in this thr

Re: [HACKERS] Should TIDs be typbyval = FLOAT8PASSBYVAL to speed up CREATE INDEX CONCURRENTLY?

2015-12-11 Thread Corey Huinker
On Fri, Dec 11, 2015 at 12:13 PM, Robert Haas wrote: > On Wed, Dec 9, 2015 at 8:16 PM, Peter Geoghegan wrote: > > On Tue, Nov 17, 2015 at 7:33 PM, Corey Huinker > wrote: > >> I'm willing, but I'm too new to the codebase to be an effective reviewer > >> (without guidance). The one thing I can of

Re: [HACKERS] Should TIDs be typbyval = FLOAT8PASSBYVAL to speed up CREATE INDEX CONCURRENTLY?

2015-12-11 Thread Robert Haas
On Wed, Dec 9, 2015 at 8:16 PM, Peter Geoghegan wrote: > On Tue, Nov 17, 2015 at 7:33 PM, Corey Huinker > wrote: >> I'm willing, but I'm too new to the codebase to be an effective reviewer >> (without guidance). The one thing I can offer in the mean time is this: my >> company/client nearly alwa

Re: [HACKERS] Should TIDs be typbyval = FLOAT8PASSBYVAL to speed up CREATE INDEX CONCURRENTLY?

2015-12-09 Thread Peter Geoghegan
On Tue, Nov 17, 2015 at 7:33 PM, Corey Huinker wrote: > I'm willing, but I'm too new to the codebase to be an effective reviewer > (without guidance). The one thing I can offer in the mean time is this: my > company/client nearly always has a few spare AWS machines on the largish > side where I ca

Re: [HACKERS] Should TIDs be typbyval = FLOAT8PASSBYVAL to speed up CREATE INDEX CONCURRENTLY?

2015-11-17 Thread Corey Huinker
On Tue, Nov 17, 2015 at 7:28 PM, Michael Paquier wrote: > On Wed, Nov 18, 2015 at 8:50 AM, Peter Geoghegan wrote: > > I seem to be able to produce these sorting patches at a much greater > > rate than they can be committed, in part because Robert is the only > > one that ever reviews them, and he

Re: [HACKERS] Should TIDs be typbyval = FLOAT8PASSBYVAL to speed up CREATE INDEX CONCURRENTLY?

2015-11-17 Thread Michael Paquier
On Wed, Nov 18, 2015 at 8:50 AM, Peter Geoghegan wrote: > I seem to be able to produce these sorting patches at a much greater > rate than they can be committed, in part because Robert is the only > one that ever reviews them, and he is only one person. Since you think > the patch is good work, per

Re: [HACKERS] Should TIDs be typbyval = FLOAT8PASSBYVAL to speed up CREATE INDEX CONCURRENTLY?

2015-11-17 Thread Peter Geoghegan
On Tue, Nov 17, 2015 at 12:53 AM, Simon Riggs wrote: > Short and sweet! Looks good. Thanks. > I would be inclined to add more comments to explain it, these things have a > habit of being forgotten. I'm not sure what additional detail I can add. I seem to be able to produce these sorting patche

Re: [HACKERS] Should TIDs be typbyval = FLOAT8PASSBYVAL to speed up CREATE INDEX CONCURRENTLY?

2015-11-17 Thread Peter Geoghegan
On Tue, Nov 17, 2015 at 7:54 AM, Tom Lane wrote: > I think this might do the wrong thing with block numbers above 0x8000 > and/or offset numbers above 0x8000. I'd be more comfortable about it if > + encoded = ((int64) block << 16) | offset; > were > + encoded = ((uint64) block <<

Re: [HACKERS] Should TIDs be typbyval = FLOAT8PASSBYVAL to speed up CREATE INDEX CONCURRENTLY?

2015-11-17 Thread Tom Lane
Peter Geoghegan writes: >>> Might be better to hack a special case right there (ie, embed TIDs into >>> int8s and sort the int8s) rather than try to change the type's SQL >>> declaration. > I suggested to someone else that he take a look at this as a project, > but I guess he was busy too. I deci

Re: [HACKERS] Should TIDs be typbyval = FLOAT8PASSBYVAL to speed up CREATE INDEX CONCURRENTLY?

2015-11-17 Thread Simon Riggs
On 17 November 2015 at 00:52, Peter Geoghegan wrote: > This patch does seem like a slam dunk, even if I do say so myself Short and sweet! Looks good. I would be inclined to add more comments to explain it, these things have a habit of being forgotten. -- Simon Riggshttp://ww

Re: [HACKERS] Should TIDs be typbyval = FLOAT8PASSBYVAL to speed up CREATE INDEX CONCURRENTLY?

2015-11-16 Thread Peter Geoghegan
On Mon, Sep 7, 2015 at 9:20 PM, Peter Geoghegan wrote: >>> This matters because a major cost during CREATE INDEX CONCURRENTLY is >>> a TID-based datum sort (this is probably most of the cost over and >>> above a conventional CREATE INDEX). >> >> Might be better to hack a special case right there (

Re: [HACKERS] Should TIDs be typbyval = FLOAT8PASSBYVAL to speed up CREATE INDEX CONCURRENTLY?

2015-09-07 Thread Peter Geoghegan
On Mon, Sep 7, 2015 at 9:03 PM, Tom Lane wrote: > I'm not sure that it would be just "a little work" --- I suspect that > the idea that pass-by-val types are 1, 2, 4, or 8 bytes is embedded in > a fair number of places, including alignment macros in which any added > complexity would have a large

Re: [HACKERS] Should TIDs be typbyval = FLOAT8PASSBYVAL to speed up CREATE INDEX CONCURRENTLY?

2015-09-07 Thread Tom Lane
Peter Geoghegan writes: > I noticed that the TID type is cataloged as typbyval = f, despite the > fact that it is 6 bytes, and so could be made typbyval = t on 64-bit > platforms (i.e. typbyval = FLOAT8PASSBYVAL) with a little work. I'm not sure that it would be just "a little work" --- I suspect

[HACKERS] Should TIDs be typbyval = FLOAT8PASSBYVAL to speed up CREATE INDEX CONCURRENTLY?

2015-09-07 Thread Peter Geoghegan
I noticed that the TID type is cataloged as typbyval = f, despite the fact that it is 6 bytes, and so could be made typbyval = t on 64-bit platforms (i.e. typbyval = FLOAT8PASSBYVAL) with a little work. This matters because a major cost during CREATE INDEX CONCURRENTLY is a TID-based datum sort (t