Re: [HACKERS] Autonomous Transaction (WIP)

2014-04-08 Thread Craig Ringer
On 04/09/2014 02:44 PM, Pavan Deolasee wrote: > On Wed, Apr 9, 2014 at 11:03 AM, Rajeev rastogi > mailto:rajeev.rast...@huawei.com>> wrote: > > > Though autonomous transaction uses mixed approach of sub-transaction > as well as main > transaction, transaction state of autonomous trans

Re: [HACKERS] Autonomous Transaction (WIP)

2014-04-08 Thread Hannu Krosing
On 04/09/2014 08:44 AM, Pavan Deolasee wrote: > On Wed, Apr 9, 2014 at 11:03 AM, Rajeev rastogi > mailto:rajeev.rast...@huawei.com>> wrote: > > > Though autonomous transaction uses mixed approach of > sub-transaction as well as main > transaction, transaction state of autonomous transac

Re: [HACKERS] Autonomous Transaction (WIP)

2014-04-08 Thread Pavan Deolasee
On Wed, Apr 9, 2014 at 11:03 AM, Rajeev rastogi wrote: > > Though autonomous transaction uses mixed approach of sub-transaction as > well as main > transaction, transaction state of autonomous transaction is handled > independently. > > Whenever I was asked to have a look at implementing this feat

[HACKERS] Pointer to structure in ECPG

2014-04-08 Thread Ashutosh Bapat
Hi All, I tried to use pointer to array to fetch results of a query. The test case test_select.pgc is attached. Changes specific to one's environment are needed before that test can be tried. Otherwise, you may try file pointer_to_struct.pgc in the patch attached, by putting it ecpg/test directory.

Re: default opclass for jsonb (was Re: [HACKERS] Call for GIST/GIN/SP-GIST opclass documentation)

2014-04-08 Thread Heikki Linnakangas
On 04/09/2014 01:18 AM, Andrew Dunstan wrote: On 04/08/2014 05:57 PM, Peter Geoghegan wrote: On Tue, Apr 8, 2014 at 2:46 PM, Tom Lane wrote: Well, let me see if I understand the situation correctly: * jsonb_ops supports more operators * jsonb_hash_ops produces smaller, better-performing ind

Re: [HACKERS] Problem with displaying "wide" tables in psql

2014-04-08 Thread Sergey Muraviov
Hi. How can I pass or set the value of pset variable for an regression test? The code with ioctl was copied from print_aligned_text function, which has a long history. 43ee2282 (Bruce Momjian 2008-05-16 16:59:05 + 680) if (ioctl(fileno(stdout), TIOCGWINSZ, &screen_size) != -1) 201

Re: [HACKERS] Autonomous Transaction (WIP)

2014-04-08 Thread Rajeev rastogi
On 09 April 2014 01:43, Tom Lane Wrote: > > I'm also pretty unconvinced that multiple PGPROCs is the right way to > > go. First, PGPROCs have a bunch of state in them that is assumed to > > exist once per backend. We might find pretty substantial code churn > > there if we try to go change that.

[HACKERS] Proposal for Merge Join for Non '=' Operators

2014-04-08 Thread Dilip kumar
I would like to propose a New merge join algorithm for optimizing non '=' operators. ('<', '<=', '>', '>=') - Currently Merge join is only supported for '=' operator. For '<' or '>' operator it chooses Nest Loop Join, or Nest loop with materialization. - I think when tuple f

Re: [HACKERS] Autonomous Transaction (WIP)

2014-04-08 Thread Rajeev rastogi
On 09 April 2014 01:09, Rover Haas Wrote: > I'm also pretty unconvinced that multiple PGPROCs is the right way to > go. First, PGPROCs have a bunch of state in them that is assumed to > exist once per backend. We might find pretty substantial code churn > there if we try to go change that. Y

Re: [HACKERS] psql \d+ and oid display

2014-04-08 Thread Tom Lane
Robert Haas writes: > Well, that's sorta my concern. I mean, right now we've got people > saying "what the heck is a replica identity?". But, if the logical > decoding stuff becomes popular, as I hope it will, that's going to be > an important thing for people to adjust, and the information need

Re: [HACKERS] [DOCS] Call for GIST/GIN/SP-GIST opclass documentation

2014-04-08 Thread Robert Haas
On Tue, Apr 8, 2014 at 4:41 PM, Tom Lane wrote: > I just created sections in the SGML manual chapters about GIST, GIN, and > SP-GIST to hold documentation about the standard opclasses provided for > them: > > http://www.postgresql.org/docs/devel/static/gist-builtin-opclasses.html > http://www.post

Re: [HACKERS] psql \d+ and oid display

2014-04-08 Thread Robert Haas
On Tue, Apr 8, 2014 at 5:37 PM, Bruce Momjian wrote: > On Tue, Apr 8, 2014 at 05:29:45PM -0400, Tom Lane wrote: >> Bruce Momjian writes: >> > If we ignore backward compatibility, then "Has OIDs" and "Identity >> > Replica" are similar. One thing that strongly (for me) supports not >> > always p

Re: [HACKERS] Minor improvements in alter_table.sgml

2014-04-08 Thread Etsuro Fujita
(2014/04/09 1:23), Robert Haas wrote: On Tue, Apr 8, 2014 at 5:05 AM, Etsuro Fujita wrote: Attached is a patch to improve the manual page for the ALTER TABLE command. Do we really need to add a section for "type_name" when we already have a section for "OF type_name"? I think that the secti

Re: [HACKERS] Buffer Allocation Concurrency Limits

2014-04-08 Thread Amit Kapila
On Tue, Apr 8, 2014 at 10:38 PM, Jason Petersen wrote: > In December, Metin (a coworker of mine) discussed an inability to scale a > simple task (parallel scans of many independent tables) to many cores (it's > here). As a ramp-up task at Citus I was tasked to figure out what the heck > was going

Re: default opclass for jsonb (was Re: [HACKERS] Call for GIST/GIN/SP-GIST opclass documentation)

2014-04-08 Thread Tom Lane
Andrew Dunstan writes: > On 04/08/2014 05:57 PM, Peter Geoghegan wrote: >> ... I didn't propose changing the default due to >> concerns about the POLA, but I'm happy to be told that those concerns >> were out of proportion to the practical benefits of a different >> default. > I tend to agree wit

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-04-08 Thread Peter Geoghegan
On Thu, Apr 3, 2014 at 12:19 PM, Thom Brown wrote: > Looking good: > > -T 100 -n -f sort.sql > > Master: 21.670467 / 21.718653 (avg: 21.69456) > Patch: 66.888756 / 66.888756 (avg: 66.888756) These were almost exactly the same figures as I saw on my machine. However, when compiling with certain ad

Re: [HACKERS] Pending 9.4 patches

2014-04-08 Thread Craig Ringer
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/09/2014 09:28 AM, Stephen Frost wrote: > Ok. I'm going to make a serious effort to find time to work on > this, at least. Right now I'm busy preparing to launch a new site > (you'll see the announce in a couple days...), etc, etc, but I > shou

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-04-08 Thread Florian Pflug
On Apr9, 2014, at 02:55 , David Rowley wrote: > On Wed, Apr 9, 2014 at 8:48 AM, Florian Pflug wrote: > > As explain above, invtrans_bool is a bit problematic, since it carries > a real risk of performance regressions. It's included for completeness' > sake, and should probably not be committed a

Re: [HACKERS] Pending 9.4 patches

2014-04-08 Thread Craig Ringer
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/09/2014 01:54 AM, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: >> On Tue, Apr 8, 2014 at 11:59 AM, Gregory Smith >> wrote: >>> I have no other projects ahead of this for the remainder of >>> this month. I just can't figur

Re: [HACKERS] Pending 9.4 patches

2014-04-08 Thread Stephen Frost
* Craig Ringer (cr...@2ndquadrant.com) wrote: > On 04/09/2014 02:00 AM, Stephen Frost wrote: > > I'm a bit confused on this point- is the only issue the > > *preexisting* bug with security barrier views? > > This thread discusses two patches. The above refers to row security > (per quoted text at

Re: [HACKERS] Pending 9.4 patches

2014-04-08 Thread Craig Ringer
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/09/2014 02:00 AM, Stephen Frost wrote: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >>> Craig Ringer writes: > On 04/05/2014 03:57 AM, Andres Freund wrote: >>> r04) Row-security based on Updatable security barrier >>> views This one's fa

Re: [HACKERS] [PATCH] Negative Transition Aggregate Functions (WIP)

2014-04-08 Thread David Rowley
On Wed, Apr 9, 2014 at 8:48 AM, Florian Pflug wrote: > > As explain above, invtrans_bool is a bit problematic, since it carries > a real risk of performance regressions. It's included for completeness' > sake, and should probably not be committed at this time. > > Did you mean to write invtrans_m

[HACKERS] New option in pg_basebackup to exclude pg_log files during base backup

2014-04-08 Thread Prabakaran, Vaishnavi
Hi all, Following the discussion in message id - cahgqgwffmor4ecugwhzpaapyqbsekdg66vmj1rvej6z-ega...@mail.gmail.com , I have developed the patch which gives option to user to exclude pg_log directory contents in pg_base

Re: [HACKERS] Default gin operator class of jsonb failing with index row size maximum reached

2014-04-08 Thread Peter Geoghegan
On Tue, Apr 8, 2014 at 4:07 PM, Michael Paquier wrote: > If this is a known limitation and no fix is planned for 9.4, could it > be possible to document it appropriately for this release? This would > surprise users. It looks like the default GIN opclass will be changed, so it becomes a matter of

Re: [HACKERS] Default gin operator class of jsonb failing with index row size maximum reached

2014-04-08 Thread Michael Paquier
On Wed, Apr 9, 2014 at 6:48 AM, Tom Lane wrote: > Oleg Bartunov writes: >> We are working to avoid this limitation. > > What do you mean by that ... do you see it as something that could be > fixed quickly, or is this a long-term improvement project? If this is a known limitation and no fix is pl

Re: [HACKERS] psql \d+ and oid display

2014-04-08 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Bruce Momjian writes: > > If we ignore backward compatibility, then "Has OIDs" and "Identity > > Replica" are similar. One thing that strongly (for me) supports not > > always printing them is that I expect more people will be confused by > > the mention o

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-04-08 Thread Peter Geoghegan
On Tue, Apr 8, 2014 at 2:48 PM, Alvaro Herrera wrote: > I think the point here is what matters is that that gain from the > strxfrm part of the patch is large, regardless of what the baseline is > (right?). If there's a small loss in an uncommon worst case, that's > probably acceptable, as long a

Re: default opclass for jsonb (was Re: [HACKERS] Call for GIST/GIN/SP-GIST opclass documentation)

2014-04-08 Thread Andrew Dunstan
On 04/08/2014 05:57 PM, Peter Geoghegan wrote: On Tue, Apr 8, 2014 at 2:46 PM, Tom Lane wrote: Well, let me see if I understand the situation correctly: * jsonb_ops supports more operators * jsonb_hash_ops produces smaller, better-performing indexes * jsonb_ops falls over on inputs with wid

Re: [HACKERS] Default gin operator class of jsonb failing with index row size maximum reached

2014-04-08 Thread Oleg Bartunov
On Wed, Apr 9, 2014 at 1:48 AM, Tom Lane wrote: > Oleg Bartunov writes: >> We are working to avoid this limitation. > > What do you mean by that ... do you see it as something that could be > fixed quickly, or is this a long-term improvement project? Unfortunately, It's long-term project VODKA,

Re: default opclass for jsonb (was Re: [HACKERS] Call for GIST/GIN/SP-GIST opclass documentation)

2014-04-08 Thread Peter Geoghegan
On Tue, Apr 8, 2014 at 2:46 PM, Tom Lane wrote: > Well, let me see if I understand the situation correctly: > > * jsonb_ops supports more operators > > * jsonb_hash_ops produces smaller, better-performing indexes > > * jsonb_ops falls over on inputs with wide field values, but > jsonb_hash_ops doe

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-04-08 Thread Alvaro Herrera
Peter Geoghegan wrote: > What I have here looks like it speeds things up a little over 200% (so > a little over 300% of the original throughput) with a single client > for many representative cases. That's a massive difference, to the > point that I don't see a lot of sense in considering fmgr-eli

Re: [HACKERS] Default gin operator class of jsonb failing with index row size maximum reached

2014-04-08 Thread Tom Lane
Oleg Bartunov writes: > We are working to avoid this limitation. What do you mean by that ... do you see it as something that could be fixed quickly, or is this a long-term improvement project? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@pos

default opclass for jsonb (was Re: [HACKERS] Call for GIST/GIN/SP-GIST opclass documentation)

2014-04-08 Thread Tom Lane
Peter Geoghegan writes: > On Tue, Apr 8, 2014 at 2:34 PM, Tom Lane wrote: >> (BTW, wasn't there some discussion of changing our minds about which >> one is the default? We already have one bug report complaining about >> jsonb_ops' size restriction, so that seems to be evidence in favor >> of ch

Re: [HACKERS] Call for GIST/GIN/SP-GIST opclass documentation

2014-04-08 Thread Peter Geoghegan
On Tue, Apr 8, 2014 at 2:34 PM, Tom Lane wrote: > (BTW, wasn't there some discussion of changing our minds about which > one is the default? We already have one bug report complaining about > jsonb_ops' size restriction, so that seems to be evidence in favor > of changing ...) Yes, there was. I

Re: [HACKERS] Call for GIST/GIN/SP-GIST opclass documentation

2014-04-08 Thread Tom Lane
Peter Geoghegan writes: > On Tue, Apr 8, 2014 at 1:41 PM, Tom Lane wrote: >> Of the two operator classes for type jsonb, jsonb_ops is the >> default. jsonb_hash_ops supports fewer operators but will work with >> larger indexed values than jsonb_ops can support. >> >> Is that accurate? Do we nee

Re: [HACKERS] Default gin operator class of jsonb failing with index row size maximum reached

2014-04-08 Thread Oleg Bartunov
We are working to avoid this limitation. On Tue, Apr 8, 2014 at 10:54 PM, Peter Geoghegan wrote: > On Mon, Apr 7, 2014 at 8:29 PM, Michael Paquier > wrote: >> Documentation of jsonb tells that jsonb documents should be kept at a >> reasonable size to reduce lock contention, but there is no menti

Re: [HACKERS] psql \d+ and oid display

2014-04-08 Thread Bruce Momjian
On Tue, Apr 8, 2014 at 05:29:45PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > If we ignore backward compatibility, then "Has OIDs" and "Identity > > Replica" are similar. One thing that strongly (for me) supports not > > always printing them is that I expect more people will be confused b

Re: [HACKERS] psql \d+ and oid display

2014-04-08 Thread Tom Lane
Bruce Momjian writes: > If we ignore backward compatibility, then "Has OIDs" and "Identity > Replica" are similar. One thing that strongly (for me) supports not > always printing them is that I expect more people will be confused by > the mention of OIDs or "Identity Replica" than will actually c

Re: [HACKERS] psql \d+ and oid display

2014-04-08 Thread Bruce Momjian
On Tue, Apr 1, 2014 at 01:36:02PM -0400, Robert Haas wrote: > Although I agree with the general principle, I'm skeptical in this > case. There are a bunch of table-level options, and I don't think > it's very reasonable to expect that users are going to remember which > ones are going to be displ

Re: [HACKERS] Call for GIST/GIN/SP-GIST opclass documentation

2014-04-08 Thread Peter Geoghegan
On Tue, Apr 8, 2014 at 1:41 PM, Tom Lane wrote: > I just created sections in the SGML manual chapters about GIST, GIN, and > SP-GIST to hold documentation about the standard opclasses provided for > them: I think that that's a good idea. I too was bothered by this omission. > Of the two oper

Re: [HACKERS] GiST support for inet datatypes

2014-04-08 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Tue, Apr 8, 2014 at 3:51 PM, Tom Lane wrote: > > I wrote: > >> [ inet-gist-v6.patch ] > > > > Committed with some additional documentation work. Thanks for > > submitting this! > > NICE. I'd like to tell you how excited I am about this part: >

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-04-08 Thread Peter Geoghegan
On Tue, Apr 8, 2014 at 12:31 PM, Robert Haas wrote: > No, we're concerned about ending up with the best possible > performance. That could mean applying the fmgr-elision but not the > other part. Whether the other part is beneficial is based on how it > compares to the performance post-fmgr-elis

[HACKERS] Call for GIST/GIN/SP-GIST opclass documentation

2014-04-08 Thread Tom Lane
I just created sections in the SGML manual chapters about GIST, GIN, and SP-GIST to hold documentation about the standard opclasses provided for them: http://www.postgresql.org/docs/devel/static/gist-builtin-opclasses.html http://www.postgresql.org/docs/devel/static/gin-builtin-opclasses.html http

Re: [HACKERS] Autonomous Transaction (WIP)

2014-04-08 Thread Andres Freund
On 2014-04-08 16:13:21 -0400, Tom Lane wrote: > Robert Haas writes: > > I'm also pretty unconvinced that multiple PGPROCs is the right way to > > go. First, PGPROCs have a bunch of state in them that is assumed to > > exist once per backend. We might find pretty substantial code churn > > there

Re: [HACKERS] Autonomous Transaction (WIP)

2014-04-08 Thread Tom Lane
Robert Haas writes: > I'm also pretty unconvinced that multiple PGPROCs is the right way to > go. First, PGPROCs have a bunch of state in them that is assumed to > exist once per backend. We might find pretty substantial code churn > there if we try to go change that. Second, why do other backe

Re: [HACKERS] GiST support for inet datatypes

2014-04-08 Thread Tom Lane
Robert Haas writes: > NICE. I'd like to tell you how excited I am about this part: > # It also handles a new network > # operator inet && inet (overlaps, a/k/a "is supernet or subnet of"), > # which is expected to be useful in exclusion constraints. > ...but I can't, because my mouth is too ful

Re: [HACKERS] GiST support for inet datatypes

2014-04-08 Thread Robert Haas
On Tue, Apr 8, 2014 at 3:51 PM, Tom Lane wrote: > I wrote: >> [ inet-gist-v6.patch ] > > Committed with some additional documentation work. Thanks for > submitting this! NICE. I'd like to tell you how excited I am about this part: # It also handles a new network # operator inet && inet (overla

Re: [HACKERS] GiST support for inet datatypes

2014-04-08 Thread Tom Lane
I wrote: > [ inet-gist-v6.patch ] Committed with some additional documentation work. Thanks for submitting this! regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mail

Re: [HACKERS] Autonomous Transaction (WIP)

2014-04-08 Thread Andres Freund
On 2014-04-08 15:39:18 -0400, Robert Haas wrote: > I'm also pretty unconvinced that multiple PGPROCs is the right way to > go. First, PGPROCs have a bunch of state in them that is assumed to > exist once per backend. We might find pretty substantial code churn > there if we try to go change that.

Re: [HACKERS] Autonomous Transaction (WIP)

2014-04-08 Thread Robert Haas
On Tue, Apr 8, 2014 at 2:43 PM, Alvaro Herrera wrote: > Greg Stark wrote: >> If the autonomous transaction is actually a separate procarray entry >> (which I suspect it would have to be, much like prepared transactions >> and the dblink connections which are commonly used to kludge >> autonomous t

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-04-08 Thread Robert Haas
On Tue, Apr 8, 2014 at 3:10 PM, Peter Geoghegan wrote: > On Tue, Apr 8, 2014 at 10:10 AM, Heikki Linnakangas > wrote: >> Right. But 1) is the baseline we need to evaluate 2) against. > > I don't agree with that. Surely we're concerned with not regressing > cases that people actually care about, w

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-04-08 Thread Peter Geoghegan
On Tue, Apr 8, 2014 at 10:10 AM, Heikki Linnakangas wrote: > Right. But 1) is the baseline we need to evaluate 2) against. I don't agree with that. Surely we're concerned with not regressing cases that people actually care about, which in practical terms means the changes of a single release. Whi

Re: [HACKERS] Default gin operator class of jsonb failing with index row size maximum reached

2014-04-08 Thread Peter Geoghegan
On Mon, Apr 7, 2014 at 8:29 PM, Michael Paquier wrote: > Documentation of jsonb tells that jsonb documents should be kept at a > reasonable size to reduce lock contention, but there is no mention of > size limitation for indexes: > http://www.postgresql.org/docs/devel/static/datatype-json.html It

Re: [HACKERS] ipc_test

2014-04-08 Thread Alvaro Herrera
Greg Stark wrote: > On Mon, Apr 7, 2014 at 10:43 AM, Robert Haas wrote: > > OK, done. One less thing to worry about when committing! > > Also one less thing to cause headaches with etags and similar tools. > It always drove me nuts that I was constantly being sent to ipc_test > files for various

Re: [HACKERS] Autonomous Transaction (WIP)

2014-04-08 Thread Alvaro Herrera
Greg Stark wrote: > If the autonomous transaction is actually a separate procarray entry > (which I suspect it would have to be, much like prepared transactions > and the dblink connections which are commonly used to kludge > autonomous transactions) then this should be fairly painless. If you > i

Re: [HACKERS] ipc_test

2014-04-08 Thread Greg Stark
On Mon, Apr 7, 2014 at 10:43 AM, Robert Haas wrote: > OK, done. One less thing to worry about when committing! Also one less thing to cause headaches with etags and similar tools. It always drove me nuts that I was constantly being sent to ipc_test files for various typedefs. Thanks! -- greg

Re: [HACKERS] [BUG FIX] Compare returned value by socket() against PGINVALID_SOCKET instead of < 0

2014-04-08 Thread Bruce Momjian
On Sun, Apr 6, 2014 at 11:45:59AM +0530, Amit Kapila wrote: > On Tue, Apr 1, 2014 at 6:31 AM, Bruce Momjian wrote: > > I reviewed this patch and you are correct that we are not handling > > socket() and accept() returns properly on Windows. We were doing it > > properly in most place in the back

Re: [HACKERS] Autonomous Transaction (WIP)

2014-04-08 Thread Greg Stark
On Mon, Apr 7, 2014 at 12:06 AM, Rajeev rastogi wrote: > > Deadlock Detection: > > It is possible that the main or upper autonomous transaction has taken a lock > on some resource, which might be required by lower autonomous transaction. If > it happens so then deadlock will occur. So in order t

Re: [HACKERS] Pending 9.4 patches

2014-04-08 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Craig Ringer writes: > > On 04/05/2014 03:57 AM, Andres Freund wrote: > >> r04) Row-security based on Updatable security barrier views > >> This one's fate seems to be hard to judge without c07. > > > Open issues remain with this patch, and resources for w

Re: [HACKERS] Pending 9.4 patches

2014-04-08 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Tue, Apr 8, 2014 at 11:59 AM, Gregory Smith > wrote: > > I have no other projects ahead of this for the remainder of this month. I > > just can't figure out what to do next until there's a committer (or > > committers, if someone else is going to

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-04-08 Thread Heikki Linnakangas
On 04/08/2014 08:02 PM, Peter Geoghegan wrote: On Tue, Apr 8, 2014 at 3:12 AM, Heikki Linnakangas wrote: 1. Avoid fmgr and shim overhead 2. Use strxfrm to produce a pseudo-leading key that's cheaper to compare. In that case, these changes need to be analyzed separately. You don't get to "make

[HACKERS] Buffer Allocation Concurrency Limits

2014-04-08 Thread Jason Petersen
In December, Metin (a coworker of mine) discussed an inability to scale a simple task (parallel scans of many independent tables) to many cores (it’s here). As a ramp-up task at Citus I was tasked to figure out what the heck was going on here. I have a pretty extensive writeup here (whose lengt

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-04-08 Thread Peter Geoghegan
t On Tue, Apr 8, 2014 at 3:12 AM, Heikki Linnakangas wrote: > 1. Avoid fmgr and shim overhead > 2. Use strxfrm to produce a pseudo-leading key that's cheaper to compare. > > In that case, these changes need to be analyzed separately. You don't get to > "make up" for the losses by the second part b

Re: [HACKERS] four minor proposals for 9.5

2014-04-08 Thread Pavel Stehule
2014-04-08 18:34 GMT+02:00 Gregory Smith : > On 4/6/14 2:46 PM, Pavel Stehule wrote: > >> >> Proposed options are interesting for "enterprise" using, when you have a >> some more smart tools for log entry processing, and when you need a complex >> view about performance of billions queries - when

Re: [HACKERS] four minor proposals for 9.5

2014-04-08 Thread Gregory Smith
On 4/6/14 2:46 PM, Pavel Stehule wrote: Proposed options are interesting for "enterprise" using, when you have a some more smart tools for log entry processing, and when you need a complex view about performance of billions queries - when cancel time and lock time is important piece in mosaic

Re: [HACKERS] Problem with displaying "wide" tables in psql

2014-04-08 Thread Greg Stark
On Tue, Apr 8, 2014 at 12:19 PM, Andres Freund wrote: > I don't think this is easily testable that way - doesn't it rely on > determining the width of the terminal? Which you won't have when started > from pg_regress? There's a pset variable to set the target width so at least the formatting code

Re: [HACKERS] Minor improvements in alter_table.sgml

2014-04-08 Thread Robert Haas
On Tue, Apr 8, 2014 at 5:05 AM, Etsuro Fujita wrote: > Attached is a patch to improve the manual page for the ALTER TABLE command. Do we really need to add a section for "type_name" when we already have a section for "OF type_name"? constraint_name is also used for adding a constraint using an i

Re: [HACKERS] Problem with displaying "wide" tables in psql

2014-04-08 Thread Andres Freund
On 2014-04-08 12:15:47 -0400, Greg Stark wrote: > With no doc changes and no regression tests I was halfway inclined to > just reject it out of hand. To be fair there were no regression tests > for wrapped output prior to the patch but still I would have wanted to > see them added. We often pare do

Re: [HACKERS] Problem with displaying "wide" tables in psql

2014-04-08 Thread Greg Stark
On Sat, Feb 15, 2014 at 11:08 AM, Emre Hasegeli wrote: > This is my review about 3th version of the patch. It is an useful > improvement in my opinion. It worked well on my environment. I'm reviewing this patch. One thing to comment: With no doc changes and no regression tests I was halfway in

Re: [HACKERS] Pending 9.4 patches

2014-04-08 Thread Robert Haas
On Tue, Apr 8, 2014 at 11:59 AM, Gregory Smith wrote: > I have no other projects ahead of this for the remainder of this month. I > just can't figure out what to do next until there's a committer (or > committers, if someone else is going to take on the locking bug) identified. > I looked at the

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-04-08 Thread Greg Stark
On Mon, Apr 7, 2014 at 7:32 PM, Peter Geoghegan wrote: > I think that Greg's choice of words was a little imprudent, but must > be viewed in the context of an offline discussion during the hall > track of pgConf NYC. Clearly Greg wasn't about to go off and > unilaterally commit this. FWIW, I think

Re: [HACKERS] Pending 9.4 patches

2014-04-08 Thread Gregory Smith
On 4/7/14 2:59 AM, Craig Ringer wrote: On 04/05/2014 03:57 AM, Andres Freund wrote: c07) Updatable security barrier views. This needs a serious look by a committer. I've been exercising it via row security and it's been looking pretty solid. It isn't a huge or intrusive patch, and it's s

Re: [HACKERS] CREATE FOREIGN TABLE ( ... LIKE ... )

2014-04-08 Thread Robert Haas
On Mon, Apr 7, 2014 at 4:24 PM, Andres Freund wrote: > On 2014-04-05 11:46:16 -0400, Tom Lane wrote: >> ISTM this is because the proposed feature is wrongheaded. The basic >> concept of CREATE TABLE LIKE is that you're copying properties from >> another object of the same type. You might or migh

Re: [HACKERS] Doc typo in "9.28. Event Trigger Functions"

2014-04-08 Thread Robert Haas
On Tue, Apr 8, 2014 at 2:39 AM, Ian Barwick wrote: > Just a single missing 's'. Thanks, committed. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscrip

Re: [HACKERS] Dynamic Shared Memory stuff

2014-04-08 Thread Robert Haas
On Fri, Apr 4, 2014 at 10:01 AM, Robert Haas wrote: > On Wed, Jan 22, 2014 at 10:17 AM, Noah Misch wrote: >> Yeah, abandoning the state file is looking attractive. > > Here's a draft patch getting rid of the state file. This should > address concerns raised by Heikki and Fujii Masao and echoed b

Re: [HACKERS] four minor proposals for 9.5

2014-04-08 Thread Pavel Stehule
2014-04-08 6:27 GMT+02:00 Amit Kapila : > On Mon, Apr 7, 2014 at 12:16 AM, Pavel Stehule > wrote: > > 2014-04-04 6:51 GMT+02:00 Amit Kapila : > >> On Tue, Apr 1, 2014 at 11:42 PM, Pavel Stehule > > >> wrote: > >> > 2014-03-27 17:56 GMT+01:00 Pavel Stehule : > >> >> So I'll prepare a some prototy

Re: Fwd: [HACKERS] Proposal: variant of regclass

2014-04-08 Thread Robert Haas
On Tue, Apr 8, 2014 at 11:01 AM, Robert Haas wrote: > On Tue, Apr 8, 2014 at 10:50 AM, Tom Lane wrote: >> Robert Haas writes: >>> Looks good, committed with a bit of further cleanup. >> >> I had not actually paid attention to the non-regclass parts of this, and >> now that I look, I've got to sa

Re: Fwd: [HACKERS] Proposal: variant of regclass

2014-04-08 Thread Robert Haas
On Tue, Apr 8, 2014 at 10:50 AM, Tom Lane wrote: > Robert Haas writes: >> Looks good, committed with a bit of further cleanup. > > I had not actually paid attention to the non-regclass parts of this, and > now that I look, I've got to say that it seems borderline insane to have > chosen to implem

Re: [HACKERS] Proposal: COUNT(*) (and related) speedup

2014-04-08 Thread Robert Haas
On Fri, Apr 4, 2014 at 1:19 PM, Tom Lane wrote: > Joshua Yanovski writes: >>> But worse, what happens if a count(*) >>> is in progress? It might or might not have scanned this page already, >>> and there's no way to get the right answer in both cases. Counter >>> updates done by VACUUM would ha

Re: Fwd: [HACKERS] Proposal: variant of regclass

2014-04-08 Thread Tom Lane
Robert Haas writes: > Looks good, committed with a bit of further cleanup. I had not actually paid attention to the non-regclass parts of this, and now that I look, I've got to say that it seems borderline insane to have chosen to implement regproc/regoper rather than regprocedure/regoperator. Th

Re: [HACKERS] WAL format and API changes (9.5)

2014-04-08 Thread Amit Kapila
On Thu, Apr 3, 2014 at 7:44 PM, Heikki Linnakangas wrote: > I'd like to do some changes to the WAL format in 9.5. I want to annotate > each WAL record with the blocks that they modify. Every WAL record already > includes that information, but it's done in an ad hoc way, differently in > every rmgr

Re: Fwd: [HACKERS] Proposal: variant of regclass

2014-04-08 Thread Robert Haas
On Tue, Apr 8, 2014 at 3:01 AM, Yugo Nagata wrote: > On Mon, 7 Apr 2014 12:00:49 -0400 > Robert Haas wrote: >> In other words, let's revert the whole refactoring of this file to >> create reg*_guts functions, and instead just copy the relevant logic >> for the name lookups into the new functions.

Re: [HACKERS] PostgreSQL Columnar Store for Analytic Workloads

2014-04-08 Thread Hadi Moshayedi
Hi Stefan, On Tue, Apr 8, 2014 at 9:28 AM, Stefan Keller wrote: > Hi Hadi > > Do you think that cstore_fd*w* is also welll suited for storing and > retrieving linked data (RDF)? > I am not very familiar with RDF. Note that cstore_fdw doesn't change the query language of PostgreSQL, so if your

Re: [HACKERS] Why is it not sane to pass ExecStoreTuple(shouldFree=true) for tuples point into buffers

2014-04-08 Thread Andres Freund
On 2014-04-08 09:37:33 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2014-04-07 21:47:38 -0400, Tom Lane wrote: > >> Well, that is certainly messy. I think you could just use a local > >> HeapTupleData variable instead of palloc'ing every time, where "local" > >> means "has lifespan simil

Re: [HACKERS] Why is it not sane to pass ExecStoreTuple(shouldFree=true) for tuples point into buffers

2014-04-08 Thread Tom Lane
Andres Freund writes: > On 2014-04-07 21:47:38 -0400, Tom Lane wrote: >> Well, that is certainly messy. I think you could just use a local >> HeapTupleData variable instead of palloc'ing every time, where "local" >> means "has lifespan similar to the slot pointer". >> There's some vaguely simila

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-04-08 Thread Heikki Linnakangas
On 04/07/2014 09:19 PM, Peter Geoghegan wrote: The only case that this patch could possibly regress is where there are strings that differ beyond about the first 8 bytes, but are not identical (we chance a memcmp() == 0 before doing a full strcoll() when tie-breaking on the semi-reliable initial

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-04-08 Thread Heikki Linnakangas
On 04/07/2014 11:35 PM, Peter Geoghegan wrote: Okay. Here is a worst-case, with the pgbench script the same as my original test-case, but with much almost maximally unsympathetic data to sort: [local]/postgres=# update customers set firstname = 'padding-padding-padding-padding' || firstname; H

Re: [HACKERS] Patch: add psql tab completion for event triggers

2014-04-08 Thread Ian Barwick
On 08/04/14 18:22, Ian Barwick wrote: As it was kind of annoying not to have this when playing around with event triggers. This also tightens up the existing tab completion for ALTER TRIGGER, which contained redundant code for table name completion, and which was also causing a spurious "RENAME

[HACKERS] Patch: add psql tab completion for event triggers

2014-04-08 Thread Ian Barwick
As it was kind of annoying not to have this when playing around with event triggers. This also tightens up the existing tab completion for ALTER TRIGGER, which contained redundant code for table name completion, and which was also causing a spurious "RENAME TO" to be inserted in this context:

[HACKERS] Minor improvements in alter_table.sgml

2014-04-08 Thread Etsuro Fujita
Attached is a patch to improve the manual page for the ALTER TABLE command. Thanks, Best regards, Etsuro Fujita diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 0b08f83..ce67c71 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_ta

Re: [HACKERS] Why is it not sane to pass ExecStoreTuple(shouldFree=true) for tuples point into buffers

2014-04-08 Thread Andres Freund
On 2014-04-07 21:47:38 -0400, Tom Lane wrote: > Andres Freund writes: > > So what I now do is essentially: > > while ((scantuple = index_getnext(scan, ForwardScanDirection)) != NULL) > > { > > ... > > ht = palloc(sizeof(HeapTupleData)); /* in the right context */ > > memcpy(ht, sca

Re: [HACKERS] Including replication slot data in base backups

2014-04-08 Thread Michael Paquier
On Tue, Apr 8, 2014 at 1:18 AM, Robert Haas wrote: > Not sure if this is exactly the right way to do it, but I agree that > something along those lines is a good idea. I also think, maybe even > importantly, that we should probably document that people using > file-copy based hot backups should s

Re: Fwd: [HACKERS] Proposal: variant of regclass

2014-04-08 Thread Yugo Nagata
On Mon, 7 Apr 2014 12:00:49 -0400 Robert Haas wrote: > In other words, let's revert the whole refactoring of this file to > create reg*_guts functions, and instead just copy the relevant logic > for the name lookups into the new functions. For to_regproc(), for > example, it would look like this