Re: [HACKERS] Declarative partitioning

2015-08-21 Thread Jim Nasby
On 8/18/15 12:31 PM, Josh Berkus wrote: Also this would be useful for range partitions: CREATE PARTITION ON parent_table USING ( start_value ); ... where start_value is the start range of the new partition. Again, easier for users to get correct. Instead of that, I think it would be more

Re: [HACKERS] (full) Memory context dump considered harmful

2015-08-21 Thread Tomas Vondra
On 08/21/2015 08:37 PM, Tom Lane wrote: Tomas Vondra tomas.von...@2ndquadrant.com writes: I also don't think logging just subset of the stats is a lost case. Sure, we can't know which of the lines are important, but for example logging just the top-level contexts with a summary of the child

Re: [HACKERS] Make HeapTupleSatisfiesMVCC more concurrent

2015-08-21 Thread Amit Kapila
On Fri, Aug 21, 2015 at 8:22 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, Aug 19, 2015 at 2:53 AM, Amit Kapila amit.kapil...@gmail.com wrote: I think one case where the patch can impact is for aborted transactions. In TransactionIdIsInProgress(), we check for aborted transactions

Re: [HACKERS] [PATCH v1] GSSAPI encryption support

2015-08-21 Thread Michael Paquier
On Sat, Aug 22, 2015 at 4:06 AM, Robbie Harwood wrote: Michael Paquier michael.paqu...@gmail.com writes: Going through the docs, the overall approach taken by the patch looks neat, and the default values as designed for both the client and the server are good things to do. Now actually

[HACKERS] Archiving done right

2015-08-21 Thread David Steele
I split this into a new thread to talk about archiving best practices. On 8/18/15 2:25 PM, Stephen Frost wrote: * Stephen Frost (sfr...@snowman.net) wrote: * Robert Haas (robertmh...@gmail.com) wrote: On Mon, Aug 17, 2015 at 2:50 PM, Peter Eisentraut pete...@gmx.net wrote: The commit message

Re: [HACKERS] Make HeapTupleSatisfiesMVCC more concurrent

2015-08-21 Thread Robert Haas
On Wed, Aug 19, 2015 at 2:53 AM, Amit Kapila amit.kapil...@gmail.com wrote: I think one case where the patch can impact is for aborted transactions. In TransactionIdIsInProgress(), we check for aborted transactions before consulting pg_subtrans whereas with patch it will consult pg_subtrans

Re: [HACKERS] Autonomous Transaction is back

2015-08-21 Thread Robert Haas
On Tue, Aug 18, 2015 at 3:06 PM, Merlin Moncure mmonc...@gmail.com wrote: On Tue, Aug 18, 2015 at 8:17 AM, Robert Haas robertmh...@gmail.com wrote: On Sat, Aug 15, 2015 at 6:47 PM, Noah Misch n...@leadboat.com wrote: CREATE TABLE t (c) AS SELECT 1; BEGIN; UPDATE t SET c = 2 WHERE c = 1;

Re: [HACKERS] [DESIGN] ParallelAppend

2015-08-21 Thread Robert Haas
On Tue, Aug 18, 2015 at 11:27 PM, Amit Kapila amit.kapil...@gmail.com wrote: Here is one other issue I found. Existing code assumes a TOC segment has only one contents per node type, so it uses pre-defined key (like PARALLEL_KEY_SCAN) per node type, however, it is problematic if we put

Re: [HACKERS] [PATCH v1] GSSAPI encryption support

2015-08-21 Thread Michael Paquier
On Fri, Jul 3, 2015 at 3:22 AM, Robbie Harwood rharw...@redhat.com wrote: Hello -hackers, As previously discussed on this list, I have coded up GSSAPI encryption support. If it is easier for anyone, this code is also available for viewing on my github:

Re: [HACKERS] Warnings around booleans

2015-08-21 Thread Stephen Frost
* Andres Freund (and...@anarazel.de) wrote: I'd rather see those split into separate commits. Doing polishing and active bugs in one commit imo isn't a good idea if the polishing goes beyond a line or two. Otherwise this looks ok to me. Done that way. Thanks!

Re: [HACKERS] Make HeapTupleSatisfiesMVCC more concurrent

2015-08-21 Thread Robert Haas
On Tue, Aug 18, 2015 at 8:36 PM, Tom Lane t...@sss.pgh.pa.us wrote: I wrote: Just thinking about this ... I wonder why we need to call TransactionIdIsInProgress() at all rather than believing the answer from the snapshot? Under what circumstances could TransactionIdIsInProgress() return true

Re: [HACKERS] [PATCH] postgres_fdw extension support

2015-08-21 Thread Alvaro Herrera
Michael Paquier wrote: if (needlabel) appendStringInfo(buf, ::%s, - format_type_with_typemod(node-consttype, - node-consttypmod)); + format_type_be_qualified(node-consttype)); Pondering more about this one, I think that we are going to need a new routine in

Re: [HACKERS] [PATCH] postgres_fdw extension support

2015-08-21 Thread David Fetter
On Fri, Aug 21, 2015 at 12:55:39PM -0300, Alvaro Herrera wrote: Michael Paquier wrote: if (needlabel) appendStringInfo(buf, ::%s, - format_type_with_typemod(node-consttype, - node-consttypmod)); + format_type_be_qualified(node-consttype)); Pondering

Re: [HACKERS] Make HeapTupleSatisfiesMVCC more concurrent

2015-08-21 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Aug 18, 2015 at 8:36 PM, Tom Lane t...@sss.pgh.pa.us wrote: I experimented with the attached patch, which replaces HeapTupleSatisfiesMVCC's calls of TransactionIdIsInProgress with XidInMVCCSnapshot, and then as a cross-check has all the return

Re: [HACKERS] Error message with plpgsql CONTINUE

2015-08-21 Thread Jim Nasby
On 8/17/15 4:46 PM, Tom Lane wrote: Jim Nasby jim.na...@bluetreble.com writes: On 8/17/15 9:48 AM, Tom Lane wrote: I'm inclined to think that if we wanted to make this better, the way to improve it would be to detect the error*at compile time*, and get rid of this hack in plpgsql_exec_function

Re: [HACKERS] DBT-3 with SF=20 got failed

2015-08-21 Thread Tomas Vondra
Hello KaiGai-san, On 08/21/2015 02:28 AM, Kouhei Kaigai wrote: ... But what is the impact on queries that actually need more than 1GB of buckets? I assume we'd only limit the initial allocation and still allow the resize based on the actual data (i.e. the 9.5 improvement), so the queries would

Re: [HACKERS] Warnings around booleans

2015-08-21 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Stephen Frost sfr...@snowman.net writes: On Friday, August 21, 2015, Tom Lane t...@sss.pgh.pa.us wrote: It is not really acceptable to leave roles hanging around after make installcheck; that would be a security hazard for the installation. Please

[HACKERS] minor typo in trigger.c

2015-08-21 Thread Merlin Moncure
diff --git a/src/backend/commands/trigger.c b/src/backend/commands/trigger.c index d169027..43421d6 100644 --- a/src/backend/commands/trigger.c +++ b/src/backend/commands/trigger.c @@ -4126,7 +4126,7 @@ AfterTriggerEndXact(bool isCommit) /* -* Forget the query stack and constrant-related

Re: [HACKERS] Using quicksort for every external sort run

2015-08-21 Thread Peter Geoghegan
On Thu, Aug 20, 2015 at 11:56 PM, Simon Riggs si...@2ndquadrant.com wrote: This will give more runs, so merging those needs some thought. It will also give a more predictable number of runs, so we'll be able to predict any merging issues ahead of time. We can more easily find out the min/max

Re: [HACKERS] Reduce ProcArrayLock contention

2015-08-21 Thread Robert Haas
On Wed, Aug 19, 2015 at 11:39 AM, Andres Freund and...@anarazel.de wrote: There's a bunch of issues with those two blocks afaics: 1) The first block (in one backend) might read INVALID_PGPROCNO before ever locking the semaphore if a second backend quickly enough writes INVALID_PGPROCNO.

Re: [HACKERS] Warnings around booleans

2015-08-21 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: On Friday, August 21, 2015, Tom Lane t...@sss.pgh.pa.us wrote: It is not really acceptable to leave roles hanging around after make installcheck; that would be a security hazard for the installation. Please drop them. The only ones which were left

Re: [HACKERS] [PATCH v1] GSSAPI encryption support

2015-08-21 Thread Robbie Harwood
Michael Paquier michael.paqu...@gmail.com writes: On Fri, Jul 3, 2015 at 3:22 AM, Robbie Harwood rharw...@redhat.com wrote: There are 8 commits in this series; I have tried to err on the side of creating too much separation rather than too little. A patch for each is attached. This is v1

Re: [HACKERS] remove unused ExecGetScanType

2015-08-21 Thread Alvaro Herrera
Alvaro Herrera wrote: I don't see any reason not to remove this. It's been unused since a191a169d6d0b9558da4519e66510c4540204a51, dated Jan 10 2007. Pushed. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services

Re: [HACKERS] Warnings around booleans

2015-08-21 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: On Friday, August 21, 2015, Piotr Stefaniak postg...@piotr-stefaniak.me wrote: If I'm not mistaken, the roles introduced in this test are never dropped, which will cause the test to fail on consequent runs. Ah, I was thinking there was a reason to not

Re: [HACKERS] Warnings around booleans

2015-08-21 Thread Stephen Frost
On Friday, August 21, 2015, Tom Lane t...@sss.pgh.pa.us wrote: Stephen Frost sfr...@snowman.net javascript:; writes: On Friday, August 21, 2015, Piotr Stefaniak postg...@piotr-stefaniak.me javascript:; wrote: If I'm not mistaken, the roles introduced in this test are never dropped,

Re: [HACKERS] A few cases of left shifting negative integers

2015-08-21 Thread Andres Freund
On 2015-08-21 13:27:22 -0400, Tom Lane wrote: Andres Freund and...@anarazel.de writes: On 2015-08-21 13:03:42 -0400, Tom Lane wrote: The behavior is well-defined, at least as long as we don't shift far enough to have integer overflow Unfortunately not: 5.8.2: The value of E1 E2 is E1

Re: [HACKERS] A few cases of left shifting negative integers

2015-08-21 Thread Tom Lane
Andres Freund and...@anarazel.de writes: On 2015-08-21 13:03:42 -0400, Tom Lane wrote: The behavior is well-defined, at least as long as we don't shift far enough to have integer overflow Unfortunately not: 5.8.2: The value of E1 E2 is E1 left-shifted E2 bit positions; vacated bits are

Re: [HACKERS] Warnings around booleans

2015-08-21 Thread Stephen Frost
On Friday, August 21, 2015, Piotr Stefaniak postg...@piotr-stefaniak.me wrote: If I'm not mistaken, the roles introduced in this test are never dropped, which will cause the test to fail on consequent runs. Ah, I was thinking there was a reason to not leave them around but couldn't think of

Re: [HACKERS] TransactionIdGetCommitTsData and its dereferenced pointers

2015-08-21 Thread Alvaro Herrera
Michael Paquier wrote: TransactionIdGetCommitTsData@commit_ts.c does the following: if (ts) *ts = entry.time; [...] return *ts != 0; This is a bad idea, because if TransactionIdGetCommitTsData is called with ts == NULL this would simply crash. It seems to me that it

Re: [HACKERS] More WITH

2015-08-21 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Aug 18, 2015 at 11:23 PM, Tom Lane t...@sss.pgh.pa.us wrote: More generally, what would you hope to accomplish with such a construct that wouldn't be better done by writing the cursor's underlying query directly in the WITH clause? Maybe I'm

Re: [HACKERS] More WITH

2015-08-21 Thread Robert Haas
On Fri, Aug 21, 2015 at 2:39 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Aug 18, 2015 at 11:23 PM, Tom Lane t...@sss.pgh.pa.us wrote: More generally, what would you hope to accomplish with such a construct that wouldn't be better done by writing the

Re: [HACKERS] More WITH

2015-08-21 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: The existing syntax for FETCH already includes a way to specify the number of rows you want to fetch, as in this example from the documentation: FETCH FORWARD 5 FROM liahona; Why wouldn't that work here too? Mm, okay, but you still have the other

Re: [HACKERS] A few cases of left shifting negative integers

2015-08-21 Thread Tom Lane
Piotr Stefaniak postg...@piotr-stefaniak.me writes: during my testing I've found cases of left-shifting negative integers during run-time and I was recently encouraged to post a report of them, so here it is (done against 960ea971e66bcd621ba88841b4cb85c7f0e7c383). What's your concern

Re: [HACKERS] A few cases of left shifting negative integers

2015-08-21 Thread Andres Freund
On 2015-08-21 13:03:42 -0400, Tom Lane wrote: The behavior is well-defined, at least as long as we don't shift far enough to have integer overflow Unfortunately not: 5.8.2: The value of E1 E2 is E1 left-shifted E2 bit positions; vacated bits are zero-filled. If E1 has an unsigned type, the

Re: [HACKERS] Reduce ProcArrayLock contention

2015-08-21 Thread Andres Freund
On 2015-08-21 14:08:36 -0400, Robert Haas wrote: On Wed, Aug 19, 2015 at 11:39 AM, Andres Freund and...@anarazel.de wrote: There's a bunch of issues with those two blocks afaics: 1) The first block (in one backend) might read INVALID_PGPROCNO before ever locking the semaphore if a

Re: [HACKERS] (full) Memory context dump considered harmful

2015-08-21 Thread Tom Lane
Tomas Vondra tomas.von...@2ndquadrant.com writes: On 08/20/2015 11:04 PM, Stefan Kaltenbrunner wrote: On 08/20/2015 06:09 PM, Tom Lane wrote: (The reason I say lobotomize is that there's no particularly good reason to assume that the first N lines will tell you what you need to know. And the

Re: [HACKERS] Reduce ProcArrayLock contention

2015-08-21 Thread Robert Haas
On Fri, Aug 21, 2015 at 2:31 PM, Andres Freund and...@anarazel.de wrote: No, if it's paired like that, I don't think it's allowed to fail. But, as the code stands, there's absolutely no guarantee you're not seeing something like: P1: a = 0; P1: b = 0; P1: PGSemaphoreLock(P1); P2: a = 1;

Re: [HACKERS] More WITH

2015-08-21 Thread Robert Haas
On Tue, Aug 18, 2015 at 11:23 PM, Tom Lane t...@sss.pgh.pa.us wrote: More generally, what would you hope to accomplish with such a construct that wouldn't be better done by writing the cursor's underlying query directly in the WITH clause? Maybe I'm stupid today, but it seems like the obvious

Re: [HACKERS] More WITH

2015-08-21 Thread Robert Haas
On Fri, Aug 21, 2015 at 2:53 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: The existing syntax for FETCH already includes a way to specify the number of rows you want to fetch, as in this example from the documentation: FETCH FORWARD 5 FROM liahona; Why

Re: [HACKERS] Patch for ginCombineData

2015-08-21 Thread Jeff Janes
On Wed, Aug 5, 2015 at 3:17 AM, Robert Abraham robert.abraha...@googlemail.com wrote: Hello, we are using gin indexes on big tables. these tables happen to have several billion rows. the index creation fails in ginCombineData in src/backend/access/ginbulk.c because repalloc is limited to 1

Re: [HACKERS] (full) Memory context dump considered harmful

2015-08-21 Thread Tom Lane
I wrote: I thought a bit more about this. Generally, what you want to know about a given situation is which contexts have a whole lot of allocations and/or a whole lot of child contexts. What you suggest above won't work very well if the problem is buried more than about two levels down in

Re: [HACKERS] PostgreSQL for VAX on NetBSD/OpenBSD

2015-08-21 Thread Greg Stark
[- the vax lists since they cause majordomo confirmation emails for anyone responding] On Thu, Aug 20, 2015 at 3:29 PM, Tom Lane t...@sss.pgh.pa.us wrote: There are some planner tests that fail with floating point exceptions -- that's probably a bug on our part. And I've seen at least one

Re: [HACKERS] Error message with plpgsql CONTINUE

2015-08-21 Thread Tom Lane
Jim Nasby jim.na...@bluetreble.com writes: On 8/17/15 9:48 AM, Tom Lane wrote: I'm inclined to think that if we wanted to make this better, the way to improve it would be to detect the error*at compile time*, and get rid of this hack in plpgsql_exec_function altogether. Here's a patch that

Re: [HACKERS] Memory allocation in spi_printtup()

2015-08-21 Thread Tom Lane
Neil Conway neil.con...@gmail.com writes: On Mon, Aug 17, 2015 at 7:56 AM, Tom Lane t...@sss.pgh.pa.us wrote: Should we back-patch this change? Seems like it's arguably a performance bug. Sounds good to me. Committed and back-patched. regards, tom lane -- Sent

Re: [HACKERS] [PATCH] postgres_fdw extension support

2015-08-21 Thread Michael Paquier
On Sat, Aug 22, 2015 at 12:55 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Michael Paquier wrote: if (needlabel) appendStringInfo(buf, ::%s, - format_type_with_typemod(node-consttype, - node-consttypmod)); +

Re: [HACKERS] DBT-3 with SF=20 got failed

2015-08-21 Thread Kouhei Kaigai
Hello KaiGai-san, On 08/21/2015 02:28 AM, Kouhei Kaigai wrote: ... But what is the impact on queries that actually need more than 1GB of buckets? I assume we'd only limit the initial allocation and still allow the resize based on the actual data (i.e. the 9.5 improvement), so the

Re: [HACKERS] CTE optimization fence on the todo list?

2015-08-21 Thread Qingqing Zhou
On Fri, May 1, 2015 at 2:39 PM, Tom Lane t...@sss.pgh.pa.us wrote: * Multiply-referenced WITH item (unless the outer query applies identical constraints to each reference, which seems silly and not worth the cycles to check for). Not sure if I understand this correctly. Look at this query, CTE

Re: [HACKERS] Error message with plpgsql CONTINUE

2015-08-21 Thread Jim Nasby
On 8/21/15 7:21 PM, Tom Lane wrote: Applied with some fixes. The major oversight was that EXIT does*not* have the same rules as CONTINUE, as is clearly documented (though in your defense, there was no regression test verifying the behavior ... there is now). Yay more tests. I refactored the

Re: [HACKERS] Declarative partitioning

2015-08-21 Thread Pavan Deolasee
On Fri, Aug 21, 2015 at 11:22 AM, Amit Langote langote_amit...@lab.ntt.co.jp wrote: On 2015-08-21 AM 06:27, David Fetter wrote: By the last sentence, do you mean only UPDATEs to the partition key that cause rows to jump partitions or simply any UPDATEs to the partition key? I don't

Re: [HACKERS] Using quicksort for every external sort run

2015-08-21 Thread Simon Riggs
On 20 August 2015 at 18:41, Peter Geoghegan p...@heroku.com wrote: On Thu, Aug 20, 2015 at 8:15 AM, Simon Riggs si...@2ndquadrant.com wrote: On 20 August 2015 at 03:24, Peter Geoghegan p...@heroku.com wrote: The patch is ~3.25x faster than master I've tried to read this post twice