Re: [HACKERS] Warnings around booleans

2015-08-21 Thread Stephen Frost
* Andres Freund (and...@anarazel.de) wrote: > I'd rather see those split into separate commits. Doing polishing and > active bugs in one commit imo isn't a good idea if the polishing goes > beyond a line or two. > > Otherwise this looks ok to me. Done that way. Thanks! S

Re: [HACKERS] [PATCH v1] GSSAPI encryption support

2015-08-21 Thread Michael Paquier
On Fri, Jul 3, 2015 at 3:22 AM, Robbie Harwood wrote: > Hello -hackers, > > As previously discussed on this list, I have coded up GSSAPI encryption > support. If it is easier for anyone, this code is also available for > viewing on my github: > > https://github.com/postgres/postgres/compare/mast

Re: [HACKERS] Autonomous Transaction is back

2015-08-21 Thread Robert Haas
On Tue, Aug 18, 2015 at 3:06 PM, Merlin Moncure wrote: > On Tue, Aug 18, 2015 at 8:17 AM, Robert Haas wrote: >> On Sat, Aug 15, 2015 at 6:47 PM, Noah Misch wrote: >>> CREATE TABLE t (c) AS SELECT 1; >>> BEGIN; >>> UPDATE t SET c = 2 WHERE c = 1; >>> BEGIN_AUTONOMOUS; >>> UPDATE t SET c = 3 WHERE

Re: [HACKERS] [DESIGN] ParallelAppend

2015-08-21 Thread Robert Haas
On Tue, Aug 18, 2015 at 11:27 PM, Amit Kapila wrote: >> Here is one other issue I found. Existing code assumes a TOC segment has >> only one contents per node type, so it uses pre-defined key (like >> PARALLEL_KEY_SCAN) per node type, however, it is problematic if we put >> multiple PlannedStmt or

[HACKERS] Archiving done right

2015-08-21 Thread David Steele
I split this into a new thread to talk about archiving best practices. On 8/18/15 2:25 PM, Stephen Frost wrote: > * Stephen Frost (sfr...@snowman.net) wrote: >> * Robert Haas (robertmh...@gmail.com) wrote: >>> On Mon, Aug 17, 2015 at 2:50 PM, Peter Eisentraut wrote: The commit message for de

Re: [HACKERS] Make HeapTupleSatisfiesMVCC more concurrent

2015-08-21 Thread Robert Haas
On Tue, Aug 18, 2015 at 8:36 PM, Tom Lane wrote: > I wrote: >> Just thinking about this ... I wonder why we need to call >> TransactionIdIsInProgress() at all rather than believing the answer from >> the snapshot? Under what circumstances could TransactionIdIsInProgress() >> return true where Xid

Re: [HACKERS] Make HeapTupleSatisfiesMVCC more concurrent

2015-08-21 Thread Robert Haas
On Wed, Aug 19, 2015 at 2:53 AM, Amit Kapila wrote: > I think one case where the patch can impact is for aborted transactions. > In TransactionIdIsInProgress(), we check for aborted transactions before > consulting pg_subtrans whereas with patch it will consult pg_subtrans > without aborted transa

Re: [HACKERS] [PATCH] postgres_fdw extension support

2015-08-21 Thread Alvaro Herrera
Michael Paquier wrote: > if (needlabel) > appendStringInfo(buf, "::%s", > - > format_type_with_typemod(node->consttype, > - > node->consttypmod)); > + > format_type_be_qualified(node->consttype)); > Pondering more about this one, I think that we are going to need a new > ro

Re: [HACKERS] [PATCH] postgres_fdw extension support

2015-08-21 Thread David Fetter
On Fri, Aug 21, 2015 at 12:55:39PM -0300, Alvaro Herrera wrote: > Michael Paquier wrote: > > > if (needlabel) > > appendStringInfo(buf, "::%s", > > - > > format_type_with_typemod(node->consttype, > > - > > node->consttypmod)); > > + > > format_type_be_qualified(node->constt

Re: [HACKERS] Make HeapTupleSatisfiesMVCC more concurrent

2015-08-21 Thread Tom Lane
Robert Haas writes: > On Tue, Aug 18, 2015 at 8:36 PM, Tom Lane wrote: >> I experimented with the attached patch, which replaces >> HeapTupleSatisfiesMVCC's calls of TransactionIdIsInProgress with >> XidInMVCCSnapshot, and then as a cross-check has all the "return false" >> exits from XidInMVCCSn

Re: [HACKERS] A few cases of left shifting negative integers

2015-08-21 Thread Tom Lane
Piotr Stefaniak writes: > during my testing I've found cases of left-shifting negative integers > during run-time and I was recently encouraged to post a report of them, > so here it is (done against 960ea971e66bcd621ba88841b4cb85c7f0e7c383). What's your concern exactly? The behavior is well-d

Re: [HACKERS] remove unused ExecGetScanType

2015-08-21 Thread Alvaro Herrera
Alvaro Herrera wrote: > I don't see any reason not to remove this. It's been unused since > a191a169d6d0b9558da4519e66510c4540204a51, dated Jan 10 2007. Pushed. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] A few cases of left shifting negative integers

2015-08-21 Thread Andres Freund
On 2015-08-21 13:03:42 -0400, Tom Lane wrote: > The behavior is well-defined, at least as long as we don't shift far > enough to have integer overflow Unfortunately not: 5.8.2: The value of E1 << E2 is E1 left-shifted E2 bit positions; vacated bits are zero-filled. If E1 has an unsigned type, the

Re: [HACKERS] A few cases of left shifting negative integers

2015-08-21 Thread Tom Lane
Andres Freund writes: > On 2015-08-21 13:03:42 -0400, Tom Lane wrote: >> The behavior is well-defined, at least as long as we don't shift far >> enough to have integer overflow > Unfortunately not: > 5.8.2: The value of E1 << E2 is E1 left-shifted E2 bit positions; > vacated bits are zero-filled.

Re: [HACKERS] Warnings around booleans

2015-08-21 Thread Stephen Frost
On Friday, August 21, 2015, Piotr Stefaniak wrote: > If I'm not mistaken, the roles introduced in this test are never dropped, > which will cause the test to fail on consequent runs. > Ah, I was thinking there was a reason to not leave them around but couldn't think of it and liked the idea of t

Re: [HACKERS] TransactionIdGetCommitTsData and its dereferenced pointers

2015-08-21 Thread Alvaro Herrera
Michael Paquier wrote: > TransactionIdGetCommitTsData@commit_ts.c does the following: > if (ts) > *ts = entry.time; > [...] > return *ts != 0; > This is a bad idea, because if TransactionIdGetCommitTsData is called > with ts == NULL this would simply crash. It seems to me that

Re: [HACKERS] Warnings around booleans

2015-08-21 Thread Tom Lane
Stephen Frost writes: > On Friday, August 21, 2015, Piotr Stefaniak > wrote: >> If I'm not mistaken, the roles introduced in this test are never dropped, >> which will cause the test to fail on consequent runs. > Ah, I was thinking there was a reason to not leave them around but couldn't > think

Re: [HACKERS] Warnings around booleans

2015-08-21 Thread Stephen Frost
On Friday, August 21, 2015, Tom Lane wrote: > Stephen Frost > writes: > > On Friday, August 21, 2015, Piotr Stefaniak > > > wrote: > >> If I'm not mistaken, the roles introduced in this test are never > dropped, > >> which will cause the test to fail on consequent runs. > > > Ah, I was thinking

Re: [HACKERS] Using quicksort for every external sort run

2015-08-21 Thread Peter Geoghegan
On Thu, Aug 20, 2015 at 11:56 PM, Simon Riggs wrote: > This will give more runs, so merging those needs some thought. It will also > give a more predictable number of runs, so we'll be able to predict any > merging issues ahead of time. We can more easily find out the min/max tuple > in each run,

Re: [HACKERS] A few cases of left shifting negative integers

2015-08-21 Thread Andres Freund
On 2015-08-21 13:27:22 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2015-08-21 13:03:42 -0400, Tom Lane wrote: > >> The behavior is well-defined, at least as long as we don't shift far > >> enough to have integer overflow > > > Unfortunately not: > > 5.8.2: The value of E1 << E2 is E1 le

Re: [HACKERS] Reduce ProcArrayLock contention

2015-08-21 Thread Robert Haas
On Wed, Aug 19, 2015 at 11:39 AM, Andres Freund wrote: > There's a bunch of issues with those two blocks afaics: > > 1) The first block (in one backend) might read INVALID_PGPROCNO before >ever locking the semaphore if a second backend quickly enough writes >INVALID_PGPROCNO. That way the

Re: [HACKERS] Warnings around booleans

2015-08-21 Thread Tom Lane
Stephen Frost writes: > On Friday, August 21, 2015, Tom Lane wrote: >> It is not really acceptable to leave roles hanging around after "make >> installcheck"; that would be a security hazard for the installation. >> Please drop them. > The only ones which were left were intentionally all NOLOGIN

Re: [HACKERS] More WITH

2015-08-21 Thread Robert Haas
On Tue, Aug 18, 2015 at 11:23 PM, Tom Lane wrote: > More generally, what would you hope to accomplish with such a construct > that wouldn't be better done by writing the cursor's underlying query > directly in the WITH clause? Maybe I'm stupid today, but it seems like the obvious use case would b

Re: [HACKERS] Reduce ProcArrayLock contention

2015-08-21 Thread Andres Freund
On 2015-08-21 14:08:36 -0400, Robert Haas wrote: > On Wed, Aug 19, 2015 at 11:39 AM, Andres Freund wrote: > > There's a bunch of issues with those two blocks afaics: > > > > 1) The first block (in one backend) might read INVALID_PGPROCNO before > >ever locking the semaphore if a second backend

Re: [HACKERS] (full) Memory context dump considered harmful

2015-08-21 Thread Tom Lane
Tomas Vondra writes: > On 08/20/2015 11:04 PM, Stefan Kaltenbrunner wrote: >> On 08/20/2015 06:09 PM, Tom Lane wrote: >>> (The reason I say "lobotomize" is that there's no particularly >>> good reason to assume that the first N lines will tell you what you >>> need to know. And the filter rule wou

Re: [HACKERS] More WITH

2015-08-21 Thread Tom Lane
Robert Haas writes: > On Tue, Aug 18, 2015 at 11:23 PM, Tom Lane wrote: >> More generally, what would you hope to accomplish with such a construct >> that wouldn't be better done by writing the cursor's underlying query >> directly in the WITH clause? > Maybe I'm stupid today, but it seems like

Re: [HACKERS] Reduce ProcArrayLock contention

2015-08-21 Thread Robert Haas
On Fri, Aug 21, 2015 at 2:31 PM, Andres Freund wrote: > No, if it's paired like that, I don't think it's allowed to fail. > > But, as the code stands, there's absolutely no guarantee you're not > seeing something like: > P1: a = 0; > P1: b = 0; > P1: PGSemaphoreLock(&P1); > P2: a = 1; > P2: PGSema

Re: [HACKERS] More WITH

2015-08-21 Thread Robert Haas
On Fri, Aug 21, 2015 at 2:39 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Aug 18, 2015 at 11:23 PM, Tom Lane wrote: >>> More generally, what would you hope to accomplish with such a construct >>> that wouldn't be better done by writing the cursor's underlying query >>> directly in the WI

Re: [HACKERS] More WITH

2015-08-21 Thread Tom Lane
Robert Haas writes: > The existing syntax for FETCH already includes a way to specify the > number of rows you want to fetch, as in this example from the > documentation: > FETCH FORWARD 5 FROM liahona; > Why wouldn't that work here too? Mm, okay, but you still have the other objections to addres

Re: [HACKERS] More WITH

2015-08-21 Thread Robert Haas
On Fri, Aug 21, 2015 at 2:53 PM, Tom Lane wrote: > Robert Haas writes: >> The existing syntax for FETCH already includes a way to specify the >> number of rows you want to fetch, as in this example from the >> documentation: >> FETCH FORWARD 5 FROM liahona; >> Why wouldn't that work here too? > >

Re: [HACKERS] [PATCH v1] GSSAPI encryption support

2015-08-21 Thread Robbie Harwood
Michael Paquier writes: > On Fri, Jul 3, 2015 at 3:22 AM, Robbie Harwood wrote: > >> There are 8 commits in this series; I have tried to err on the side of >> creating too much separation rather than too little. A patch for each >> is attached. This is v1 of the series. > > I just had a quick

[HACKERS] minor typo in trigger.c

2015-08-21 Thread Merlin Moncure
diff --git a/src/backend/commands/trigger.c b/src/backend/commands/trigger.c index d169027..43421d6 100644 --- a/src/backend/commands/trigger.c +++ b/src/backend/commands/trigger.c @@ -4126,7 +4126,7 @@ AfterTriggerEndXact(bool isCommit) /* -* Forget the query stack and constrant-related

Re: [HACKERS] DBT-3 with SF=20 got failed

2015-08-21 Thread Tomas Vondra
Hello KaiGai-san, On 08/21/2015 02:28 AM, Kouhei Kaigai wrote: ... But what is the impact on queries that actually need more than 1GB of buckets? I assume we'd only limit the initial allocation and still allow the resize based on the actual data (i.e. the 9.5 improvement), so the queries would

Re: [HACKERS] Warnings around booleans

2015-08-21 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > On Friday, August 21, 2015, Tom Lane wrote: > >> It is not really acceptable to leave roles hanging around after "make > >> installcheck"; that would be a security hazard for the installation. > >> Please drop them. > > > The onl

Re: [HACKERS] Error message with plpgsql CONTINUE

2015-08-21 Thread Jim Nasby
On 8/17/15 4:46 PM, Tom Lane wrote: Jim Nasby writes: On 8/17/15 9:48 AM, Tom Lane wrote: I'm inclined to think that if we wanted to make this better, the way to improve it would be to detect the error*at compile time*, and get rid of this hack in plpgsql_exec_function altogether. So split

Re: [HACKERS] Patch for ginCombineData

2015-08-21 Thread Jeff Janes
On Wed, Aug 5, 2015 at 3:17 AM, Robert Abraham < robert.abraha...@googlemail.com> wrote: > Hello, > > we are using gin indexes on big tables. these tables happen to have > several billion rows. > the index creation fails in ginCombineData in src/backend/access/ginbulk.c > because repalloc is limit

Re: [HACKERS] (full) Memory context dump considered harmful

2015-08-21 Thread Tom Lane
I wrote: > I thought a bit more about this. Generally, what you want to know about > a given situation is which contexts have a whole lot of allocations > and/or a whole lot of child contexts. What you suggest above won't work > very well if the problem is buried more than about two levels down i

Re: [HACKERS] pg_dump quietly ignore missing tables - is it bug?

2015-08-21 Thread Jim Nasby
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, failed Spec compliant: not tested Documentation:not tested The feature doesn't seem to work: pg_dump -t t -t 'ii*' --strict-name

Re: [HACKERS] CTE optimization fence on the todo list?

2015-08-21 Thread Qingqing Zhou
On Fri, May 1, 2015 at 2:39 PM, Tom Lane wrote: > * Multiply-referenced WITH item (unless the outer query applies > identical constraints to each reference, which seems silly and not > worth the cycles to check for). > Not sure if I understand this correctly. Look at this query, CTE q is reference

Re: [HACKERS] PostgreSQL for VAX on NetBSD/OpenBSD

2015-08-21 Thread Greg Stark
[- the vax lists since they cause majordomo confirmation emails for anyone responding] On Thu, Aug 20, 2015 at 3:29 PM, Tom Lane wrote: > >> There are some planner tests that fail with floating point exceptions >> -- that's probably a bug on our part. And I've seen at least one >> server crash (m

Re: [HACKERS] Error message with plpgsql CONTINUE

2015-08-21 Thread Tom Lane
Jim Nasby writes: > On 8/17/15 9:48 AM, Tom Lane wrote: >>> I'm inclined to think that if we wanted to make this better, the way to >>> improve it would be to detect the error*at compile time*, and get rid of >>> this hack in plpgsql_exec_function altogether. > Here's a patch that does that. This

Re: [HACKERS] Error message with plpgsql CONTINUE

2015-08-21 Thread Jim Nasby
On 8/21/15 7:21 PM, Tom Lane wrote: Applied with some fixes. The major oversight was that EXIT does*not* have the same rules as CONTINUE, as is clearly documented (though in your defense, there was no regression test verifying the behavior ... there is now). Yay more tests. >I refactored the

Re: [HACKERS] Memory allocation in spi_printtup()

2015-08-21 Thread Tom Lane
Neil Conway writes: > On Mon, Aug 17, 2015 at 7:56 AM, Tom Lane wrote: >> Should we back-patch this change? Seems like it's arguably a >> performance bug. > Sounds good to me. Committed and back-patched. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsq

Re: [HACKERS] [PATCH] postgres_fdw extension support

2015-08-21 Thread Michael Paquier
On Sat, Aug 22, 2015 at 12:55 AM, Alvaro Herrera wrote: > Michael Paquier wrote: > > > if (needlabel) > > appendStringInfo(buf, "::%s", > > - > > format_type_with_typemod(node->consttype, > > - > > node->consttypmod)); > > + > > format_type_be_qualified(node->consttype));

Re: [HACKERS] DBT-3 with SF=20 got failed

2015-08-21 Thread Kouhei Kaigai
> Hello KaiGai-san, > > On 08/21/2015 02:28 AM, Kouhei Kaigai wrote: > ... > >> > >> But what is the impact on queries that actually need more than 1GB > >> of buckets? I assume we'd only limit the initial allocation and > >> still allow the resize based on the actual data (i.e. the 9.5 > >> impro

Re: [HACKERS] Declarative partitioning

2015-08-21 Thread Jim Nasby
On 8/18/15 12:31 PM, Josh Berkus wrote: Also this would be useful for range partitions: CREATE PARTITION ON parent_table USING ( start_value ); ... where start_value is the start range of the new partition. Again, easier for users to get correct. Instead of that, I think it would be more foo

Re: [HACKERS] (full) Memory context dump considered harmful

2015-08-21 Thread Tomas Vondra
On 08/21/2015 08:37 PM, Tom Lane wrote: Tomas Vondra writes: I also don't think logging just subset of the stats is a lost case. Sure, we can't know which of the lines are important, but for example logging just the top-level contexts with a summary of the child contexts would be OK. I thoug

Re: [HACKERS] Make HeapTupleSatisfiesMVCC more concurrent

2015-08-21 Thread Amit Kapila
On Fri, Aug 21, 2015 at 8:22 PM, Robert Haas wrote: > > On Wed, Aug 19, 2015 at 2:53 AM, Amit Kapila wrote: > > I think one case where the patch can impact is for aborted transactions. > > In TransactionIdIsInProgress(), we check for aborted transactions before > > consulting pg_subtrans whereas

Re: [HACKERS] [PATCH v1] GSSAPI encryption support

2015-08-21 Thread Michael Paquier
On Sat, Aug 22, 2015 at 4:06 AM, Robbie Harwood wrote: > > Michael Paquier writes: > > Going through the docs, the overall approach taken by the patch looks neat, > > and the default values as designed for both the client and the server are > > good things to do. Now actually looking at the code I

Re: [HACKERS] [DESIGN] ParallelAppend

2015-08-21 Thread Amit Kapila
On Fri, Aug 21, 2015 at 7:40 PM, Robert Haas wrote: > > On Tue, Aug 18, 2015 at 11:27 PM, Amit Kapila wrote: > >> Here is one other issue I found. Existing code assumes a TOC segment has > >> only one contents per node type, so it uses pre-defined key (like > >> PARALLEL_KEY_SCAN) per node type,

Re: [HACKERS] Test code is worth the space

2015-08-21 Thread Noah Misch
On Tue, Aug 18, 2015 at 02:03:19PM +0100, Greg Stark wrote: > On Tue, Aug 18, 2015 at 6:57 AM, Noah Misch wrote: > > My own position is based on having maintained a pg_regress suite an order of > > magnitude larger than that. I don't know why that outcome was so different. > And does your pg_reg

Re: [HACKERS] Autonomous Transaction is back

2015-08-21 Thread Noah Misch
On Fri, Aug 21, 2015 at 10:06:44AM -0400, Robert Haas wrote: > On Tue, Aug 18, 2015 at 3:06 PM, Merlin Moncure wrote: > > On Tue, Aug 18, 2015 at 8:17 AM, Robert Haas wrote: > >> On Sat, Aug 15, 2015 at 6:47 PM, Noah Misch wrote: > >>> [1] That's not to say it must use the shmem lock structures

Re: [HACKERS] PATCH: numeric timestamp in log_line_prefix

2015-08-21 Thread Fabien COELHO
Hello Tomas, from time to time I need to correlate PostgreSQL logs to other logs, containing numeric timestamps - a prime example of that is pgbench. With %t and %m that's not quite trivial, because of timezones etc. I propose adding two new log_line_prefix escape sequences - %T and %M, doing