Re: [HACKERS] tap tests remove working directories

2015-08-07 Thread Andrew Dunstan
On 08/07/2015 05:11 PM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: One of the things that makes the TAP tests very difficult and annoying to debug is their insistence on removing their data directories. I'm not sure why they are doing that. We don't do that with pg_regress.

Re: [HACKERS] Reduce ProcArrayLock contention

2015-08-07 Thread Andres Freund
On 2015-08-07 14:20:55 -0400, Jesper Pedersen wrote: On 08/07/2015 02:03 PM, Andres Freund wrote: Confirmed. Running w/o -P x and the problem goes away. Pushed the fix. Thanks for pointing the problem out! - Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Noah Misch
On Fri, Aug 07, 2015 at 03:18:06PM +0200, Andres Freund wrote: On 2015-08-06 23:23:43 -0400, Noah Misch wrote: On Thu, Aug 06, 2015 at 05:34:50PM +0200, Andres Freund wrote: On 2015-08-06 12:29:15 -0300, Alvaro Herrera wrote: Andres Freund wrote: @@ -0,0 +1,56 @@

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Andres Freund
On 2015-08-07 20:16:20 -0400, Noah Misch wrote: I agree that lock.h offers little to frontend code. Headers that the lockdefs.h patch made usable in the frontend, particularly genam.h and hash.h, are no better. It's not that simple. Those two, and tuptoaster.h, are actually somewhat validly

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Tom Lane
Andres Freund and...@anarazel.de writes: On 2015-08-07 20:16:20 -0400, Noah Misch wrote: On another note, I'm perplexed by the high speed commits from this thread. Commit de6fd1c landed just 191 minutes after you posted the first version of its patch. Now lockdefs.h is committed, right in the

Re: [HACKERS] Bug? Small samples in TABLESAMPLE SYSTEM returns zero rows

2015-08-07 Thread Josh Berkus
Petr, Just user-tested SYSTEM_ROWS and SYSTEM_TIME. They work as expected. Useful! -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] [DESIGN] ParallelAppend

2015-08-07 Thread Amit Kapila
On Fri, Aug 7, 2015 at 2:15 PM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: On Sat, Aug 1, 2015 at 6:39 PM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: If we pull Funnel here, I think the plan shall be as follows: Funnel -- SeqScan on rel1 -- PartialSeqScan on rel2

Re: [HACKERS] Re: [COMMITTERS] pgsql: Map basebackup tablespaces using a tablespace_map file

2015-08-07 Thread Amit Kapila
On Tue, Aug 4, 2015 at 8:45 AM, Amit Kapila amit.kapil...@gmail.com wrote: On Mon, Aug 3, 2015 at 7:44 PM, Fujii Masao masao.fu...@gmail.com wrote: BTW, while reading the code related to tablespace_map, I found that CancelBackup() emits the WARNING message online backup mode was not

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Andres Freund
On 2015-08-07 14:48:43 -0400, Tom Lane wrote: Indeed, but the buildfarm results say that the set of such platforms is nearly empty. It's very likely that a lot of third-party authors won't ever care if their code doesn't build on such platforms; certainly not nearly as much as they'll care if

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Andres Freund
On 2015-08-06 23:23:43 -0400, Noah Misch wrote: On Thu, Aug 06, 2015 at 05:34:50PM +0200, Andres Freund wrote: On 2015-08-06 12:29:15 -0300, Alvaro Herrera wrote: Andres Freund wrote: @@ -0,0 +1,56 @@ +/*- +

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Andres Freund
On 2015-08-05 21:39:26 -0400, Noah Misch wrote: On Wed, Aug 05, 2015 at 10:32:48AM -0400, Tom Lane wrote: Andres Freund and...@anarazel.de writes: Wasn't the point that 32 bit AIX as a whole hasn't been supported for a couple years now? My willingness to expend effort for that is rather

[HACKERS] ALTER SYSTEM and GUC_LIST_QUOTE

2015-08-07 Thread Adam Brightwell
All, While testing some behaviors with ALTER SYSTEM I discovered that GUC parameters with the GUC_LIST_QUOTE flag have a potential issue. As an example, ALTER SYSTEM SET shared_preload_libraries = ''; Results in the following output in postgresql.auto.conf: shared_preload_libraries = '';

Re: [HACKERS] tap tests remove working directories

2015-08-07 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: One of the things that makes the TAP tests very difficult and annoying to debug is their insistence on removing their data directories. I'm not sure why they are doing that. We don't do that with pg_regress. Instead we have clean targets to remove

Re: [HACKERS] 9.5 release notes

2015-08-07 Thread Gavin Flower
On 08/08/15 06:43, Bruce Momjian wrote: On Fri, Aug 7, 2015 at 11:53:30AM -0400, Robert Haas wrote: [...] Well, we could just throw a Postgres 9.5 is faster release note item in there and call it a day. ;-) Nah! Just say it is Much Shinier, I'll buy it. Unfortunately, we have to have much

Re: [HACKERS] WIP: SCRAM authentication

2015-08-07 Thread Michael Paquier
On Sat, Aug 8, 2015 at 3:45 AM, Heikki Linnakangas hlinn...@iki.fi wrote: On 08/07/2015 09:26 PM, Robert Haas wrote: Maybe I'm chiming in too late here but I am sorta unimpressed by this. If the user's password is stored both MD5-hashed and hashed some other way in the system catalogs, that's

Re: [HACKERS] Using quicksort and a merge step to significantly improve on tuplesort's single run external sort

2015-08-07 Thread Peter Geoghegan
On Fri, Jul 31, 2015 at 12:59 AM, Heikki Linnakangas hlinn...@iki.fi wrote: On 07/31/2015 02:01 AM, Peter Geoghegan wrote: What prevents the tuple at the top of the in-memory heap at the point of tuplesort_performsort() (say, one of the ones added to the heap as our glut of memory

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-08-07 Thread Kouhei Kaigai
I could have a discussion with Fujita-san about this topic. Also, let me share with the discussion towards entire solution. The primitive reason of this problem is, Scan node with scanrelid==0 represents a relation join that can involve multiple relations, thus, its TupleDesc of the records

[HACKERS] [sqlsmith] ERROR: too late to create a new PlaceHolderInfo

2015-08-07 Thread Andreas Seltenreich
Hi, on master at 36d9345, the attached queries raised too late to create a new PlaceHolderInfo. regards, Andreas select subq_218206.c0 as c0 from (select rel_1375794.sl_name as c0, coalesce(rel_1375793.f2, rel_1375793.f2) as c1 from public.subselect_tbl

Re: [HACKERS] [DESIGN] ParallelAppend

2015-08-07 Thread Kouhei Kaigai
On Sat, Aug 1, 2015 at 6:39 PM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: On Tue, Jul 28, 2015 at 6:08 PM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: I am not sure, but what problem do you see in putting Funnel node for one of the relation scans and not for the others. At

Re: [HACKERS] 9.5 release notes

2015-08-07 Thread Robert Haas
On Thu, Aug 6, 2015 at 10:24 PM, Bruce Momjian br...@momjian.us wrote: * 2014-10-02 [3acc10c9] Robert..: Increase the number of buffer mapping partition.. should we mention this? This has been patched by a number of downstream vendors and users, so it's probably worth calling out? Uh, I

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Robert Haas
On Thu, Aug 6, 2015 at 11:34 AM, Andres Freund and...@anarazel.de wrote: On 2015-08-06 12:29:15 -0300, Alvaro Herrera wrote: Andres Freund wrote: I had to split of three things: LOCKMASK, the individual lock levels and xl_standby_lock to be able to prohibit lock.h to be included by frontend

Re: [HACKERS] Reduce ProcArrayLock contention

2015-08-07 Thread Jesper Pedersen
On 08/07/2015 12:41 AM, Amit Kapila wrote: On Thu, Aug 6, 2015 at 9:36 PM, Robert Haas robertmh...@gmail.com wrote: OK, committed. Thank you. Fyi, there is something in pgbench that has caused a testing regression - havn't tracked down what yet. Against 9.6 server

Re: [HACKERS] Reduce ProcArrayLock contention

2015-08-07 Thread Robert Haas
On Fri, Aug 7, 2015 at 10:30 AM, Jesper Pedersen jesper.peder...@redhat.com wrote: Just thought I would post it in this thread, because this change does help on the performance numbers compared to 9.5 :) So are you saying that the performance was already worse before this patch landed, and then

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Noah Misch
On Fri, Aug 07, 2015 at 03:20:00PM +0200, Andres Freund wrote: Unless you have a better idea I'll now move the detection of that case to aix.h. Nope, that seemed right to me. I rather liked being able to emit a warning about disabling inlines *once* during configuration, but it's also

Re: [HACKERS] 9.5 release notes

2015-08-07 Thread Andres Freund
On 2015-08-07 11:53:30 -0400, Robert Haas wrote: On Thu, Aug 6, 2015 at 10:24 PM, Bruce Momjian br...@momjian.us wrote: * 2014-10-02 [3acc10c9] Robert..: Increase the number of buffer mapping partition.. should we mention this? This has been patched by a number of downstream vendors

Re: [HACKERS] Reduce ProcArrayLock contention

2015-08-07 Thread Amit Kapila
On Fri, Aug 7, 2015 at 8:00 PM, Jesper Pedersen jesper.peder...@redhat.com wrote: On 08/07/2015 12:41 AM, Amit Kapila wrote: On Thu, Aug 6, 2015 at 9:36 PM, Robert Haas robertmh...@gmail.com wrote: OK, committed. Thank you. Fyi, there is something in pgbench that has caused a

Re: [HACKERS] Reduce ProcArrayLock contention

2015-08-07 Thread Andres Freund
On 2015-08-07 20:17:28 +0530, Amit Kapila wrote: On Fri, Aug 7, 2015 at 8:00 PM, Jesper Pedersen jesper.peder...@redhat.com wrote: On 08/07/2015 12:41 AM, Amit Kapila wrote: On Thu, Aug 6, 2015 at 9:36 PM, Robert Haas robertmh...@gmail.com wrote: OK, committed. Thank you.

Re: [HACKERS] Autonomous Transaction is back

2015-08-07 Thread Robert Haas
On Thu, Aug 6, 2015 at 11:04 PM, Merlin Moncure mmonc...@gmail.com wrote: I don't necessarily disagree with what you're saying, but it's not clear to me what the proposed behavior is. Since the AT can commit before the outer, ISTM *any* ungranted lock requested by the AT but held by the outer

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Alvaro Herrera
Andres Freund wrote: lock.h includes lwlock.h only for the benefit of the three LockHashPartition* macros. Those macros are pieces of the lock.c implementation that cross into proc.c, not pieces of the lock.c public API. I argued that way as well upthread. But I do think that Tom

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Robert Haas
On Fri, Aug 7, 2015 at 2:27 PM, Andres Freund and...@anarazel.de wrote: On 2015-08-07 14:15:58 -0400, Robert Haas wrote: On Fri, Aug 7, 2015 at 1:11 PM, Andres Freund and...@anarazel.de wrote: On 2015-08-07 12:30:04 -0400, Robert Haas wrote: It may not be included from any IN CORE frontend

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Tom Lane
Andres Freund and...@anarazel.de writes: On 2015-08-07 14:32:35 -0400, Tom Lane wrote: Eventually I think we're going to have to spend some effort on making a clearer separation between front end safe and not front end safe header files. Until we do that, though, adding these #error

Re: [HACKERS] Reduce ProcArrayLock contention

2015-08-07 Thread Andres Freund
Hi, On 2015-08-07 19:30:46 +0200, Andres Freund wrote: On 2015-08-07 12:49:20 -0400, Jesper Pedersen wrote: No, this patch helps on performance - there is an improvement in numbers between

Re: [HACKERS] WIP: SCRAM authentication

2015-08-07 Thread Robert Haas
On Fri, Aug 7, 2015 at 3:22 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Tue, Aug 4, 2015 at 4:20 PM, Michael Paquier wrote: I have been looking more in depths at this one, which adds essential infrastructure to support multiple authentication hashes for more protocols. Here are

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Well, I just work here, but it seems silly to me to reorgnize the headers so that you can include fewer definitions where necessary, but then not revise the existing headers to use the slimmed-down versions where possible. Yeah, somebody might have to

Re: [HACKERS] 9.5 release notes

2015-08-07 Thread Bruce Momjian
On Fri, Aug 7, 2015 at 11:53:30AM -0400, Robert Haas wrote: On Thu, Aug 6, 2015 at 10:24 PM, Bruce Momjian br...@momjian.us wrote: * 2014-10-02 [3acc10c9] Robert..: Increase the number of buffer mapping partition.. should we mention this? This has been patched by a number of

Re: [HACKERS] 9.5 release notes

2015-08-07 Thread Andres Freund
On 2015-08-07 14:43:11 -0400, Bruce Momjian wrote: Well, we could just throw a Postgres 9.5 is faster release note item in there and call it a day. ;-) Based on my experience one of the prime reason people move to a new version of postgres is performance. And it's not like 'faster!!!' is

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Andres Freund
On 2015-08-07 12:30:04 -0400, Robert Haas wrote: It may not be included from any IN CORE frontend code, but that is not the same thing as saying it's not included from any frontend code at all. For example, EDB has code that includes namespace.h in frontend code. That compiled before this

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Andres Freund
On 2015-08-07 14:32:35 -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: Well, I just work here, but it seems silly to me to reorgnize the headers so that you can include fewer definitions where necessary, but then not revise the existing headers to use the slimmed-down

Re: [HACKERS] Reduce ProcArrayLock contention

2015-08-07 Thread Jesper Pedersen
On 08/07/2015 11:40 AM, Robert Haas wrote: On Fri, Aug 7, 2015 at 10:30 AM, Jesper Pedersen jesper.peder...@redhat.com wrote: Just thought I would post it in this thread, because this change does help on the performance numbers compared to 9.5 :) So are you saying that the performance was

Re: [HACKERS] WIP: SCRAM authentication

2015-08-07 Thread Heikki Linnakangas
On 08/07/2015 09:26 PM, Robert Haas wrote: Maybe I'm chiming in too late here but I am sorta unimpressed by this. If the user's password is stored both MD5-hashed and hashed some other way in the system catalogs, that's less secure than storing it in the least secure of those ways. And I'm

Re: [HACKERS] Reduce ProcArrayLock contention

2015-08-07 Thread Andres Freund
On 2015-08-07 12:49:20 -0400, Jesper Pedersen wrote: On 08/07/2015 11:40 AM, Robert Haas wrote: On Fri, Aug 7, 2015 at 10:30 AM, Jesper Pedersen jesper.peder...@redhat.com wrote: Just thought I would post it in this thread, because this change does help on the performance numbers compared to

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Robert Haas
On Fri, Aug 7, 2015 at 1:11 PM, Andres Freund and...@anarazel.de wrote: On 2015-08-07 12:30:04 -0400, Robert Haas wrote: It may not be included from any IN CORE frontend code, but that is not the same thing as saying it's not included from any frontend code at all. For example, EDB has code

Re: [HACKERS] Reduce ProcArrayLock contention

2015-08-07 Thread Jesper Pedersen
On 08/07/2015 02:03 PM, Andres Freund wrote: but you will have to use a 9.5 pgbench to see it, especially with higher client counts. Hm, you were using -P X, is that right? This bisects down to 1bc90f7a7b7441a88e2c6d4a0e9b6f9c1499ad30 - Remove thread-emulation support from pgbench. And the

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Andres Freund
On 2015-08-07 19:11:52 +0200, Andres Freund wrote: I think the likelihood of actually breaking correct working extension code that uses namespace.h that'd be broken if we removed lock.h from namespace.h is an order of magnitude bigger than the possible impact on frontend code. Same with

Re: [HACKERS] All-zero page in GIN index causes assertion failure

2015-08-07 Thread Alvaro Herrera
-- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training Services diff --git a/src/backend/access/brin/brin_pageops.c b/src/backend/access/brin/brin_pageops.c index 0b257d9..909e4c6 100644 ---

Re: [HACKERS] All-zero page in GIN index causes assertion failure

2015-08-07 Thread Alvaro Herrera
Heikki Linnakangas wrote: BRIN update is not quite right, however. brin_getinsertbuffer() can initialize a page, but the caller might bail out without using the page and WAL-logging the change. If that happens, the next update that uses the same page will WAL-log the change but it will not

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Andres Freund
On 2015-08-07 14:15:58 -0400, Robert Haas wrote: On Fri, Aug 7, 2015 at 1:11 PM, Andres Freund and...@anarazel.de wrote: On 2015-08-07 12:30:04 -0400, Robert Haas wrote: It may not be included from any IN CORE frontend code, but that is not the same thing as saying it's not included from

Re: [HACKERS] Race conditions in shm_mq.c

2015-08-07 Thread Robert Haas
On Thu, Aug 6, 2015 at 5:59 PM, Antonin Houska a...@cybertec.at wrote: Robert Haas robertmh...@gmail.com wrote: On Thu, Aug 6, 2015 at 2:38 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Aug 6, 2015 at 10:10 AM, Antonin Houska a...@cybertec.at wrote: During my experiments with parallel

Re: [HACKERS] Reduce ProcArrayLock contention

2015-08-07 Thread Jesper Pedersen
On 08/07/2015 10:47 AM, Amit Kapila wrote: Fyi, there is something in pgbench that has caused a testing regression - havn't tracked down what yet. Against 9.6 server (846f8c9483a8f31e45bf949db1721706a2765771) 9.6 pgbench: progress: 10.0 s, 53525.0 tps, lat 1.485 ms stddev 0.523

Re: [HACKERS] ALTER SYSTEM and GUC_LIST_QUOTE

2015-08-07 Thread Tom Lane
Adam Brightwell adam.brightw...@crunchydatasolutions.com writes: While testing some behaviors with ALTER SYSTEM I discovered that GUC parameters with the GUC_LIST_QUOTE flag have a potential issue. As an example, ALTER SYSTEM SET shared_preload_libraries = ''; Results in the following output

Re: [HACKERS] 9.5 release notes

2015-08-07 Thread Peter Geoghegan
On Fri, Aug 7, 2015 at 12:02 PM, Andres Freund and...@anarazel.de wrote: Based on my experience one of the prime reason people move to a new version of postgres is performance. And it's not like 'faster!!!' is really helpful for them to evaluate the benefits appropriately. A scalability

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-07 Thread Alvaro Herrera
Andres Freund wrote: You re-added the #ifndef FRONTEND there in a9baeb361d635 referencing a buildfarm failure. It seems to originally have been added in 7507b193bc54 referencing buildfarm member warthog which unfortunately doesn't exist anymore... FWIW we make a point of not reusing

[HACKERS] tap tests remove working directories

2015-08-07 Thread Andrew Dunstan
One of the things that makes the TAP tests very difficult and annoying to debug is their insistence on removing their data directories. I'm not sure why they are doing that. We don't do that with pg_regress. Instead we have clean targets to remove them if necessary. I suggest that we either