Re: [HACKERS] Parallelized polymorphic aggs, and aggtype vs aggoutputtype

2016-06-21 Thread David Rowley
On 20 June 2016 at 22:19, David Rowley wrote: > I've gone and implemented the dummy argument approach for > deserialization functions. > > If we go with this, I can then write the docs for 35.10 which'll serve > to explain parallel user defined aggregates in detail. I've attached my proposed patc

Re: [HACKERS] primary_conninfo missing from pg_stat_wal_receiver

2016-06-21 Thread Michael Paquier
On Wed, Jun 22, 2016 at 12:04 AM, Stephen Frost wrote: > Ugh. I would certainly rather not have yet another special, hard-coded, > bit of logic that magically makes things available to a superuser when > it's not available to regular users. > What that results in is the need to have a new default

Re: [HACKERS] Missing checks when malloc returns NULL...

2016-06-21 Thread Michael Paquier
On Tue, Jun 21, 2016 at 10:46 PM, Tom Lane wrote: > Michael Paquier writes: >> - mcxt.c uses that, which is surprising: >> @@ -704,7 +704,8 @@ MemoryContextCreate(NodeTag tag, Size size, >> { >> /* Special case for startup: use good ol' malloc */ >> node = (MemoryContext) mall

[HACKERS] signed division in hash_search_with_hash_value(ENTER) has high overhead

2016-06-21 Thread Andres Freund
Hi, During several profile runs I've seen the division in if (action == HASH_ENTER || action == HASH_ENTER_NULL) { /* * Can't split if running in partitioned mode, nor if frozen, nor if * table is the subject of any active hash_seq

Re: [HACKERS] Speaking of breaking compatibility...standard_conforming_strings

2016-06-21 Thread Bruce Momjian
On Tue, May 24, 2016 at 04:07:02PM -0400, David G. Johnston wrote: > On Tue, May 24, 2016 at 3:57 PM, Tom Lane wrote: > > "David G. Johnston" writes: > > I just noticed this comment in scan.l: > > /* > >  * GUC variables.  This is a DIRECT violation of the warning given at > the

Re: [HACKERS] MultiXactId error after upgrade to 9.3.4

2016-06-21 Thread Alvaro Herrera
Alvaro Herrera wrote: > Robert Haas wrote: > > On Fri, Jun 17, 2016 at 9:33 AM, Andrew Gierth > > wrote: > > >> "Robert" == Robert Haas writes: > > > >> Why is the correct rule not "check for and ignore pre-upgrade mxids > > > >> before even trying to fetch members"? > > > > > > Robert> I

Re: [HACKERS] Requesting external_pid_file with postgres -C when not initialized lead to coredump

2016-06-21 Thread Michael Paquier
On Wed, Jun 22, 2016 at 2:02 AM, Alvaro Herrera wrote: > alain radix wrote: >> So, here is my first patch for PostgreSQL. > > Looking forward to further ones, A comment on this patch: this is an incorrect approach anyway. PostmasterMain relies on this value being NULL to decide if this PID file s

Re: [HACKERS] Rethinking behavior of force_parallel_mode = regress

2016-06-21 Thread Tom Lane
Robert Haas writes: > On Sat, Jun 18, 2016 at 4:49 PM, Tom Lane wrote: >> With that thought in mind, I propose that the behavior of >> force_parallel_mode = regress is ill-designed so far as EXPLAIN is >> concerned. What it ought to do is suppress *all* Gathers from the output, >> not just ones

Re: [HACKERS] Rethinking behavior of force_parallel_mode = regress

2016-06-21 Thread Robert Haas
On Sat, Jun 18, 2016 at 4:49 PM, Tom Lane wrote: > As of HEAD it is possible to get through all of our regression tests > with these settings: > > alter system set force_parallel_mode = regress; > alter system set max_parallel_workers_per_gather = 2; > alter system set parallel_tuple_cost = 0; > a

Re: [HACKERS] Reviewing freeze map code

2016-06-21 Thread Robert Haas
On Tue, Jun 21, 2016 at 3:46 PM, Andres Freund wrote: > On 2016-06-21 15:38:25 -0400, Robert Haas wrote: >> On Tue, Jun 21, 2016 at 1:49 PM, Andres Freund wrote: >> >> I'm also a bit dubious that LockAcquire is safe to call in general >> >> with interrupts held. >> > >> > Looks like we could just

[HACKERS] EXISTS clauses not being optimized in the face of 'one time pass' optimizable expressions

2016-06-21 Thread Merlin Moncure
Hello hackers, Observe the following test case (apologies if this is a well understood problem): create temp table foo as select generate_series(1,100) id; create index on foo(id); create temp table bar as select id, id % 10 = 0 as good from generate_series(1,100) id; create index on

Re: [HACKERS] MultiXactId error after upgrade to 9.3.4

2016-06-21 Thread Alvaro Herrera
Robert Haas wrote: > On Fri, Jun 17, 2016 at 9:33 AM, Andrew Gierth > wrote: > >> "Robert" == Robert Haas writes: > > >> Why is the correct rule not "check for and ignore pre-upgrade mxids > > >> before even trying to fetch members"? > > > > Robert> I entirely believe that's the correct ru

Re: [HACKERS] Reviewing freeze map code

2016-06-21 Thread Andres Freund
On 2016-06-21 15:38:25 -0400, Robert Haas wrote: > On Tue, Jun 21, 2016 at 1:49 PM, Andres Freund wrote: > >> I'm also a bit dubious that LockAcquire is safe to call in general > >> with interrupts held. > > > > Looks like we could just acquire the tuple-lock *before* doing the > > toast_insert_or

Re: [HACKERS] [COMMITTERS] pgsql: Try again to fix the way the scanjoin_target is used with partia

2016-06-21 Thread Tom Lane
Robert Haas writes: > On Tue, Jun 21, 2016 at 1:28 PM, Tom Lane wrote: >> Also, I got rid of the target_parallel argument to >> apply_projection_to_path, as I thought that that was just way too much >> interconnection between apply_projection_to_path and its callers than >> is justified for what

Re: [HACKERS] Reviewing freeze map code

2016-06-21 Thread Robert Haas
On Tue, Jun 21, 2016 at 1:49 PM, Andres Freund wrote: >> I'm also a bit dubious that LockAcquire is safe to call in general >> with interrupts held. > > Looks like we could just acquire the tuple-lock *before* doing the > toast_insert_or_update/RelationGetBufferForTuple, but after releasing > the

[HACKERS] Re: [COMMITTERS] pgsql: Try again to fix the way the scanjoin_target is used with partia

2016-06-21 Thread Robert Haas
On Tue, Jun 21, 2016 at 1:28 PM, Tom Lane wrote: > I wrote: >> If we keep it like this, we definitely ought to refactor things so that >> fewer places are aware of the possibility of the Result getting omitted. >> Maybe push that logic into create_projection_path? If we did, we might >> not need

Re: [HACKERS] pg_bsd_indent - improvements around offsetof and sizeof

2016-06-21 Thread Bruce Momjian
On Tue, Jun 21, 2016 at 03:22:09PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Wed, May 25, 2016 at 03:13:23PM -0400, Tom Lane wrote: > >> ... If we make a change like this, I think we should > >> *strongly* consider reindenting all the live back branches along with > >> HEAD. > > > Uh,

Re: [HACKERS] pg_bsd_indent - improvements around offsetof and sizeof

2016-06-21 Thread Tom Lane
Bruce Momjian writes: > On Wed, May 25, 2016 at 03:13:23PM -0400, Tom Lane wrote: >> ... If we make a change like this, I think we should >> *strongly* consider reindenting all the live back branches along with >> HEAD. > Uh, we have been running on back branches anytime the pgindent rules > chan

Re: [HACKERS] Gin index on array of uuid

2016-06-21 Thread Tom Lane
Enrique MailingLists writes: > Currently creating an index on an array of UUID involves defining an > operator class. I was wondering if this would be a valid request to add as > part of the uuid-ossp extension? This seems like a reasonable operator to > support as a default for UUIDs. This makes

Re: [HACKERS] pg_bsd_indent - improvements around offsetof and sizeof

2016-06-21 Thread Bruce Momjian
On Wed, May 25, 2016 at 03:13:23PM -0400, Tom Lane wrote: > Robert Haas writes: > > On Sun, May 22, 2016 at 4:16 PM, Piotr Stefaniak > > wrote: > >> I think I've managed to improve pg_bsd_indent's handling of two types of > >> cases. > > > Wow, that seems pretty great. I haven't scrutinized you

Re: [HACKERS] Gin index on array of uuid

2016-06-21 Thread Peter Geoghegan
On Tue, Jun 21, 2016 at 11:42 AM, Enrique MailingLists wrote: > This would be helpful for people trying to use arrays of UUIDs in cloud > environments which limit root access. I have personally seen numerous requests for this from users of Heroku Postgres. So, I agree that there is a demand for t

[HACKERS] Gin index on array of uuid

2016-06-21 Thread Enrique MailingLists
Currently creating an index on an array of UUID involves defining an operator class. I was wondering if this would be a valid request to add as part of the uuid-ossp extension? This seems like a reasonable operator to support as a default for UUIDs. Any downsides to adding this as a default? http:

Re: [HACKERS] Reviewing freeze map code

2016-06-21 Thread Andres Freund
On 2016-06-21 13:03:24 -0400, Robert Haas wrote: > On Tue, Jun 21, 2016 at 12:54 PM, Andres Freund wrote: > >> I don't quite understand the intended semantics of this proposed flag. > > > > Whenever the flag is set, we have to acquire the heavyweight tuple lock > > before continuing. That guarante

Re: [HACKERS] 10.0

2016-06-21 Thread José Luis Tallón
On 06/20/2016 10:14 PM, Robert Haas wrote: On Mon, Jun 20, 2016 at 4:00 PM, David G. Johnston wrote: 10.x is the desired output. 10.x is the output that some people desire. (explicitly skipped up-thread to add this -- please forgive my jumping in) Since we are still (as a community) debatin

Re: [HACKERS] Requesting external_pid_file with postgres -C when not initialized lead to coredump

2016-06-21 Thread Alvaro Herrera
alain radix wrote: > Hi, > > I faced a coredump when reading the value of the parameter > "external_pid_file" when it was not initialized in postgresql.conf > This came from the value not being specified to be initialized to en empty > string in guc.c in the ConfigureNamesString array. > the behav

Re: [HACKERS] Reviewing freeze map code

2016-06-21 Thread Robert Haas
On Tue, Jun 21, 2016 at 12:54 PM, Andres Freund wrote: >> I don't quite understand the intended semantics of this proposed flag. > > Whenever the flag is set, we have to acquire the heavyweight tuple lock > before continuing. That guarantees nobody else can modify the tuple, > while the lock is re

Re: [HACKERS] MultiXactId error after upgrade to 9.3.4

2016-06-21 Thread Robert Haas
On Fri, Jun 17, 2016 at 9:33 AM, Andrew Gierth wrote: >> "Robert" == Robert Haas writes: > >> Why is the correct rule not "check for and ignore pre-upgrade mxids > >> before even trying to fetch members"? > > Robert> I entirely believe that's the correct rule, but doesn't > Robert> implem

Re: [HACKERS] Reviewing freeze map code

2016-06-21 Thread Andres Freund
On 2016-06-21 10:50:36 -0400, Robert Haas wrote: > On Mon, Jun 20, 2016 at 11:51 PM, Andres Freund wrote: > >> > So far the best idea I have - and it's really not a good one - is to > >> > invent a new hint-bit that tells concurrent updates to acquire a > >> > heavyweight tuple lock, while releasi

Re: [HACKERS] [GENERAL] PgQ and pg_dump

2016-06-21 Thread Martín Marqués
2016-06-21 13:08 GMT-03:00 Robert Haas : > On Thu, Jun 16, 2016 at 1:46 PM, Martín Marqués > wrote: >> The comment is accurate on what is going to be dumpable and what's not >> from the code. In our case, as the pgq schema is not dumpable becaause >> it comes from an extension, other objects it c

Re: [HACKERS] Choosing the cheapest optimizer cost

2016-06-21 Thread Bruce Momjian
On Tue, Jun 21, 2016 at 11:17:19AM -0400, Robert Haas wrote: > If the index scans are parameterized by values from the seq scan, > which is likely the situation in which this sort of plan will be > generated, we'll pay the extra cost of building the hash table once > per row in something_big. > >

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

2016-06-21 Thread Bruce Momjian
On Tue, Jun 21, 2016 at 12:12:34PM -0400, Robert Haas wrote: > >> > What is confusing you? > >> > >> I don't think I'm confused. Sure, you can do that, but the effects of > >> any writes performed on the new cluster will not be there when you > >> revert back to the old cluster. So you will have

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

2016-06-21 Thread Bruce Momjian
On Tue, Jun 21, 2016 at 08:56:09PM +0800, Craig Ringer wrote: > Also, if you run *with* --link, IIRC there's no guarantee that the old version > will be happy to see any new infomask bits etc introduced by the new Pg. I Well, we only write system tables in pg_upgrade in the new cluster, and those

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

2016-06-21 Thread Robert Haas
On Tue, Jun 21, 2016 at 11:34 AM, Bruce Momjian wrote: > On Tue, Jun 21, 2016 at 08:19:55AM -0400, Robert Haas wrote: >> On Mon, Jun 20, 2016 at 10:08 PM, Bruce Momjian wrote: >> >> No, not really. Once you let write transactions into the new cluster, >> >> there's no way to get back to the old

Re: [HACKERS] [GENERAL] PgQ and pg_dump

2016-06-21 Thread Robert Haas
On Thu, Jun 16, 2016 at 1:46 PM, Martín Marqués wrote: > The comment is accurate on what is going to be dumpable and what's not > from the code. In our case, as the pgq schema is not dumpable becaause > it comes from an extension, other objects it contain will not be > dumpable as well. > > That's

Re: [HACKERS] Hash Indexes

2016-06-21 Thread Robert Haas
On Thu, Jun 16, 2016 at 3:28 AM, Amit Kapila wrote: >> Incomplete splits can be completed either by vacuum or insert as both >> needs exclusive lock on bucket. If vacuum finds split-in-progress flag on a >> bucket then it will complete the split operation, vacuum won't see this flag >> if actuall

Re: [HACKERS] Hash Indexes

2016-06-21 Thread Robert Haas
On Tue, May 10, 2016 at 8:09 AM, Amit Kapila wrote: > > For making hash indexes usable in production systems, we need to improve its > concurrency and make them crash-safe by WAL logging them. The first problem > I would like to tackle is improve the concurrency of hash indexes. First > advan

[HACKERS] Requesting external_pid_file with postgres -C when not initialized lead to coredump

2016-06-21 Thread alain radix
Hi, I faced a coredump when reading the value of the parameter "external_pid_file" when it was not initialized in postgresql.conf This came from the value not being specified to be initialized to en empty string in guc.c in the ConfigureNamesString array. the behavior can easily been tested with t

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

2016-06-21 Thread Bruce Momjian
On Tue, Jun 21, 2016 at 08:19:55AM -0400, Robert Haas wrote: > On Mon, Jun 20, 2016 at 10:08 PM, Bruce Momjian wrote: > >> No, not really. Once you let write transactions into the new cluster, > >> there's no way to get back to the old server version no matter which > >> option you used. > > > >

Re: [HACKERS] Choosing the cheapest optimizer cost

2016-06-21 Thread Robert Haas
On Wed, Jun 15, 2016 at 3:46 PM, Bruce Momjian wrote: > Right now, the optimizer chooses the path with the cheapest cost. > > However, do we take into account the behavior of the plan in handling > mis-estimated row counts? No. > For example, if a path has a log(n) behavior > for changes in the

Re: [HACKERS] primary_conninfo missing from pg_stat_wal_receiver

2016-06-21 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Michael Paquier writes: > > On Tue, Jun 21, 2016 at 11:29 AM, Tom Lane wrote: > >> What I would want to know is whether this specific change is actually a > >> good idea. In particular, I'm concerned about the possible security > >> implications of exposi

Re: [HACKERS] Reviewing freeze map code

2016-06-21 Thread Robert Haas
On Tue, Jun 21, 2016 at 10:47 AM, Tom Lane wrote: > Robert Haas writes: >> On Mon, Jun 20, 2016 at 5:59 PM, Andres Freund wrote: >>> Consider what happens if we, after restarting at l2, notice that we >>> can't actually insert, but return in the !HeapTupleMayBeUpdated >>> branch. > >> OK, I see

Re: [HACKERS] Reviewing freeze map code

2016-06-21 Thread Robert Haas
On Mon, Jun 20, 2016 at 11:51 PM, Andres Freund wrote: >> > So far the best idea I have - and it's really not a good one - is to >> > invent a new hint-bit that tells concurrent updates to acquire a >> > heavyweight tuple lock, while releasing the page-level lock. If that >> > hint bit does not re

Re: [HACKERS] Reviewing freeze map code

2016-06-21 Thread Tom Lane
Robert Haas writes: > On Mon, Jun 20, 2016 at 5:59 PM, Andres Freund wrote: >> Consider what happens if we, after restarting at l2, notice that we >> can't actually insert, but return in the !HeapTupleMayBeUpdated >> branch. > OK, I see what you mean. Still, that doesn't seem like such a > terr

Re: [HACKERS] Reviewing freeze map code

2016-06-21 Thread Robert Haas
On Mon, Jun 20, 2016 at 5:59 PM, Andres Freund wrote: > On 2016-06-20 17:55:19 -0400, Robert Haas wrote: >> On Mon, Jun 20, 2016 at 4:24 PM, Andres Freund wrote: >> > On 2016-06-20 16:10:23 -0400, Robert Haas wrote: >> >> I >> >> mean, suppose we just don't do any of that before we go off to do >

Re: [HACKERS] Precedence of new phrase search tsquery operator

2016-06-21 Thread Teodor Sigaev
On Wed, Jun 8, 2016 at 7:13 PM, Tom Lane wrote: It appears that the new <-> operator has been made to have exactly the same grammatical precedence as the existing & (AND) operator. Thus, for example, 'a & b <-> c'::tsquery means something different from 'b <-> c & a'::tsquery: I find this surpr

Re: [HACKERS] Missing checks when malloc returns NULL...

2016-06-21 Thread Tom Lane
Michael Paquier writes: > - mcxt.c uses that, which is surprising: > @@ -704,7 +704,8 @@ MemoryContextCreate(NodeTag tag, Size size, > { > /* Special case for startup: use good ol' malloc */ > node = (MemoryContext) malloc(needed); > - Assert(node != NULL); > + if (

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

2016-06-21 Thread Craig Ringer
On 21 June 2016 at 20:19, Robert Haas wrote: > On Mon, Jun 20, 2016 at 10:08 PM, Bruce Momjian wrote: > > On Fri, May 20, 2016 at 07:40:53PM -0400, Robert Haas wrote: > >> On Mon, May 16, 2016 at 3:36 AM, Bruce Momjian > wrote: > >> > On Sun, May 15, 2016 at 03:23:52PM -0500, Jim Nasby wrote: >

Re: [HACKERS] Postgres_fdw join pushdown - wrong results with whole-row reference

2016-06-21 Thread Etsuro Fujita
On 2016/06/21 21:37, Ashutosh Bapat wrote: How about using a system column eg, ctid, for the CASE WHEN conversion; in Rushabh's example the reference to "r1" would be converted with "CASE WHEN r1.ctid IS NOT NULL THEN ROW(r1.empno, r1.ename, r1.job, r1.mgr, r1.hiredate, r1.sal, r

Re: [HACKERS] Postgres_fdw join pushdown - wrong results with whole-row reference

2016-06-21 Thread Ashutosh Bapat
> How about using a system column eg, ctid, for the CASE WHEN conversion; in > Rushabh's example the reference to "r1" would be converted with "CASE WHEN > r1.ctid IS NOT NULL THEN ROW(r1.empno, r1.ename, r1.job, r1.mgr, > r1.hiredate, r1.sal, r1.comm, r1.deptno) END". IMO I think that that would

Re: [HACKERS] Postgres_fdw join pushdown - wrong results with whole-row reference

2016-06-21 Thread Etsuro Fujita
On 2016/06/21 20:42, Ashutosh Bapat wrote: On Tue, Jun 21, 2016 at 4:36 PM, Amit Langote mailto:langote_amit...@lab.ntt.co.jp>> wrote: On 2016/06/21 16:27, Rushabh Lathia wrote: > Now I was under impression the IS NOT NULL should be always in inverse of > IS NULL, but clearly h

Re: [HACKERS] Lets (not) break all the things. Was: [pgsql-advocacy] 9.6 -> 10.0

2016-06-21 Thread Robert Haas
On Mon, Jun 20, 2016 at 10:08 PM, Bruce Momjian wrote: > On Fri, May 20, 2016 at 07:40:53PM -0400, Robert Haas wrote: >> On Mon, May 16, 2016 at 3:36 AM, Bruce Momjian wrote: >> > On Sun, May 15, 2016 at 03:23:52PM -0500, Jim Nasby wrote: >> >> 2) There's no ability at all to revert, other than r

Re: [HACKERS] Parallel query and temp_file_limit

2016-06-21 Thread Robert Haas
On Mon, Jun 20, 2016 at 11:01 PM, Tom Lane wrote: > Peter Geoghegan writes: >> On Wed, May 18, 2016 at 3:40 AM, Robert Haas wrote: >>> What I'm tempted to do is trying to document that, as a point of >>> policy, parallel query in 9.6 uses up to (workers + 1) times the >>> resources that a single

Re: [HACKERS] Postgres_fdw join pushdown - wrong results with whole-row reference

2016-06-21 Thread Ashutosh Bapat
On Tue, Jun 21, 2016 at 4:36 PM, Amit Langote wrote: > On 2016/06/21 16:27, Rushabh Lathia wrote: > > Now I was under impression the IS NOT NULL should be always in inverse of > > IS NULL, but clearly here its not the case with wholerow. But further > > looking at > > the document its saying diff

Re: [HACKERS] Postgres_fdw join pushdown - wrong results with whole-row reference

2016-06-21 Thread Amit Langote
On 2016/06/21 16:27, Rushabh Lathia wrote: > Now I was under impression the IS NOT NULL should be always in inverse of > IS NULL, but clearly here its not the case with wholerow. But further > looking at > the document its saying different thing for wholerow: > > https://www.postgresql.org/docs/9.

Re: [HACKERS] 10.0

2016-06-21 Thread Cédric Villemain
On 20/06/2016 22:41, Alvaro Herrera wrote: > Tom Lane wrote: >> Robert Haas writes: >>> On Mon, Jun 20, 2016 at 4:00 PM, David G. Johnston >>> wrote: 10.x is the desired output. >> >>> 10.x is the output that some people desire. A significant number of >>> people, including me, would prefer

Re: [HACKERS] Declarative partitioning

2016-06-21 Thread Amit Langote
On 2016/06/21 16:49, Ashutosh Bapat wrote: > Hi Amit, > I tried creating partitioned table by range on an expression like > CREATE TABLE pt1_e (a int, b int, c varchar) PARTITION BY RANGE(a + > b); > ERROR: syntax error at or near > "+" > > LINE 1: ...E pt1_e (a int, b int, c varchar) PARTITI

Re: [HACKERS] Declarative partitioning

2016-06-21 Thread Ashutosh Bapat
Hi Amit, I tried creating partitioned table by range on an expression like CREATE TABLE pt1_e (a int, b int, c varchar) PARTITION BY RANGE(a + b); ERROR: syntax error at or near "+" LINE 1: ...E pt1_e (a int, b int, c varchar) PARTITION BY RANGE(a + b); But when I try to create partitioned t

[HACKERS] Postgres_fdw join pushdown - wrong results with whole-row reference

2016-06-21 Thread Rushabh Lathia
Below query returns the wrong result when join getting pushdown to the remote server. (PFA fdw_setup.sql, to create objects for the test) postgres=# select e, e.empno, d.deptno, d.dname from f_emp e left join f_dept d on e.deptno = d.deptno and e.sal > 3000 order by 2, 3 limit 10;