Re: [HACKERS] [COMMITTERS] pgsql: Allow Pin/UnpinBuffer to operate in a lockfree manner.

2016-04-13 Thread Andres Freund
On 2016-04-13 20:09:46 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2016-04-13 11:20:19 -0700, Andres Freund wrote: > >> Heh, I was wondering the same aftering sending the last email. Will do > >> that then. > > > Pushed. Let's see what pademelon says... > > I lit off a run, but it'll

Re: [HACKERS] Postgres_fdw join pushdown - INNER - FULL OUTER join combination generating wrong result

2016-04-13 Thread Ashutosh Bapat
On Thu, Apr 14, 2016 at 8:42 AM, Etsuro Fujita wrote: > On 2016/03/29 23:20, Ashutosh Bapat wrote: > >> I think the reason for that is in foreign_join_ok. This in that >> function: >> >> wrongly pulls up remote_conds from joining relations in the FULL >> JOIN case. I think we sh

Re: [HACKERS] Detrimental performance impact of ringbuffers on performance

2016-04-13 Thread Jeff Janes
On Tue, Apr 12, 2016 at 11:38 AM, Andres Freund wrote: > >> The bottom line >> here, IMHO, is not that there's anything wrong with our ring buffer >> implementation, but that if you run PostgreSQL on a system where the >> I/O is hitting a 5.25" floppy (not to say 8") the performance may be >> le

Re: [HACKERS] [BUGS] Breakage with VACUUM ANALYSE + partitions

2016-04-13 Thread Abhijit Menon-Sen
At 2016-04-12 09:00:57 -0400, robertmh...@gmail.com wrote: > > On Mon, Apr 11, 2016 at 1:17 PM, Andres Freund wrote: > > > > 3) Actually handle the case of the last open segment not being > >RELSEG_SIZE properly in _mdfd_getseg() - mdnblocks() does so. > > #3 seems like it's probably about 15

Re: [HACKERS] Detrimental performance impact of ringbuffers on performance

2016-04-13 Thread Peter Geoghegan
On Tue, Apr 12, 2016 at 11:38 AM, Andres Freund wrote: >> And, on the other hand, if we don't do something like that, it will be >> quite an exceptional case to find anything on the free list. Doing it >> just to speed up developer benchmarking runs seems like the wrong >> idea. > > I don't think

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-13 Thread Michael Paquier
On Thu, Apr 14, 2016 at 11:45 AM, Amit Kapila wrote: > Yes, this is what I was trying to explain to Fujii-san upthread and I have > also verified that the same works on Windows. If you could, it would be nice as well to check that nothing breaks with VS when using vcregress recoverycheck. -- Mich

Re: [HACKERS] [COMMITTERS] pgsql: Add regression tests for multiple synchronous standbys.

2016-04-13 Thread Michael Paquier
On Thu, Apr 14, 2016 at 11:29 AM, Fujii Masao wrote: > On Wed, Apr 13, 2016 at 4:54 PM, Michael Paquier > wrote: >> On Fri, Apr 8, 2016 at 4:49 PM, Fujii Masao wrote: >>> Add regression tests for multiple synchronous standbys. >>> >>> Authors: Suraj Kharage, Michael Paquier, Masahiko Sawada, ref

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-13 Thread Kyotaro HORIGUCHI
At Thu, 14 Apr 2016 12:42:06 +0900, Fujii Masao wrote in > > Yes, this is what I was trying to explain to Fujii-san upthread and I have > > also verified that the same works on Windows. > > Oh, okay, understood. Thanks for explaining that! > > > I think one point which we > > should try to ens

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-04-13 Thread Robert Haas
On Wed, Apr 13, 2016 at 11:21 PM, Etsuro Fujita wrote: >>> 2. When a join is pushed down, deparse system columns using something >>> like "CASE WHEN r1.* IS NOT NULL THEN 0 END", except for the table OID >>> column, which gets deparsed with the table OID in place of 0. This >>> delivers the corre

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-13 Thread Fujii Masao
On Thu, Apr 14, 2016 at 11:45 AM, Amit Kapila wrote: > On Wed, Apr 13, 2016 at 1:44 PM, Kyotaro HORIGUCHI > wrote: >> >> At Wed, 13 Apr 2016 04:43:35 +0900, Fujii Masao >> wrote in >> >> > >>> Thank you for reviewing. >> > >>> >> > >>> SyncRepUpdateConfig() seems to be no longer necessary. >> >

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-04-13 Thread Etsuro Fujita
On 2016/04/14 12:04, Etsuro Fujita wrote: On 2016/04/14 4:57, Robert Haas wrote: 2. When a join is pushed down, deparse system columns using something like "CASE WHEN r1.* IS NOT NULL THEN 0 END", except for the table OID column, which gets deparsed with the table OID in place of 0. This delive

Re: [HACKERS] Performance degradation in commit 6150a1b0

2016-04-13 Thread Amit Kapila
On Wed, Apr 13, 2016 at 9:10 AM, Robert Haas wrote: > > On Tue, Apr 12, 2016 at 10:30 PM, Noah Misch wrote: > > That sounds like this open item is ready for CLOSE_WAIT status; is it? > > I just retested this on power2. Here are the results. I retested > 3fed4174 and 6150a1b0 plus master as of d

Re: [HACKERS] Postgres_fdw join pushdown - INNER - FULL OUTER join combination generating wrong result

2016-04-13 Thread Etsuro Fujita
On 2016/03/29 23:20, Ashutosh Bapat wrote: I think the reason for that is in foreign_join_ok. This in that function: wrongly pulls up remote_conds from joining relations in the FULL JOIN case. I think we should not pull up such conditions in the FULL JOIN case. Right. Fo

Re: [HACKERS] Performance degradation in commit 6150a1b0

2016-04-13 Thread Noah Misch
On Tue, Apr 12, 2016 at 11:40:43PM -0400, Robert Haas wrote: > On Tue, Apr 12, 2016 at 10:30 PM, Noah Misch wrote: > > That sounds like this open item is ready for CLOSE_WAIT status; is it? > > I just retested this on power2. > So, yes, I would say this should go to CLOSE_WAIT at this point, > u

Re: [HACKERS] Detrimental performance impact of ringbuffers on performance

2016-04-13 Thread Amit Kapila
On Wed, Apr 13, 2016 at 12:08 AM, Andres Freund wrote: > > On 2016-04-12 14:29:10 -0400, Robert Haas wrote: > > On Wed, Apr 6, 2016 at 6:57 AM, Andres Freund wrote: > > > While benchmarking on hydra > > > (c.f. http://archives.postgresql.org/message-id/20160406104352.5bn3ehkcsceja65c%40alap3.anar

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-04-13 Thread Etsuro Fujita
On 2016/04/14 4:57, Robert Haas wrote: On Wed, Apr 13, 2016 at 2:36 PM, Robert Haas wrote: On Wed, Apr 13, 2016 at 2:11 PM, Robert Haas wrote: So, clearly that's not good. It should at least be consistent. But more than that, the fact that postgres_fdw sets the xmax to 0x is also pr

[HACKERS] EXPLAIN VERBOSE with parallel Aggregate

2016-04-13 Thread David Rowley
There's 2 problems: 1) I recently noticed that EXPLAIN VERBOSE is a bit bogus when it comes to parallel aggregates with FILTER (WHERE ...) clauses. We get; Output: pg_catalog.sum((sum(num) FILTER (WHERE (num > 0 FILTER (WHERE (num > 0)) Which is simply a lie, we only filter on the partial

Re: [HACKERS] Choosing parallel_degree

2016-04-13 Thread Amit Kapila
On Wed, Apr 13, 2016 at 10:47 PM, Robert Haas wrote: > > > I would be inclined to view this as a reasonable 9.6 cleanup of > parallel query, but other people may wish to construe things more > strictly than I would. > +1. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-04-13 Thread Amit Kapila
On Wed, Apr 13, 2016 at 1:44 PM, Kyotaro HORIGUCHI < horiguchi.kyot...@lab.ntt.co.jp> wrote: > > At Wed, 13 Apr 2016 04:43:35 +0900, Fujii Masao wrote in > > >>> Thank you for reviewing. > > >>> > > >>> SyncRepUpdateConfig() seems to be no longer necessary. > > >> > > >> Really? I was thinking th

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-13 Thread Andres Freund
On 2016-04-14 07:59:07 +0530, Amit Kapila wrote: > What you want to see by prewarming? Prewarming appears to greatly reduce the per-run variance on that machine, making it a lot easier to get meaningful results. Thus it'd make it easier to compare pre/post padding numbers. > Will it have safe ef

Re: [HACKERS] [COMMITTERS] pgsql: Add regression tests for multiple synchronous standbys.

2016-04-13 Thread Fujii Masao
On Wed, Apr 13, 2016 at 4:54 PM, Michael Paquier wrote: > On Fri, Apr 8, 2016 at 4:49 PM, Fujii Masao wrote: >> Add regression tests for multiple synchronous standbys. >> >> Authors: Suraj Kharage, Michael Paquier, Masahiko Sawada, refactored by me >> Reviewed-By: Kyotaro Horiguchi > > Well, we a

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-04-13 Thread Amit Kapila
On Tue, Apr 12, 2016 at 9:32 PM, Andres Freund wrote: > > On 2016-04-12 19:42:11 +0530, Amit Kapila wrote: > > Andres suggested me on IM to take performance data on x86 m/c > > by padding PGXACT and the data for the same is as below: > > > > median of 3, 5-min runs > > Thanks for running these. >

Re: [HACKERS] Reserved roles and user mapping

2016-04-13 Thread Amit Langote
Hi Stephen, On 2016/04/14 9:24, Stephen Frost wrote: > Amit, > > * Amit Langote (langote_amit...@lab.ntt.co.jp) wrote: >> Currently in CreateUserMapping(): >> >> /* Additional check to protect reserved role names */ >> check_rolespec_name(stmt->user, >> "Cannot sp

Re: [HACKERS] documentation build bleat

2016-04-13 Thread Fujii Masao
On Thu, Apr 14, 2016 at 9:08 AM, Tom Lane wrote: > Some fairly recent commit is causing the HTML docs build to moan about > > collateindex.pl: duplicated index entry found: PG_LOGICAL_EMIT_MESSAGE > 1 entries ignored... This is my fault. Sorry... - >pg_logical_emit_message +

Re: [HACKERS] Suspicious behaviour on applying XLOG_HEAP2_VISIBLE.

2016-04-13 Thread Andres Freund
On 2016-03-31 01:02:06 -0400, Noah Misch wrote: > On Thu, Mar 10, 2016 at 01:04:11AM +0900, Masahiko Sawada wrote: > > As a result of looked into code around the recvoery, ISTM that the > > cause is related to relation cache clear. > > In heap_xlog_visible, if the standby server receives WAL record

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2016-04-13 Thread Etsuro Fujita
On 2016/04/13 21:50, Michael Paquier wrote: On Wed, Apr 13, 2016 at 9:46 PM, Robert Haas wrote: On Tue, Apr 12, 2016 at 10:24 PM, Etsuro Fujita wrote: How about we encapsulate the while (PQisBusy(...)) loop into a new function pgfdw_get_result(), which can be called after first calling PQsend

Re: [HACKERS] SET ROLE and reserved roles

2016-04-13 Thread Stephen Frost
Amit, * Amit Langote (langote_amit...@lab.ntt.co.jp) wrote: > On 2016/04/14 2:10, Stephen Frost wrote: > >> Amit Langote > writes: > >>> I observe this: > >> > >>> postgres=# SET ROLE TO NONE; > >>> SET > >>> postgres=# SET ROLE TO nonexistent; > >>> ERROR: role "nonexistent" does not exist > >>>

Re: [HACKERS] SET ROLE and reserved roles

2016-04-13 Thread Amit Langote
Hi Stephen, On 2016/04/14 2:10, Stephen Frost wrote: >> Amit Langote > writes: >>> I observe this: >> >>> postgres=# SET ROLE TO NONE; >>> SET >>> postgres=# SET ROLE TO nonexistent; >>> ERROR: role "nonexistent" does not exist >>> postgres=# SET ROLE TO pg_signal_backend; >>> ERROR: invalid va

[HACKERS] Safe to apply HeapTupleHeaderGetDatum to a tuple from syscache?

2016-04-13 Thread Chapman Flack
I'm still learning, and looking at the HeapTupleHeaderGetDatum comment that says This must *not* get applied to an on-disk tuple; the tuple should be freshly made by heap_form_tuple or some wrapper routine for it (such as BuildTupleFromCStrings). Be sure also that the tupledesc used to bu

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-04-13 Thread Robert Haas
On Wed, Apr 13, 2016 at 5:33 PM, Tom Lane wrote: > Robert Haas writes: >> 2. When a join is pushed down, deparse system columns using something >> like "CASE WHEN r1.* IS NOT NULL THEN 0 END", except for the table OID >> column, which gets deparsed with the table OID in place of 0. This >> deliv

Re: [HACKERS] Pglogical questions and problems

2016-04-13 Thread Andres Freund
Hi, On 2016-04-13 09:38:39 +0100, Simon Riggs wrote: > If we want this in 9.7 I desperately want logical replication for 9.7. And I'm planning to put in a good chunk of work to make that happen in some way. > we'll need lots of people's support, design commentary and assistance > with bug fixin

Re: [HACKERS] Pglogical questions and problems

2016-04-13 Thread Craig Ringer
On 14 April 2016 at 00:48, Robert Haas wrote: > > I think this would be a good topic to discuss at PGCon. > > I'm not going to be able to be at PGCon, and I don't think Petr is either. That's unfortunate in terms of planning for this. -- Craig Ringer http://www.2ndQuadrant.c

Re: [HACKERS] SET ROLE and reserved roles

2016-04-13 Thread Stephen Frost
* David G. Johnston (david.g.johns...@gmail.com) wrote: > From what I've read here I'm thinking Stephen has the right idea. Thanks. Additionally, your comments make me realize an existing issue, which is superuser-only but I'll address shortly anyway (we have far too many users running around as

Re: [HACKERS] Proposal: Generic WAL logical messages

2016-04-13 Thread Andres Freund
On 2016-04-07 19:53:56 +0200, Petr Jelinek wrote: > On 07/04/16 12:26, Andres Freund wrote: > >Hi, > > > >On 2016-04-06 20:03:20 +0200, Petr Jelinek wrote: > >>Attached patch adds filtering of both database and origin. Added tests with > >>slightly less hardcoding than what you proposed above. > >

Re: [HACKERS] Why doesn't src/backend/port/win32/socket.c implement bind()?

2016-04-13 Thread Michael Paquier
On Thu, Apr 14, 2016 at 8:46 AM, Tom Lane wrote: > Alvaro Herrera writes: >> Michael Paquier wrote: >>> Well, yes. That's true, we could do without. Even if this could give >>> an indication about a node running, as long as a port has been >>> associated to a node once, we just need to be sure th

Re: [HACKERS] Reserved roles and user mapping

2016-04-13 Thread Stephen Frost
Amit, * Amit Langote (langote_amit...@lab.ntt.co.jp) wrote: > Currently in CreateUserMapping(): > > /* Additional check to protect reserved role names */ > check_rolespec_name(stmt->user, > "Cannot specify reserved role as mapping user."); > > User mapping termino

Re: [HACKERS] [COMMITTERS] pgsql: Allow Pin/UnpinBuffer to operate in a lockfree manner.

2016-04-13 Thread Tom Lane
Andres Freund writes: > On 2016-04-13 11:20:19 -0700, Andres Freund wrote: >> Heh, I was wondering the same aftering sending the last email. Will do >> that then. > Pushed. Let's see what pademelon says... I lit off a run, but it'll be a few hours till we have results...

[HACKERS] documentation build bleat

2016-04-13 Thread Tom Lane
Some fairly recent commit is causing the HTML docs build to moan about collateindex.pl: duplicated index entry found: PG_LOGICAL_EMIT_MESSAGE 1 entries ignored... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes t

Re: [HACKERS] \crosstabview fixes

2016-04-13 Thread Tom Lane
I wrote: > "Daniel Verite" writes: >> To avoid the confusion between "2:4" and "2":"4" or 2:4, >> and the ambiguity with a possibly existing "2:4" column, >> maybe we should abandon this syntax and require the optional >> scolH to be on its own at the end of the command. > That would be OK with m

Re: [HACKERS] [COMMITTERS] pgsql: Allow Pin/UnpinBuffer to operate in a lockfree manner.

2016-04-13 Thread Andres Freund
On 2016-04-13 11:20:19 -0700, Andres Freund wrote: > On 2016-04-13 14:16:15 -0400, Tom Lane wrote: > > Good point, it would be absolutely duplicative. What I'd suggest, > > actually, is that we convert this to the same info as what elog > > provides (file+line+function name). > > Heh, I was wonde

Re: [HACKERS] SET ROLE and reserved roles

2016-04-13 Thread David G. Johnston
On Wed, Apr 13, 2016 at 3:53 PM, Stephen Frost wrote: > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > Stephen Frost writes: > > > On Wednesday, April 13, 2016, Tom Lane wrote: > > >> If you want to prevent that, I think it needs to be done somewhere > else > > >> than here. What about "ALTER OWNE

Re: [HACKERS] Why doesn't src/backend/port/win32/socket.c implement bind()?

2016-04-13 Thread Tom Lane
Alvaro Herrera writes: > Michael Paquier wrote: >> Well, yes. That's true, we could do without. Even if this could give >> an indication about a node running, as long as a port has been >> associated to a node once, we just need to be sure that a new port is >> not allocated. On Windows, I am not

Re: [HACKERS] sign function with INTERVAL?

2016-04-13 Thread David G. Johnston
On Wed, Apr 13, 2016 at 3:48 PM, Daniel Lenski wrote: > On Wed, Apr 13, 2016 at 12:35 PM, Tom Lane wrote: > > Jim Nasby writes: > >> Actually, after looking at the code for interval_lt, all that needs to > >> happen to add this support is to expose interval_cmp_internal() as a > >> strict funct

Re: [HACKERS] Why doesn't src/backend/port/win32/socket.c implement bind()?

2016-04-13 Thread Alvaro Herrera
Michael Paquier wrote: > On Wed, Apr 13, 2016 at 10:33 PM, Tom Lane wrote: > > Michael Paquier writes: > >> On Wed, Apr 13, 2016 at 9:06 AM, Tom Lane wrote: > >>> If there's other stuff using high ports on a particular buildfarm machine, > >>> you'd expect occasional random test failures due to

Re: [HACKERS] fd.c: flush data problems on osx

2016-04-13 Thread Tom Lane
Andres Freund writes: > On 2016-04-13 18:09:18 -0400, Tom Lane wrote: >> BTW, I just noticed another issue here, which is that FileWriteback >> and the corresponding smgr routines are declared with bogusly narrow >> "amount" arguments --- eg, it's silly that FileWriteback only takes >> an int amou

Re: [HACKERS] Why doesn't src/backend/port/win32/socket.c implement bind()?

2016-04-13 Thread Michael Paquier
On Wed, Apr 13, 2016 at 10:33 PM, Tom Lane wrote: > Michael Paquier writes: >> On Wed, Apr 13, 2016 at 9:06 AM, Tom Lane wrote: >>> If there's other stuff using high ports on a particular buildfarm machine, >>> you'd expect occasional random test failures due to this. The observed >>> fact that

Re: [HACKERS] SET ROLE and reserved roles

2016-04-13 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > On Wednesday, April 13, 2016, Tom Lane wrote: > >> If you want to prevent that, I think it needs to be done somewhere else > >> than here. What about "ALTER OWNER TO pg_signal_backend", for instance? > > > Checks are included in

Re: [HACKERS] sign function with INTERVAL?

2016-04-13 Thread Daniel Lenski
On Wed, Apr 13, 2016 at 12:35 PM, Tom Lane wrote: > Jim Nasby writes: >> Actually, after looking at the code for interval_lt, all that needs to >> happen to add this support is to expose interval_cmp_internal() as a >> strict function. It already does exactly what you want. > > interval_cmp() is

Re: [HACKERS] fd.c: flush data problems on osx

2016-04-13 Thread Andres Freund
On 2016-04-13 18:09:18 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2016-04-13 17:44:41 -0400, Tom Lane wrote: > >> fd.c tracks seek position for open files. I'm not sure that that > >> function can get called with amount == 0, but if it did, the caller > >> would certainly not be expect

Re: [HACKERS] Missing PG_INT32_MIN in numutils.c

2016-04-13 Thread Michael Paquier
On Wed, Apr 13, 2016 at 11:13 PM, Robert Haas wrote: > On Wed, Apr 13, 2016 at 10:11 AM, Tom Lane wrote: >> Robert Haas writes: >>> On Wed, Apr 13, 2016 at 9:38 AM, Tom Lane wrote: I am not very convinced that this is an improvement, because you took what had been two hard-wired const

Re: [HACKERS] fd.c: flush data problems on osx

2016-04-13 Thread Tom Lane
Andres Freund writes: > On 2016-04-13 17:44:41 -0400, Tom Lane wrote: >> fd.c tracks seek position for open files. I'm not sure that that >> function can get called with amount == 0, but if it did, the caller >> would certainly not be expecting the file position to change. > Ok, fair enough. (A

Re: [HACKERS] fd.c: flush data problems on osx

2016-04-13 Thread Andres Freund
On 2016-04-13 17:44:41 -0400, Tom Lane wrote: > Andres Freund writes: > > I'm not entirely sure what > > > + /* > > +* Caution: do not call pg_flush_data with amount = 0, it could trash > > the > > +* file's seek position. > > +*/ > > + if (amount <= 0) > > + return; > > +

Re: [HACKERS] fd.c: flush data problems on osx

2016-04-13 Thread Tom Lane
Andres Freund writes: > I'm not entirely sure what > + /* > +* Caution: do not call pg_flush_data with amount = 0, it could trash the > +* file's seek position. > +*/ > + if (amount <= 0) > + return; > + > is about? fd.c tracks seek position for open files. I'm not sure t

Re: [HACKERS] fd.c: flush data problems on osx

2016-04-13 Thread Andres Freund
On 2016-04-13 17:21:01 -0400, Tom Lane wrote: > I concluded that sharing the code would be more trouble than it's worth, > because initdb.c's version of this is in fact not broken: it was never > taught about mmap, and it doesn't need to be IMO, because it's not that > performance-critical. Agreed

Re: [HACKERS] Incomplete startup packet errors

2016-04-13 Thread Tatsuo Ishii
>> Is it possible a user want the log because he/she wants to notice that >> the system is being attacked? > > Yeah, but it doesn't seem very likely, because: > > 1. If the system is on the Internet, it's definitely being attacked, and > > 2. The attacks that connect to a port and then disconnec

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-04-13 Thread Tom Lane
Robert Haas writes: > 2. When a join is pushed down, deparse system columns using something > like "CASE WHEN r1.* IS NOT NULL THEN 0 END", except for the table OID > column, which gets deparsed with the table OID in place of 0. This > delivers the correct behavior in the presence of outer joins.

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Andres Freund
On 2016-04-13 16:05:25 -0500, Kevin Grittner wrote: > OK, thanks. I can't think of anything else to ask for at this > point. If you feel that you have enough to press for some > particular course of action, go for it. I think we, at the very least, need a clear proposal how to resolve the scalab

Re: [HACKERS] fd.c: flush data problems on osx

2016-04-13 Thread Tom Lane
I wrote: > Michael Paquier writes: >> A similar change seems to be needed in initdb.c's pre_sync_fname. > Hmm, do we need to move this logic into src/common? I concluded that sharing the code would be more trouble than it's worth, because initdb.c's version of this is in fact not broken: it was

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Kevin Grittner
On Wed, Apr 13, 2016 at 3:47 PM, Andres Freund wrote: > On 2016-04-13 15:21:31 -0500, Kevin Grittner wrote: >> What is the kernel on which these tests were run? > > 3.16. I can upgrade to 4.4 if necessary. No, I'm not aware of any problems from 3.8 on. > But I still believe very strongly that t

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Andres Freund
On 2016-04-13 15:21:31 -0500, Kevin Grittner wrote: > On Wed, Apr 13, 2016 at 3:01 PM, Andres Freund wrote: > > > If you want me to rn some other tests I can, but ISTM we have the > > data we need? > > Thanks for the additional detail on how this was run. I think I > still need a little more co

Re: [HACKERS] [patch] \crosstabview documentation

2016-04-13 Thread Daniel Verite
David G. Johnston wrote: > > ​So I guess: > > "​ > crosstabview with only 2 output columns > ​ "​ > > ​https://wiki.postgresql.org/wiki/Crosstabview > ​(last section on that page) > > ​never got implemented It was implemented but eventually removed. I will update shortly this wiki

Re: [HACKERS] \crosstabview fixes

2016-04-13 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> (I also took the trouble to make the error messages conform >> to project style.) > Not sure about this part. Many psql error messages are full sentences (start > with uppercase, end in period); others start with the \ command being > complained about.

Re: [HACKERS] \crosstabview fixes

2016-04-13 Thread Tom Lane
"Daniel Verite" writes: > To avoid the confusion between "2:4" and "2":"4" or 2:4, > and the ambiguity with a possibly existing "2:4" column, > maybe we should abandon this syntax and require the optional > scolH to be on its own at the end of the command. That would be OK with me; it's certainly

Re: [HACKERS] [patch] \crosstabview documentation

2016-04-13 Thread Alvaro Herrera
David G. Johnston wrote: > "​ > crosstabview with only 2 output columns > ​ "​ > > ​https://wiki.postgresql.org/wiki/Crosstabview > ​(last section on that page) > > ​never got implemented It was implemented in Daniel's patch. I removed it before commit and failed to notice the reference i

Re: [HACKERS] [patch] \crosstabview documentation

2016-04-13 Thread David G. Johnston
On Wed, Apr 13, 2016 at 12:29 PM, Tom Lane wrote: > Christoph Berg writes: > > Another thing about \crosstabview: > > > # select 1,2 \crosstabview > > The query must return at least two columns to be shown in crosstab > > > s/two/three/, I guess. > > Yeah, I noticed that. See > http://www.postg

Re: [HACKERS] \crosstabview fixes

2016-04-13 Thread Alvaro Herrera
Tom Lane wrote: > I noticed that the \crosstabview documentation asserts that column name > arguments are handled per standard SQL semantics. In point of fact, > though, the patch expends a couple hundred lines to implement what is > NOT standard SQL semantics: matching unquoted names case-insensi

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Kevin Grittner
On Wed, Apr 13, 2016 at 3:01 PM, Andres Freund wrote: > If you want me to rn some other tests I can, but ISTM we have the > data we need? Thanks for the additional detail on how this was run. I think I still need a little more context, though: What is the kernel on which these tests were run?

Re: [HACKERS] \crosstabview fixes

2016-04-13 Thread Daniel Verite
Tom Lane wrote: > I noticed that the \crosstabview documentation asserts that column name > arguments are handled per standard SQL semantics. In point of fact, > though, the patch expends a couple hundred lines to implement what is > NOT standard SQL semantics: matching unquoted names cas

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Andres Freund
Hi Kevin, On 2016-04-13 12:21:10 -0700, Andres Freund wrote: > 0: > progress: 100.0 s, 593351.0 tps, lat 0.215 ms stddev 0.118 > progress: 200.0 s, 594035.9 tps, lat 0.215 ms stddev 0.118 > progress: 300.0 s, 594013.3 tps, lat 0.215 ms stddev 0.117 > > -1: > progress: 100.0 s, 600835.3 tps, lat 0

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-04-13 Thread Robert Haas
On Wed, Apr 13, 2016 at 2:36 PM, Robert Haas wrote: > On Wed, Apr 13, 2016 at 2:11 PM, Robert Haas wrote: >> So, clearly that's not good. It should at least be consistent. But >> more than that, the fact that postgres_fdw sets the xmax to 0x >> is also pretty wacky. We might use such a

Re: [HACKERS] sign function with INTERVAL?

2016-04-13 Thread Tom Lane
Jim Nasby writes: > Actually, after looking at the code for interval_lt, all that needs to > happen to add this support is to expose interval_cmp_internal() as a > strict function. It already does exactly what you want. interval_cmp() is already SQL-accessible. regards,

Re: [HACKERS] [patch] \crosstabview documentation

2016-04-13 Thread Tom Lane
Christoph Berg writes: > Another thing about \crosstabview: > # select 1,2 \crosstabview > The query must return at least two columns to be shown in crosstab > s/two/three/, I guess. Yeah, I noticed that. See http://www.postgresql.org/message-id/10276.1460569...@sss.pgh.pa.us

Re: [HACKERS] [GENERAL] pg_upgrade error regarding hstore operator

2016-04-13 Thread Tom Lane
"Feld, Michael (IMS)" writes: > In addition, I noticed the following differences in the pg_depend catalog, > there are 4 entries in 9.1 that are missing in 9.5 which I have separated > with ***: > select classid::regclass, objid, objsubid, refclassid::regclass, refobjid, > refobjsubid, deptype,

Re: [HACKERS] SET ROLE and reserved roles

2016-04-13 Thread Robert Haas
On Wed, Apr 13, 2016 at 2:10 PM, Tom Lane wrote: > Stephen Frost writes: >> On Wednesday, April 13, 2016, Tom Lane wrote: >>> If you want to prevent that, I think it needs to be done somewhere else >>> than here. What about "ALTER OWNER TO pg_signal_backend", for instance? > >> Checks are inclu

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Andres Freund
On 2016-04-13 14:08:49 -0500, Kevin Grittner wrote: > On Wed, Apr 13, 2016 at 1:56 PM, Andres Freund wrote: > > > I'll run with -1 once the current (longer) run has finished. > > Just for the record, were any of the other results purporting to be > with the feature "off" also actually running wi

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Robert Haas
On Wed, Apr 13, 2016 at 3:08 PM, Kevin Grittner wrote: > On Wed, Apr 13, 2016 at 1:56 PM, Andres Freund wrote: > >> I'll run with -1 once the current (longer) run has finished. > > Just for the record, were any of the other results purporting to be > with the feature "off" also actually running w

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Andres Freund
On 2016-04-13 13:52:15 -0500, Kevin Grittner wrote: > On Wed, Apr 13, 2016 at 12:25 PM, Robert Haas wrote: > > > [test results with old_snapshot_threshold = 0 and 10] > > From the docs: > > | A value of -1 disables this feature, and is the default. > > > Yuck. Aside from the fact that performance

Re: [HACKERS] [patch] \crosstabview documentation

2016-04-13 Thread Christoph Berg
Re: Tom Lane 2016-04-13 <1854.1460562...@sss.pgh.pa.us> > Hm, we do not have entries attached to any other psql > meta-commands. Maybe they all should have one, or maybe not, but > I'm unconvinced about adding one for just this command. What I did > instead was to make a link target (which there

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Jim Nasby
On 4/12/16 12:30 PM, Tom Lane wrote: It'd be good if you document the problems you found somewhere, before you forget them, just in case somebody does want to try to lift the restriction. I agree that scattered code comments wouldn't be the way. Just a quick email to -hackers to get the info int

Re: [HACKERS] sign function with INTERVAL?

2016-04-13 Thread Jim Nasby
On 4/13/16 1:36 PM, Daniel Lenski wrote: Hi all, Is there a good reason why the SIGN() function does not work with the INTERVAL type? (It is only defined for numeric types.) (http://www.postgresql.org/docs/9.5/static/functions-math.html) The only thing that comes to mind is you can get some str

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Kevin Grittner
On Wed, Apr 13, 2016 at 1:56 PM, Andres Freund wrote: > I'll run with -1 once the current (longer) run has finished. Just for the record, were any of the other results purporting to be with the feature "off" also actually running with the feature set for its fastest possible timeout? -- Kevin G

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Andres Freund
On 2016-04-13 13:52:15 -0500, Kevin Grittner wrote: > On Wed, Apr 13, 2016 at 12:25 PM, Robert Haas wrote: > > > [test results with old_snapshot_threshold = 0 and 10] > > From the docs: > > | A value of -1 disables this feature, and is the default. Hm, ok, let me run that as well then. The rea

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Kevin Grittner
On Wed, Apr 13, 2016 at 12:25 PM, Robert Haas wrote: > [test results with old_snapshot_threshold = 0 and 10] >From the docs: | A value of -1 disables this feature, and is the default. > Yuck. Aside from the fact that performance tanks when the feature is > turned on, it seems that there is a

[HACKERS] sign function with INTERVAL?

2016-04-13 Thread Daniel Lenski
Hi all, Is there a good reason why the SIGN() function does not work with the INTERVAL type? (It is only defined for numeric types.) (http://www.postgresql.org/docs/9.5/static/functions-math.html) select sign(-3); -- okay select sign(interval '4 years'); -- ERROR: function sign(interval) does not

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-04-13 Thread Robert Haas
On Wed, Apr 13, 2016 at 2:11 PM, Robert Haas wrote: > So, clearly that's not good. It should at least be consistent. But > more than that, the fact that postgres_fdw sets the xmax to 0x > is also pretty wacky. We might use such a value as a sentinel for > some data type, but for transac

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-04-13 Thread Tom Lane
Robert Haas writes: > On Wed, Apr 13, 2016 at 1:36 PM, Robert Haas wrote: >> I tend to favor zeroes rather than NULLs, because that's what we >> typically use to represent an invalid value of those types, and I'm >> not aware of any current case where those values are NULL. > In fact, see heap_a

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-04-13 Thread Robert Haas
On Wed, Apr 13, 2016 at 1:36 PM, Robert Haas wrote: > I tend to favor zeroes rather than NULLs, because that's what we > typically use to represent an invalid value of those types, and I'm > not aware of any current case where those values are NULL. In fact, see heap_attisnull. -- Robert Haas E

Re: [HACKERS] Choosing parallel_degree

2016-04-13 Thread Julien Rouhaud
On 13/04/2016 19:17, Robert Haas wrote: > On Tue, Apr 12, 2016 at 6:31 PM, Julien Rouhaud > wrote: >> On 11/04/2016 22:53, Julien Rouhaud wrote: >>> On 11/04/2016 17:44, Robert Haas wrote: We should probably add the number of workers actually obtained to the EXPLAIN ANALYZE output. That

Re: [HACKERS] [COMMITTERS] pgsql: Allow Pin/UnpinBuffer to operate in a lockfree manner.

2016-04-13 Thread Andres Freund
On 2016-04-13 14:16:15 -0400, Tom Lane wrote: > Good point, it would be absolutely duplicative. What I'd suggest, > actually, is that we convert this to the same info as what elog > provides (file+line+function name). Heh, I was wondering the same aftering sending the last email. Will do that th

Re: [HACKERS] [COMMITTERS] pgsql: Allow Pin/UnpinBuffer to operate in a lockfree manner.

2016-04-13 Thread Tom Lane
Andres Freund writes: > On 2016-04-12 11:52:01 -0400, Tom Lane wrote: >> It strikes me that that means you could stick with this initialization >> method if you made the macro argument be a literal constant string name, >> like "buffer spinlock", and printed that rather than the address in >> s_lo

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-04-13 Thread Robert Haas
On Wed, Apr 13, 2016 at 1:36 PM, Robert Haas wrote: > I tend to favor zeroes rather than NULLs, because that's what we > typically use to represent an invalid value of those types, and I'm > not aware of any current case where those values are NULL. Actually, come to think of it, what we *really*

Re: [HACKERS] SET ROLE and reserved roles

2016-04-13 Thread Tom Lane
Stephen Frost writes: > On Wednesday, April 13, 2016, Tom Lane wrote: >> If you want to prevent that, I think it needs to be done somewhere else >> than here. What about "ALTER OWNER TO pg_signal_backend", for instance? > Checks are included in that code path to disallow it. If there are such

Re: [HACKERS] [COMMITTERS] pgsql: Allow Pin/UnpinBuffer to operate in a lockfree manner.

2016-04-13 Thread Andres Freund
On 2016-04-12 11:52:01 -0400, Tom Lane wrote: > I wrote: > > It looks like that compiler adheres to the C89 restriction that an > > initializer for an array or struct must contain only link-time-constant > > expressions, even if the target object is of dynamic scope. > > The macro works with a link

Re: [HACKERS] SET ROLE and reserved roles

2016-04-13 Thread Stephen Frost
Tom, On Wednesday, April 13, 2016, Tom Lane wrote: > Stephen Frost > writes: > > On Wednesday, April 13, 2016, Robert Haas > wrote: > >> Well ... yeah. But that doesn't mean it should be impossible to SET > >> to that role itself. I'm a little worried that could create strange > >> corner cas

Re: [HACKERS] SET ROLE and reserved roles

2016-04-13 Thread Tom Lane
Stephen Frost writes: > On Wednesday, April 13, 2016, Robert Haas wrote: >> Well ... yeah. But that doesn't mean it should be impossible to SET >> to that role itself. I'm a little worried that could create strange >> corner cases. > Being able to create objects owned by a default role was one

[HACKERS] \crosstabview fixes

2016-04-13 Thread Tom Lane
I noticed that the \crosstabview documentation asserts that column name arguments are handled per standard SQL semantics. In point of fact, though, the patch expends a couple hundred lines to implement what is NOT standard SQL semantics: matching unquoted names case-insensitively is anything but t

Re: [HACKERS] Odd system-column handling in postgres_fdw join pushdown patch

2016-04-13 Thread Robert Haas
On Tue, Apr 5, 2016 at 4:54 AM, Ashutosh Bapat wrote: > With this patch, all instances of tableoid, cmin, cmax etc. will get a > non-NULL value irrespective of whether they appear on nullable side of the > join or not. > > e.g. select t1.c1, t1.tableoid, t2.c1, t2.tableoid from ft4 t1 left join ft

Re: [HACKERS] Re: [COMMITTERS] pgsql: Avoid extra locks in GetSnapshotData if old_snapshot_threshold

2016-04-13 Thread Andres Freund
On 2016-04-13 13:25:14 -0400, Robert Haas wrote: > > With -c old_snapshot_threshold=0: > > > > latency average = 0.218 ms > > latency stddev = 0.154 ms > > tps = 584666.289753 (including connections establishing) > > tps = 584867.785569 (excluding connections establishing) > > > > > > With -c old_s

Re: [HACKERS] Parallel Aggregate costs don't consider combine/serial/deserial funcs

2016-04-13 Thread Robert Haas
On Tue, Apr 12, 2016 at 5:38 PM, David Rowley wrote: >>> One small point which I was a little unsure of in the attached is, >>> should the "if (aggref->aggdirectargs)" part of >>> count_agg_clauses_walker() be within the "if >>> (!context->combineStates)". I simply couldn't decide. We currently >>

Re: [HACKERS] SET ROLE and reserved roles

2016-04-13 Thread Robert Haas
On Wed, Apr 13, 2016 at 1:24 PM, Stephen Frost wrote: > On Wednesday, April 13, 2016, Robert Haas wrote: >> >> On Wed, Apr 13, 2016 at 1:10 PM, Stephen Frost wrote: >> >> What I'd like to know is why it rejects that at all. What's the point >> >> of having roles you can't SET to? >> > >> > To u

  1   2   >