[HACKERS] Dead code in Create/RenameRole() after RoleSpec changes related to CURRENT/SESSION_USER

2015-06-09 Thread Jeevan Chalke
Hi, I found some dead code in CREATE/RENAME ROLE code path. Attached patch to remove those. We have introduced RoleSpec and handled public and none role names in grammar itself. We do have these handling in CreateRole() and RenameRole() which is NO more valid now. Here is the related commit:

Re: [HACKERS] CREATE POLICY and RETURNING

2015-06-09 Thread Dean Rasheed
On 8 June 2015 at 16:53, Stephen Frost sfr...@snowman.net wrote: I definitely don't like the idea of returning a portion of the data in a RETURNING clause. Returning an error if the RETURNING clause ends up not passing the SELECT policy is an interesting idea, but I do have doubts about how

Re: [HACKERS] Cancel race condition

2015-06-09 Thread Shay Rojansky
Ah, OK - I wasn't aware that cancellation was actually delivered as a regular POSIX signal... You're right about the lack of guarantees then. In that case, I'm guessing not much could be do to guarantee sane cancellation behavior... I do understand the best effort idea around cancellations.

[HACKERS] The Future of Aggregation

2015-06-09 Thread David Rowley
It appears to me that there's quite a few new features and optimisations on the not too distant horizon which will require adding yet more fields into pg_aggregate. These are things along the lines of: 1. Parallel Aggregation (computes each aggregate state in parallel worker processes and then

Re: [CORE] [HACKERS] back-branch multixact fixes 9.5 alpha/beta: schedule

2015-06-09 Thread Robert Haas
On Tue, Jun 9, 2015 at 3:04 AM, David Gould da...@sonic.net wrote: On Mon, 8 Jun 2015 13:53:42 -0300 Alvaro Herrera alvhe...@2ndquadrant.com wrote: * people with the wrong oldestMulti setting in pg_control (which would be due to a buggy pg_upgrade being used long ago) will be unable to start

Re: [HACKERS] bugfix: incomplete implementation of errhidecontext

2015-06-09 Thread Jeevan Chalke
On Mon, Jun 8, 2015 at 8:19 PM, Andres Freund and...@anarazel.de wrote: On 2015-06-08 14:44:53 +, Jeevan Chalke wrote: The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec

Re: [HACKERS] Information of pg_stat_ssl visible to all users

2015-06-09 Thread Magnus Hagander
On Jun 9, 2015 6:00 AM, Michael Paquier michael.paqu...@gmail.com wrote: Hi all, I should have noticed that before, but it happens that pg_stat_ssl leaks information about the SSL status of all the users connected to a server. Let's imagine for example: 1) Session 1 connected through SSL

Re: [HACKERS] Re: [COMMITTERS] pgsql: Map basebackup tablespaces using a tablespace_map file

2015-06-09 Thread Amit Kapila
On Tue, Jun 9, 2015 at 10:56 AM, Fujii Masao masao.fu...@gmail.com wrote: On Tue, Jun 9, 2015 at 1:04 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Tue, Jun 9, 2015 at 9:09 AM, Fujii Masao masao.fu...@gmail.com wrote: On Tue, May 12, 2015 at 10:42 PM, Andrew Dunstan and...@dunslane.net

Re: [CORE] [HACKERS] back-branch multixact fixes 9.5 alpha/beta: schedule

2015-06-09 Thread David Gould
On Mon, 8 Jun 2015 13:53:42 -0300 Alvaro Herrera alvhe...@2ndquadrant.com wrote: * people with the wrong oldestMulti setting in pg_control (which would be due to a buggy pg_upgrade being used long ago) will be unable to start if they upgrade to 9.3.7 or 9.3.8. A solution for them would be to

Re: [HACKERS] Re: [COMMITTERS] pgsql: Map basebackup tablespaces using a tablespace_map file

2015-06-09 Thread Andrew Dunstan
On 06/08/2015 11:19 PM, Amit Kapila wrote: On Tue, Jun 9, 2015 at 12:27 AM, Andrew Dunstan and...@dunslane.net mailto:and...@dunslane.net wrote: On 06/08/2015 11:16 AM, Amit Kapila wrote: I have to retry that operation, but for me unlink hasn't deleted the file on

[HACKERS] could not adopt C locale failure at startup on Windows

2015-06-09 Thread Tom Lane
We've seen several reports of $SUBJECT lately. I have no idea what's going on, but it occurred to me that it could be informative to tweak init_locale() so that it reports which category failed to be set. Any objections to squeezing that into today's releases? regards,

Re: [HACKERS] could not adopt C locale failure at startup on Windows

2015-06-09 Thread Andres Freund
On 2015-06-09 11:20:06 -0400, Tom Lane wrote: We've seen several reports of $SUBJECT lately. I have no idea what's going on, but it occurred to me that it could be informative to tweak init_locale() so that it reports which category failed to be set. Any objections to squeezing that into

Re: [HACKERS] Aggregate Supporting Functions

2015-06-09 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: Kevin Grittner kgri...@ymail.com writes: David Rowley david.row...@2ndquadrant.com wrote: [ avoid duplicate calculations for related aggregates ] From the information you have proposed storing, with cost factors associated with the functions, it seems

Re: [HACKERS] The Future of Aggregation

2015-06-09 Thread Andres Freund
On 2015-06-09 17:19:33 +0200, Tomas Vondra wrote: ... and yet another use case for 'aggregate state combine' that I just remembered about is grouping sets. What GROUPING SET (ROLLUP, ...) do currently is repeatedly sorting the input, once for each grouping. Actually, that's not really what

Re: [HACKERS] The Future of Aggregation

2015-06-09 Thread Tomas Vondra
Hi, On 06/09/15 12:58, David Rowley wrote: These are things along the lines of: 1. Parallel Aggregation (computes each aggregate state in parallel worker processes and then merges these states in serial mode) 2. Aggregate push-down / Aggregate before join (requires passing partially computed

Re: [HACKERS] The Future of Aggregation

2015-06-09 Thread Kevin Grittner
Alvaro Herrera alvhe...@2ndquadrant.com wrote: Kevin Grittner wrote: David Rowley david.row...@2ndquadrant.com wrote: 5. Dependant Aggregates Item 5 makes items 1-4 a bit more complex as with this item there's opportunity for very good performance improvements by allowing aggregates like

Re: [HACKERS] The Future of Aggregation

2015-06-09 Thread Alvaro Herrera
Kevin Grittner wrote: David Rowley david.row...@2ndquadrant.com wrote: 5. Dependant Aggregates Item 5 makes items 1-4 a bit more complex as with this item there's opportunity for very good performance improvements by allowing aggregates like AVG(x) also perform all the required work

Re: [HACKERS] The Future of Aggregation

2015-06-09 Thread Tomas Vondra
On 06/09/15 16:10, Tomas Vondra wrote: Hi, On 06/09/15 12:58, David Rowley wrote: ... Items 1-4 above I believe require support of Aggregate State Combine Support - https://commitfest.postgresql.org/5/131/ which I believe will need to be modified to implement complex database types to

Re: [HACKERS] The Future of Aggregation

2015-06-09 Thread Alvaro Herrera
Kevin Grittner wrote: Alvaro Herrera alvhe...@2ndquadrant.com wrote: Uh, this also requires serialization and deserialization of non- finalized transition state, no? For that sort of optimization to incremental maintenance of materialized views (when we get there), yes. That will be one

Re: [HACKERS] Aggregate Supporting Functions

2015-06-09 Thread Tom Lane
Kevin Grittner kgri...@ymail.com writes: David Rowley david.row...@2ndquadrant.com wrote: [ avoid duplicate calculations for related aggregates ] From the information you have proposed storing, with cost factors associated with the functions, it seems technically possible to infer that you

Re: [HACKERS] The Future of Aggregation

2015-06-09 Thread Kevin Grittner
David Rowley david.row...@2ndquadrant.com wrote: It appears to me that there's quite a few new features and optimisations on the not too distant horizon which will require adding yet more fields into pg_aggregate. These are things along the lines of: 3. Auto-updating Materialized views

Re: [HACKERS] could not adopt C locale failure at startup on Windows

2015-06-09 Thread Tom Lane
Andres Freund and...@anarazel.de writes: On 2015-06-09 11:20:06 -0400, Tom Lane wrote: We've seen several reports of $SUBJECT lately. I have no idea what's going on, but it occurred to me that it could be informative to tweak init_locale() so that it reports which category failed to be set.

Re: [HACKERS] The Future of Aggregation

2015-06-09 Thread Tomas Vondra
On 06/09/15 17:27, Andres Freund wrote: On 2015-06-09 17:19:33 +0200, Tomas Vondra wrote: ... and yet another use case for 'aggregate state combine' that I just remembered about is grouping sets. What GROUPING SET (ROLLUP, ...) do currently is repeatedly sorting the input, once for each

[HACKERS] Draft release notes for 9.4.4 et al

2015-06-09 Thread Tom Lane
I've pushed up draft release notes for 9.4.4 at http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=21187cfc7dfd82461db9119377a76366c00d27c3 Please review and comment ASAP, particularly if the instructions regarding avoiding emergency autovacuuming are not accurate.

Re: [HACKERS] Draft release notes for 9.4.4 et al

2015-06-09 Thread Andres Freund
Hi, On 2015-06-09 13:09:27 -0400, Tom Lane wrote: I've pushed up draft release notes for 9.4.4 at http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=21187cfc7dfd82461db9119377a76366c00d27c3 Please review and comment ASAP, particularly if the instructions regarding avoiding

[HACKERS] Why no jsonb_exists_path()?

2015-06-09 Thread Josh Berkus
Dmitry, Alexander: I'm noticing a feature gap for JSONB operators; we have no way to do this: jsonb_col ? ARRAY['key1','key2','key3'] ... that is, there is no way for us to check for key existence in an indexable fashion. Given that @ already can check the whole path including the value, is

Re: [HACKERS] Draft release notes for 9.4.4 et al

2015-06-09 Thread Tom Lane
Andres Freund and...@anarazel.de writes: On 2015-06-09 13:09:27 -0400, Tom Lane wrote: I've pushed up draft release notes for 9.4.4 at http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=21187cfc7dfd82461db9119377a76366c00d27c3 Please review and comment ASAP, particularly if the

Re: [HACKERS] Draft release notes for 9.4.4 et al

2015-06-09 Thread Alvaro Herrera
Andres Freund wrote: Hi, On 2015-06-09 13:09:27 -0400, Tom Lane wrote: I've pushed up draft release notes for 9.4.4 at http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=21187cfc7dfd82461db9119377a76366c00d27c3 Please review and comment ASAP, particularly if the

Re: [HACKERS] Why no jsonb_exists_path()?

2015-06-09 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: I'm noticing a feature gap for JSONB operators; we have no way to do this: jsonb_col ? ARRAY['key1','key2','key3'] ... that is, there is no way for us to check for key existence in an indexable fashion. Given that @ already can check the whole path

Re: [HACKERS] [COMMITTERS] pgsql: Add pg_audit, an auditing extension

2015-06-09 Thread David Steele
Hi Noah, On 6/8/15 10:13 AM, Noah Misch wrote: My condemnation of the pg_audit commits probably hurt you as the feature's authors. I am sorry for that. Your code was better than most Ready for Committer code, and I hope you submit more patches in the future. I appreciate you saying this and

Re: [HACKERS] Information of pg_stat_ssl visible to all users

2015-06-09 Thread Michael Paquier
On Tue, Jun 9, 2015 at 3:27 PM, Magnus Hagander mag...@hagander.net wrote: On Jun 9, 2015 6:00 AM, Michael Paquier michael.paqu...@gmail.com wrote: Hi all, I should have noticed that before, but it happens that pg_stat_ssl leaks information about the SSL status of all the users connected to

Re: [HACKERS] [COMMITTERS] pgsql: Add pg_audit, an auditing extension

2015-06-09 Thread Tom Lane
David Steele da...@pgmasters.net writes: My honest, albeit novice, opinion is that it was a mistake to pull pg_audit from contrib. I know more than anyone that it had flaws, mostly owing to its implementation as an extension, but it also provided capability that simply does not exist right

[HACKERS] The purpose of the core team

2015-06-09 Thread Bruce Momjian
There has been some confusion by old and new community members about the purpose of the core team, and this lack of understanding has caused some avoidable problems. Therefore, the core team has written a core charter and published it on our website:

Re: [HACKERS] Draft release notes for 9.4.4 et al

2015-06-09 Thread Josh Berkus
On 06/09/2015 04:38 PM, Michael Paquier wrote: On Wed, Jun 10, 2015 at 8:31 AM, Josh Berkus j...@agliodbs.com wrote: Tom, all: First draft of the release announcement. Please improve/edit/correct. Thanks! Some comments: s/expecially/especially. Thanks, fixed This bug fix is not

[HACKERS] Checkpoints vs restartpoints

2015-06-09 Thread Thomas Munro
Hi Why do standby servers not simply treat every checkpoint as a restartpoint? As I understand it, setting checkpoint_timeout and checkpoint_segments higher on a standby server effectively instruct standby servers to skip some checkpoints. Even with the same settings on both servers, the server

Re: [HACKERS] Draft release notes for 9.4.4 et al

2015-06-09 Thread Josh Berkus
Tom, all: First draft of the release announcement. Please improve/edit/correct. Thanks! -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com 2015-06-12 Update Release = The PostgreSQL Global Development Group has released an update to all supported versions of

Re: [HACKERS] Draft release notes for 9.4.4 et al

2015-06-09 Thread Michael Paquier
On Wed, Jun 10, 2015 at 8:41 AM, Josh Berkus j...@agliodbs.com wrote: On 06/09/2015 04:38 PM, Michael Paquier wrote: On Wed, Jun 10, 2015 at 8:31 AM, Josh Berkus j...@agliodbs.com wrote: Tom, all: First draft of the release announcement. Please improve/edit/correct. Thanks! Some

Re: [HACKERS] Draft release notes for 9.4.4 et al

2015-06-09 Thread Michael Paquier
On Wed, Jun 10, 2015 at 8:31 AM, Josh Berkus j...@agliodbs.com wrote: Tom, all: First draft of the release announcement. Please improve/edit/correct. Thanks! Some comments: s/expecially/especially. This bug fix is not mentioned (worth it?): Avoid deadlock between incoming sessions and

Re: [HACKERS] Checkpoints vs restartpoints

2015-06-09 Thread Jeff Janes
On Tue, Jun 9, 2015 at 4:20 PM, Thomas Munro thomas.mu...@enterprisedb.com wrote: Hi Why do standby servers not simply treat every checkpoint as a restartpoint? As I understand it, setting checkpoint_timeout and checkpoint_segments higher on a standby server effectively instruct standby

Re: [HACKERS] Why no jsonb_exists_path()?

2015-06-09 Thread Andrew Dunstan
On 06/09/2015 05:30 PM, Andrew Dunstan wrote: On 06/09/2015 02:40 PM, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: I'm noticing a feature gap for JSONB operators; we have no way to do this: jsonb_col ? ARRAY['key1','key2','key3'] ... that is, there is no way for us to check for key

Re: [HACKERS] Why no jsonb_exists_path()?

2015-06-09 Thread Andrew Dunstan
On 06/09/2015 02:40 PM, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: I'm noticing a feature gap for JSONB operators; we have no way to do this: jsonb_col ? ARRAY['key1','key2','key3'] ... that is, there is no way for us to check for key existence in an indexable fashion. Given that @

[HACKERS] pg_archivecleanup bug (invalid filename input)

2015-06-09 Thread Joshua D. Drake
Hello, Trying to use pg_archivecleanup as a: standalone archive cleaner Results in an error of: pg_archivecleanup: invalid filename input Try pg_archivecleanup --help for more information. /usr/pgsql-9.4/bin/pg_archivecleanup /backups/db1/archive 0001074800B1.00015838.backup

Re: [HACKERS] psql :: support for \ev viewname and \sv viewname

2015-06-09 Thread Jeevan Chalke
Hi Patch looks excellent now. No issues. Found a typo which I have fixed in the attached patch. Thanks -- Jeevan B Chalke Principal Software Engineer, Product Development EnterpriseDB Corporation The Enterprise PostgreSQL Company diff --git a/doc/src/sgml/ref/psql-ref.sgml

Re: [HACKERS] psql :: support for \ev viewname and \sv viewname

2015-06-09 Thread Jeevan Chalke
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed Patch looks good to pass to committer. The new status of

Re: [HACKERS] Checkpoints vs restartpoints

2015-06-09 Thread Bruce Momjian
On Tue, Jun 9, 2015 at 05:20:23PM -0700, Jeff Janes wrote: On Tue, Jun 9, 2015 at 4:20 PM, Thomas Munro thomas.mu...@enterprisedb.com wrote: Hi Why do standby servers not simply treat every checkpoint as a restartpoint?  As I understand it, setting checkpoint_timeout and

Re: [HACKERS] pg_archivecleanup bug (invalid filename input)

2015-06-09 Thread Michael Paquier
On Wed, Jun 10, 2015 at 7:27 AM, Joshua D. Drake j...@commandprompt.com wrote: Trying to use pg_archivecleanup as a: standalone archive cleaner Results in an error of: pg_archivecleanup: invalid filename input Try pg_archivecleanup --help for more information.

Re: [HACKERS] Checkpoints vs restartpoints

2015-06-09 Thread Michael Paquier
On Wed, Jun 10, 2015 at 9:33 AM, Bruce Momjian wrote: Ah, so even thought standbys don't have to write WAL, they are fsyncing shared buffers. Where is the restart point recorded, in pg_controldata? c Yep. Latest checkpoint's REDO location, or ControlFile-checkPointCopy.redo. During recovery,

Re: [HACKERS] [patch] PL/Python is too lossy with floats

2015-06-09 Thread Peter Eisentraut
On 6/1/15 5:34 AM, Marko Kreen wrote: On Wed, Mar 11, 2015 at 9:49 PM, Peter Eisentraut pete...@gmx.net wrote: On 3/3/15 9:32 AM, Marko Kreen wrote: PL/Python uses str(v) to convert float data, but is lossy by design. Only repr(v) is guaranteed to have enough precision to make floats

Re: [HACKERS] Aggregate Supporting Functions

2015-06-09 Thread David Rowley
On 10 June 2015 at 01:53, Kevin Grittner kgri...@ymail.com wrote: David Rowley david.row...@2ndquadrant.com wrote: 3. Add logic in the planner to look for look for supporting cases. With logic something along the lines of: a. Does the query have any aggregates? If not - return; b.

Re: [HACKERS] The Future of Aggregation

2015-06-09 Thread David Rowley
On 10 June 2015 at 02:52, Kevin Grittner kgri...@ymail.com wrote: David Rowley david.row...@2ndquadrant.com wrote: The idea I discussed in the link in item 5 above gets around this problem, but it's a perhaps more surprise filled implementation as it will mean select avg(x),sum(x),count(x)

Re: [HACKERS] Run pgindent now?

2015-06-09 Thread Bruce Momjian
On Tue, Jun 9, 2015 at 09:56:13PM -0400, Robert Haas wrote: What I really don't want to do is apply the pgindent diff somewhat blindly, without really knowing how many cases we're improving and how many cases we're making worse. The number of times we've run pgindent and then realized later

Re: [HACKERS] Run pgindent now?

2015-06-09 Thread Robert Haas
On Tue, May 26, 2015 at 2:55 PM, Robert Haas robertmh...@gmail.com wrote: This is kind of why I think that reindenting the back branches is unlikely to be productive: it only helps if you can get pgindent to do the same thing on all branches, and I bet that's going to be tough. ...but having

Re: [HACKERS] [patch] PL/Python is too lossy with floats

2015-06-09 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On 6/1/15 5:34 AM, Marko Kreen wrote: Please reconsider backporting. It's been like this forever, so I don't think it's appropriate to backpatch this. http://www.postgresql.org/docs/devel/static/plpython-data.html states in so many words that floats

Re: [HACKERS] The Future of Aggregation

2015-06-09 Thread David Rowley
On 10 June 2015 at 03:25, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Kevin Grittner wrote: Alvaro Herrera alvhe...@2ndquadrant.com wrote: Uh, this also requires serialization and deserialization of non- finalized transition state, no? For that sort of optimization to incremental

Re: [HACKERS] Aggregate Supporting Functions

2015-06-09 Thread Kevin Grittner
David Rowley david.row...@2ndquadrant.com wrote: The idea is that we skip a major chunk of processing in situations like: SELECT avg(x),sum(x),count(x) FROM bigtable; Because avg(x) already technically knows what the values of sum(x) and count(x) are. That has occurred to me as a possible

[HACKERS] Expending the use of xlog_internal.h's macros

2015-06-09 Thread Michael Paquier
Hi all, While looking at the code of pg_archivecleanup.c, I noticed that there is some code present to detect if a given string has the format of a WAL segment file name or of a backup file. The recent commit 179cdd09 has introduced in xlog_internal.h a set of macros to facilitate checks of

Re: [HACKERS] Restore-reliability mode

2015-06-09 Thread Noah Misch
On Wed, Jun 03, 2015 at 04:18:37PM +0200, Andres Freund wrote: On 2015-06-03 09:50:49 -0400, Noah Misch wrote: Second, I would define the subject matter as bug fixes, testing and review, not restructuring, testing and review. Different code structures are clearest to different hackers.

Re: [HACKERS] The Future of Aggregation

2015-06-09 Thread Alvaro Herrera
David Rowley wrote: On 10 June 2015 at 03:25, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Kevin Grittner wrote: Alvaro Herrera alvhe...@2ndquadrant.com wrote: Uh, this also requires serialization and deserialization of non- finalized transition state, no? For that sort

Re: [HACKERS] The Future of Aggregation

2015-06-09 Thread Jim Nasby
On 6/9/15 9:52 AM, Kevin Grittner wrote: Yeah, I think we want to preserve the ability of count() to have a simple state, and implement dependent aggregates as discussed in the other thread -- where (as I understood it) having sum(x), count(x), and avg(x) in a query would avoid the row-by-row

Re: [HACKERS] pg_archivecleanup bug (invalid filename input)

2015-06-09 Thread Joshua D. Drake
On 06/09/2015 05:54 PM, Michael Paquier wrote: Looking at the documentation what is expected is not a path to a segment file, but only a segment file name: http://www.postgresql.org/docs/devel/static/pgarchivecleanup.html So the current behavior is correct, it is actually what

Re: [HACKERS] could not adopt C locale failure at startup on Windows

2015-06-09 Thread Noah Misch
On Tue, Jun 09, 2015 at 12:24:02PM -0400, Tom Lane wrote: Andres Freund and...@anarazel.de writes: The error seems odd. The only even remotely related change between 9.4.1 and .2 seems to be ca325941. Could also be a build environment change. Yeah, my first instinct was to blame ca325941

[HACKERS] Typo fix loged vs logged.

2015-06-09 Thread David Rowley
The attached fixes a small typo in a comment. -- David Rowley http://www.2ndQuadrant.com/ http://www.2ndquadrant.com/ PostgreSQL Development, 24x7 Support, Training Services diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h index fea99c7..e526cd9

[HACKERS] [idea] table partition + hash join

2015-06-09 Thread Kouhei Kaigai
Hello, It might be a corner case optimization, however, it looks to me worth to share the idea and have discussion. Table partition + Hash join pushdown Hash-join logic works most effectively when inner relation can be stored within a hash table. So, it is a