Re: [HACKERS] Proposal: knowing detail of config files via SQL

2015-01-30 Thread Mike Blackwell
​This would default to being available to superusers only, right? Details of the file system shouldn't be available to any random user.​ __ *Mike Blackwell | Technical Analyst, Distribution Services/Rollout

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-01-30 Thread Bruce Momjian
On Fri, Jan 30, 2015 at 11:12:43AM -0500, Robert Haas wrote: I think everyone who has read this mailing list for a while is probably already aware of this problem. When you create a tablespace somewhere inside the data directory, weird things happen. If you pg_upgrade and then incautiously

Re: [HACKERS] Providing catalog view to pg_hba.conf file - Patch submission

2015-01-30 Thread Robert Haas
On Thu, Jan 29, 2015 at 10:13 PM, Amit Kapila amit.kapil...@gmail.com wrote: I think the big problem you are mentioning can be resolved in a similar way as we have done for ALTER SYSTEM which is to have a separate file (.auto.conf) for settings done via ALTER SYSTEM command, do you see any

Re: [HACKERS] pg_dump with both --serializable-deferrable and -j

2015-01-30 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: Kevin Grittner kgri...@ymail.com writes: I propose to apply the attached to master and back-patch to 9.3 Objections? Only the nit-picky one that I quite dislike putting a comment block inside an if-condition like that. Comment moved above the

Re: [HACKERS] Exposing the stats snapshot timestamp to SQL

2015-01-30 Thread David Fetter
On Fri, Jan 30, 2015 at 12:07:22AM -0500, Tom Lane wrote: Matt Kelly mkell...@gmail.com writes: On Thu, Jan 29, 2015 at 8:42 PM, Tom Lane t...@sss.pgh.pa.us wrote: ... or double. Have you checked that the code behaves properly with --disable-integer-timestamps? I presume you meant

Re: [HACKERS] Buildfarm broken for 9.3 and up

2015-01-30 Thread Kevin Grittner
Kevin Grittner kgri...@ymail.com wrote: My push to branches 9.3 and up seems to have broken the buildfarm with this: error: object file /home/pgfarm/buildroot/pgmirror.git/objects/93/d7706cbf2ce58e63ab8bbc9d16453b2c792ed4 is empty error: unable to find

Re: [HACKERS] Buildfarm broken for 9.3 and up

2015-01-30 Thread Andres Freund
On 2015-01-30 15:34:02 +, Kevin Grittner wrote: My push to branches 9.3 and up seems to have broken the buildfarm with this: I think this isn't anything related to your commit. Both racoon and macaque have failed for a while. They just happened to run quickly after your commit, giving the

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-01-30 Thread Joshua D. Drake
On 01/30/2015 08:19 AM, Bruce Momjian wrote: On Fri, Jan 30, 2015 at 11:12:43AM -0500, Robert Haas wrote: I think everyone who has read this mailing list for a while is probably already aware of this problem. When you create a tablespace somewhere inside the data directory, weird things

Re: [HACKERS] Proposal: knowing detail of config files via SQL

2015-01-30 Thread David Fetter
On Fri, Jan 30, 2015 at 09:38:10PM +0900, Sawada Masahiko wrote: On Tue, Jan 27, 2015 at 3:34 AM, Robert Haas robertmh...@gmail.com wrote: On Thu, Jan 22, 2015 at 5:19 PM, Tom Lane t...@sss.pgh.pa.us wrote: David Johnston david.g.johns...@gmail.com writes: On Thu, Jan 22, 2015 at 3:04 PM,

Re: [HACKERS] Buildfarm broken for 9.3 and up

2015-01-30 Thread Stephen Frost
Kevin, * Kevin Grittner (kgri...@ymail.com) wrote: My push to branches 9.3 and up seems to have broken the buildfarm with this: I don't think it has anything to do with your push. A number of members have built with your latest and look fine at this point, and I'm not seeing any issues here

Re: [HACKERS] Buildfarm broken for 9.3 and up

2015-01-30 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: * Kevin Grittner (kgri...@ymail.com) wrote: My push to branches 9.3 and up seems to have broken the buildfarm with this: I don't think it has anything to do with your push. A number of members have built with your latest and look fine at this point,

Re: [HACKERS] Proposal: knowing detail of config files via SQL

2015-01-30 Thread Sawada Masahiko
On Sat, Jan 31, 2015 at 12:58 AM, Mike Blackwell mike.blackw...@rrd.com wrote: This would default to being available to superusers only, right? Details of the file system shouldn't be available to any random user. This WIP patch does not behave like that, but I agree. This view would be

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-01-30 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: Given all this, it seems like a good idea to at least give a warning if somebody tries to create a tablespace instead the data directory. A warning seems like a good idea. I actually thought we *did* prevent it.. Arguably, we should prohibit it

[HACKERS] Re: Overhauling our interrupt handling (was Escaping from blocked send() reprised.)

2015-01-30 Thread Andres Freund
On 2015-01-30 09:29:59 -0500, Robert Haas wrote: On Wed, Jan 14, 2015 at 9:03 PM, Andres Freund and...@2ndquadrant.com wrote: 0002: Use a nonblocking socket for FE/BE communication and block using latches. Has previously been reviewed by Heikki. I think Noah also had a

Re: [HACKERS] Proposal: knowing detail of config files via SQL

2015-01-30 Thread Sawada Masahiko
On Sat, Jan 31, 2015 at 12:24 AM, David Fetter da...@fetter.org wrote: On Fri, Jan 30, 2015 at 09:38:10PM +0900, Sawada Masahiko wrote: On Tue, Jan 27, 2015 at 3:34 AM, Robert Haas robertmh...@gmail.com wrote: On Thu, Jan 22, 2015 at 5:19 PM, Tom Lane t...@sss.pgh.pa.us wrote: David Johnston

[HACKERS] tablespaces inside $PGDATA considered harmful

2015-01-30 Thread Robert Haas
I think everyone who has read this mailing list for a while is probably already aware of this problem. When you create a tablespace somewhere inside the data directory, weird things happen. If you pg_upgrade and then incautiously run the delete_old_cluster.sh script thus created, you will blow

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-01-30 Thread Robert Haas
On Fri, Jan 30, 2015 at 11:43 AM, Joshua D. Drake j...@commandprompt.com wrote: I mean yes a warning is good but it is after the fact, the tablespace is already created. We know that tablespaces in $PGDATA are a bad idea, why not protect the user? Please go back and read the discussion of

[HACKERS] Buildfarm broken for 9.3 and up

2015-01-30 Thread Kevin Grittner
My push to branches 9.3 and up seems to have broken the buildfarm with this: error: object file /home/pgfarm/buildroot/pgmirror.git/objects/93/d7706cbf2ce58e63ab8bbc9d16453b2c792ed4 is empty error: unable to find 93d7706cbf2ce58e63ab8bbc9d16453b2c792ed4 fatal: SHA1 COLLISION FOUND WITH

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-01-30 Thread David Steele
On 1/30/15 11:43 AM, Joshua D. Drake wrote: On 01/30/2015 08:19 AM, Bruce Momjian wrote: On Fri, Jan 30, 2015 at 11:12:43AM -0500, Robert Haas wrote: I think everyone who has read this mailing list for a while is probably already aware of this problem. When you create a tablespace somewhere

Re: [HACKERS] Hot Standby WAL reply uses heavyweight session locks, but doesn't have enough infrastructure set up

2015-01-30 Thread Andres Freund
On 2015-01-29 11:01:51 -0500, Robert Haas wrote: On Wed, Jan 28, 2015 at 2:41 AM, Michael Paquier michael.paqu...@gmail.com wrote: Andres Freund wrote: I think this isn't particularly pretty, but it seems to be working well enough, and changing it would be pretty invasive. So keeping in

Re: [HACKERS] Buildfarm broken for 9.3 and up

2015-01-30 Thread Andrew Dunstan
On 01/30/2015 11:00 AM, Tom Lane wrote: I do see that macque is having issues, but it's been busted for the past 3 days it looks like. Yeah. Several of the critters have been having git issues for weeks to months. I wonder whether we couldn't teach the buildfarm script to recover from this

Re: [HACKERS] Possible typo in create_policy.sgml

2015-01-30 Thread Stephen Frost
* Dean Rasheed (dean.a.rash...@gmail.com) wrote: Don't forget the ALTER POLICY page. This and some of the other things being discussed on this thread ought to be copied there too. Thanks, I've fixed this also. Stephen signature.asc Description: Digital signature

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-01-30 Thread Josh Berkus
Robert, Stephen, etc.: Apparently you can create a tablespace in the tablespace directory: josh=# create tablespace tbl location '/home/josh/pg94/data/pg_tblspc/'; CREATE TABLESPACE josh=# create table test_tbl ( test text ) tablespace tbl; CREATE TABLE josh=# \q

Re: [HACKERS] Possible typo in create_policy.sgml

2015-01-30 Thread Stephen Frost
* Dean Rasheed (dean.a.rash...@gmail.com) wrote: On 30 January 2015 at 03:40, Stephen Frost sfr...@snowman.net wrote: * Robert Haas (robertmh...@gmail.com) wrote: On Thu, Jan 29, 2015 at 9:04 PM, Stephen Frost sfr...@snowman.net wrote: A policy grants the ability to SELECT, INSERT, UPDATE,

Re: [HACKERS] Parallel Seq Scan

2015-01-30 Thread Stephen Frost
Daniel, * Daniel Bausch (bau...@dvs.tu-darmstadt.de) wrote: I have been researching this topic long time ago. One notably fact is that active prefetching disables automatic readahead prefetching (by Linux kernel), which can occour in larger granularities than 8K. Automatic readahead

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE} 2.0

2015-01-30 Thread Peter Geoghegan
On Fri, Jan 30, 2015 at 6:59 AM, Geoff Winkless pgsqlad...@geoff.dj wrote: I appreciate the work you're doing and (as a user rather than a pg-hacker) don't want to butt in but if it would be possible to allow support for IGNORE for unique but not exclusion constraints that would be really

Re: [HACKERS] Providing catalog view to pg_hba.conf file - Patch submission

2015-01-30 Thread Jim Nasby
On 1/29/15 9:13 PM, Amit Kapila wrote: Aside from Tom's concern about sets not being a good way to handle this (which I agree with), the idea of editing pg_hba.conf via SQL raises all the problems that were brought up when ALTER SYSTEM was being developed. One of the big problems is a question

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-01-30 Thread David G Johnston
Robert Haas wrote Arguably, we should prohibit it altogether, but there are obviously people that want to do it, and there could even be somewhat valid reasons for that, Lots of hand-waving here and it is just as likely they simply are not aware of the downsides and the only reason they put

Re: [HACKERS] basebackups during ALTER DATABASE ... SET TABLESPACE ... not safe?

2015-01-30 Thread Andres Freund
On 2015-01-27 20:16:43 +0100, Andres Freund wrote: Here's an alternative approach. I think it generally is superior and going in the right direction, but I'm not sure it's backpatchable. It basically consists out of: 1) Make GetLockConflicts() actually work. already commited as being a

Re: [HACKERS] Proposal: knowing detail of config files via SQL

2015-01-30 Thread Tom Lane
David Johnston david.g.johns...@gmail.com writes: On Fri, Jan 30, 2015 at 12:05 PM, David Fetter da...@fetter.org wrote: Why might the contents of pg_settings be different from what would be in pg_file_settings, apart from the existence of this column? ​​The contents of pg_settings uses

Re: [HACKERS] Safe memory allocation functions

2015-01-30 Thread Robert Haas
On Fri, Jan 30, 2015 at 1:10 AM, Michael Paquier michael.paqu...@gmail.com wrote: I wrote: Yes, this refactoring was good for testing actually... Oops, I have been too hasty when sending previous patch, there was a bug related to huge allocations. Patch correcting this bug is attached.

Re: [HACKERS] Proposal: knowing detail of config files via SQL

2015-01-30 Thread David Johnston
On Fri, Jan 30, 2015 at 12:05 PM, David Fetter da...@fetter.org wrote: On Sat, Jan 31, 2015 at 12:50:20AM +0900, Sawada Masahiko wrote: [postgres][5432](1)=# select * from pg_file_settings where name = 'work_mem'; -[ RECORD 1 ]-- name

[HACKERS] Re: Overhauling our interrupt handling (was Escaping from blocked send() reprised.)

2015-01-30 Thread Heikki Linnakangas
On 01/15/2015 03:03 AM, Andres Freund wrote: 0004: Process 'die' interrupts while reading/writing from the client socket. This is the reason Horiguchi-san started this thread. + ProcessClientWriteInterrupt(!port-noblock); ... +/* + * ProcessClientWriteInterrupt() -

[HACKERS] Fwd: [GENERAL] 4B row limit for CLOB tables

2015-01-30 Thread Roger Pack
On 1/29/15, Roger Pack rogerdpa...@gmail.com wrote: Hello. I see on this page a mention of basically a 4B row limit for tables that have BLOB's Oops I meant for BYTEA or TEXT columns, but it's possible the reasoning is the same... It only applies to large objects, not bytea or text. OK I

[HACKERS]

2015-01-30 Thread jeff . janes
From nobody Fri Jan 30 18:20:23 2015 Content-Type: text/plain; charset=utf-8 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: Re: Parallel Seq Scan To: pgsql-hackers@postgresql.org From: Jeff Janes jeff.ja...@gmail.com Date: Fri, 30 Jan 2015 18:20:23 + User-Agent: pgcommitfest

Re: [HACKERS] tablespaces inside $PGDATA considered harmful

2015-01-30 Thread Josh Berkus
On 01/30/2015 09:19 AM, Stephen Frost wrote: * Robert Haas (robertmh...@gmail.com) wrote: Given all this, it seems like a good idea to at least give a warning if somebody tries to create a tablespace instead the data directory. A warning seems like a good idea. I actually thought we *did*

Re: [HACKERS] PATCH: Reducing lock strength of trigger and foreign key DDL

2015-01-30 Thread Andreas Karlsson
On 01/30/2015 07:48 AM, Michael Paquier wrote: Ok, so the deal is to finally reduce the locks to ShareRowExclusiveLock for the following commands : - CREATE TRIGGER - ALTER TABLE ENABLE/DISABLE - ALTER TABLE ADD CONSTRAINT Correct. I personally still find this useful enough to justify a patch.

Re: [HACKERS] Proposal: knowing detail of config files via SQL

2015-01-30 Thread David Fetter
On Sat, Jan 31, 2015 at 12:50:20AM +0900, Sawada Masahiko wrote: On Sat, Jan 31, 2015 at 12:24 AM, David Fetter da...@fetter.org wrote: On Fri, Jan 30, 2015 at 09:38:10PM +0900, Sawada Masahiko wrote: On Tue, Jan 27, 2015 at 3:34 AM, Robert Haas robertmh...@gmail.com wrote: On Thu, Jan 22,

Re: [HACKERS] Fwd: [GENERAL] 4B row limit for CLOB tables

2015-01-30 Thread Jim Nasby
On 1/30/15 11:54 AM, Roger Pack wrote: On 1/29/15, Roger Pack rogerdpa...@gmail.com wrote: Hello. I see on this page a mention of basically a 4B row limit for tables that have BLOB's Oops I meant for BYTEA or TEXT columns, but it's possible the reasoning is the same... It only applies to

Re: [HACKERS] parallel mode and parallel contexts

2015-01-30 Thread Jim Nasby
On 1/30/15 11:08 AM, Robert Haas wrote: The final patch attached her (parallel-dummy-v2.patch) has been updated slightly to incorporate some prefetching logic. It's still just demo code and is not intended for commit. I'm not sure whether the prefetching logic can actually be made to improve

[HACKERS] Re: Overhauling our interrupt handling (was Escaping from blocked send() reprised.)

2015-01-30 Thread Heikki Linnakangas
On 01/15/2015 03:03 AM, Andres Freund wrote: 0002: Use a nonblocking socket for FE/BE communication and block using latches. s/suceeds/succeeds/ 0004: Process 'die' interrupts while reading/writing from the client socket. +* Check for interrupts here, in

Re: [HACKERS] PageRepairFragmentation performance

2015-01-30 Thread Peter Geoghegan
On Tue, Nov 18, 2014 at 10:03 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: So there's clearly some room for improvement here. A couple of ideas: 1. Replace the qsort with something cheaper. The itemid arrays being sorted are small, a few hundred item at most, usually even smaller. In

Re: [HACKERS] Proposal: knowing detail of config files via SQL

2015-01-30 Thread Sawada Masahiko
On Sat, Jan 31, 2015 at 4:56 AM, Tom Lane t...@sss.pgh.pa.us wrote: David Johnston david.g.johns...@gmail.com writes: On Fri, Jan 30, 2015 at 12:05 PM, David Fetter da...@fetter.org wrote: Why might the contents of pg_settings be different from what would be in pg_file_settings, apart from the

Re: [HACKERS] Providing catalog view to pg_hba.conf file - Patch submission

2015-01-30 Thread Amit Kapila
On Fri, Jan 30, 2015 at 10:58 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Jan 29, 2015 at 10:13 PM, Amit Kapila amit.kapil...@gmail.com wrote: I think the big problem you are mentioning can be resolved in a similar way as we have done for ALTER SYSTEM which is to have a separate

Re: [HACKERS] Proposal: knowing detail of config files via SQL

2015-01-30 Thread Sawada Masahiko
On Sat, Jan 31, 2015 at 3:20 PM, Sawada Masahiko sawada.m...@gmail.com wrote: On Sat, Jan 31, 2015 at 2:00 PM, Sawada Masahiko sawada.m...@gmail.com wrote: On Sat, Jan 31, 2015 at 4:56 AM, Tom Lane t...@sss.pgh.pa.us wrote: David Johnston david.g.johns...@gmail.com writes: On Fri, Jan 30,

Re: [HACKERS] Proposal: knowing detail of config files via SQL

2015-01-30 Thread Sawada Masahiko
On Sat, Jan 31, 2015 at 2:00 PM, Sawada Masahiko sawada.m...@gmail.com wrote: On Sat, Jan 31, 2015 at 4:56 AM, Tom Lane t...@sss.pgh.pa.us wrote: David Johnston david.g.johns...@gmail.com writes: On Fri, Jan 30, 2015 at 12:05 PM, David Fetter da...@fetter.org wrote: Why might the contents of

[HACKERS] Re: Overhauling our interrupt handling (was Escaping from blocked send() reprised.)

2015-01-30 Thread Robert Haas
On Wed, Jan 14, 2015 at 9:03 PM, Andres Freund and...@2ndquadrant.com wrote: 0002: Use a nonblocking socket for FE/BE communication and block using latches. Has previously been reviewed by Heikki. I think Noah also had a look, although I'm not sure how close that was.

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE} 2.0

2015-01-30 Thread Geoff Winkless
On Thu, Jan 29, 2015 at 11:38 PM, Peter Geoghegan pg(at)heroku(dot)com wrote: Simply removing IGNORE support until such time as we straighten that all out (9.6?) seems like the simplest solution. No need to block the progress of UPSERT, since exclusion constraint support was only ever going to

Re: [HACKERS] Possible typo in create_policy.sgml

2015-01-30 Thread Dean Rasheed
On 30 January 2015 at 03:40, Stephen Frost sfr...@snowman.net wrote: * Robert Haas (robertmh...@gmail.com) wrote: On Thu, Jan 29, 2015 at 9:04 PM, Stephen Frost sfr...@snowman.net wrote: A policy grants the ability to SELECT, INSERT, UPDATE, or DELETE rows which match the relevant policy

[HACKERS] Odd behavior of updatable security barrier views on foreign tables

2015-01-30 Thread Etsuro Fujita
Hi, I noticed that when updating security barrier views on foreign tables, we fail to give FOR UPDATE to selection queries issued at ForeignScan. Here is an example. postgres=# create foreign table base_ftbl (person text, visibility text) server loopback options (table_name 'base_tbl'); CREATE

Re: [HACKERS] jsonb, unicode escapes and escaped backslashes

2015-01-30 Thread Peter Geoghegan
On Thu, Jan 29, 2015 at 11:28 PM, Tom Lane t...@sss.pgh.pa.us wrote: The point of JSONB is that we take a position on certain aspects like this. We're bridging a pointedly loosey goosey interchange format, JSON, with native PostgreSQL types. For example, we take a firm position on encoding.

Re: [HACKERS] Redesigning checkpoint_segments

2015-01-30 Thread Heikki Linnakangas
On 01/30/2015 04:48 AM, Venkata Balaji N wrote: I performed series of tests for this patch and would like to share the results. My comments are in-line. Thanks for the testing! *Test 1 :* In this test, i see removed+recycled segments = 3 (except for the first 3 checkpoint cycles) and has

Re: [HACKERS] Performance improvement for joins where outer side is unique

2015-01-30 Thread David Rowley
On 1 January 2015 at 02:47, David Rowley dgrowle...@gmail.com wrote: Hi, I've been hacking a bit at the join code again... This time I've been putting some effort into optimising the case where the inner side of the join is known to be unique. For example, given the tables: create table

Re: [HACKERS] Proposal: knowing detail of config files via SQL

2015-01-30 Thread Sawada Masahiko
On Tue, Jan 27, 2015 at 3:34 AM, Robert Haas robertmh...@gmail.com wrote: On Thu, Jan 22, 2015 at 5:19 PM, Tom Lane t...@sss.pgh.pa.us wrote: David Johnston david.g.johns...@gmail.com writes: On Thu, Jan 22, 2015 at 3:04 PM, Tom Lane t...@sss.pgh.pa.us wrote: Is that a requirement, and if so

[HACKERS] NULL checks of deferenced pointers in picksplit method of intarray

2015-01-30 Thread Michael Paquier
Hi all, Coverity is pointing out that _int_split.c has unnecessary checks for deferenced pointers in 5 places. - if (inter_d != (ArrayType *) NULL) - pfree(inter_d); + pfree(inter_d); In this case inter_d is generated by