Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-06-11 Thread David Rowley
On Mon, Jun 9, 2014 at 11:20 PM, Marti Raudsepp ma...@juffo.org wrote: On Sun, Jun 8, 2014 at 3:36 PM, David Rowley dgrowle...@gmail.com wrote: Currently pull_up_sublinks_qual_recurse only changes the plan for NOT EXISTS queries and leaves NOT IN alone. The reason for this is because the

Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-06-11 Thread David Rowley
On Tue, Jun 10, 2014 at 2:19 PM, Tom Lane t...@sss.pgh.pa.us wrote: Jeff Janes jeff.ja...@gmail.com writes: If you are using NOT IN, then once you find a NULL in the outer input (if the outer input is the in-list: clearly you can't reverse the two inputs in this case), you don't even need

Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-06-11 Thread Marti Raudsepp
On Wed, Jun 11, 2014 at 11:53 AM, David Rowley dgrowle...@gmail.com wrote: The only way to consistently guarantee nullability is through primary key constraints. Fortunately that addresses most of the use cases of NOT IN(), in my experience. See the comment in check_functional_grouping: I

Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-06-11 Thread Marti Raudsepp
On Sun, Jun 8, 2014 at 3:36 PM, David Rowley dgrowle...@gmail.com wrote: Currently pull_up_sublinks_qual_recurse only changes the plan for NOT EXISTS queries and leaves NOT IN alone. The reason for this is because the values returned by a subquery in the IN clause could have NULLs. There's a

Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-06-11 Thread David Rowley
On Wed, Jun 11, 2014 at 9:32 PM, Marti Raudsepp ma...@juffo.org wrote: On Sun, Jun 8, 2014 at 3:36 PM, David Rowley dgrowle...@gmail.com wrote: Currently pull_up_sublinks_qual_recurse only changes the plan for NOT EXISTS queries and leaves NOT IN alone. The reason for this is because the

[HACKERS] replication commands and log_statements

2014-06-11 Thread Fujii Masao
Hi, Replication commands like IDENTIFY_COMMAND are not logged even when log_statements is set to all. Some users who use log_statements to audit *all* statements might dislike this current situation. So I'm thinking to change log_statements or add something like log_replication so that we can log

Re: [HACKERS] Compression of full-page-writes

2014-06-11 Thread Fujii Masao
On Wed, Jun 11, 2014 at 10:05 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Tue, Jun 10, 2014 at 11:49 PM, Rahila Syed rahilasye...@gmail.com wrote: Hello , In order to facilitate changing of compression algorithms and to be able to recover using WAL records compressed with

Re: [HACKERS] Compression of full-page-writes

2014-06-11 Thread Pavan Deolasee
On Wed, Jun 11, 2014 at 4:19 PM, Fujii Masao masao.fu...@gmail.com wrote: IIUC even when we adopt only one algorithm, additional at least one bit is necessary to see whether this backup block is compressed or not. This flag is necessary only for backup block, so there is no need to use the

Re: [HACKERS] replication commands and log_statements

2014-06-11 Thread Andres Freund
On 2014-06-11 19:34:25 +0900, Fujii Masao wrote: Hi, Replication commands like IDENTIFY_COMMAND are not logged even when log_statements is set to all. Some users who use log_statements to audit *all* statements might dislike this current situation. So I'm thinking to change log_statements

Re: [HACKERS] replication commands and log_statements

2014-06-11 Thread Magnus Hagander
On Wed, Jun 11, 2014 at 12:34 PM, Fujii Masao masao.fu...@gmail.com wrote: Hi, Replication commands like IDENTIFY_COMMAND are not logged even when log_statements is set to all. Some users who use log_statements to audit *all* statements might dislike this current situation. So I'm thinking

Re: [HACKERS] replication commands and log_statements

2014-06-11 Thread Andres Freund
On 2014-06-11 13:42:58 +0200, Magnus Hagander wrote: On Wed, Jun 11, 2014 at 12:34 PM, Fujii Masao masao.fu...@gmail.com wrote: Hi, Replication commands like IDENTIFY_COMMAND are not logged even when log_statements is set to all. Some users who use log_statements to audit *all*

Re: [HACKERS] replication commands and log_statements

2014-06-11 Thread Magnus Hagander
On Wed, Jun 11, 2014 at 2:17 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-06-11 13:42:58 +0200, Magnus Hagander wrote: On Wed, Jun 11, 2014 at 12:34 PM, Fujii Masao masao.fu...@gmail.com wrote: Hi, Replication commands like IDENTIFY_COMMAND are not logged even when

Re: [HACKERS] replication commands and log_statements

2014-06-11 Thread Andres Freund
On 2014-06-11 14:22:43 +0200, Magnus Hagander wrote: Yes, but how would you specify for example i want DDL and all replication commands (which is quite a reasonable thing to log, I believe). If you actually require it to be set to all, most people won't have any use at all for it... That's a

Re: [HACKERS] tests for client programs

2014-06-11 Thread Andres Freund
Hi, On 2014-06-04 20:40:40 -0400, Peter Eisentraut wrote: On Wed, 2014-05-07 at 03:08 +0200, Andres Freund wrote: As an additional issue it currently doesn't seem to work in VPATH builds. That's imo a must fix. A cd $(srcdir) .. in prove_installcheck and prove_check seems to do the

Re: [HACKERS] replication commands and log_statements

2014-06-11 Thread Magnus Hagander
On Wed, Jun 11, 2014 at 2:35 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-06-11 14:22:43 +0200, Magnus Hagander wrote: Yes, but how would you specify for example i want DDL and all replication commands (which is quite a reasonable thing to log, I believe). If you actually

Re: [HACKERS] Inaccuracy in VACUUM's tuple count estimates

2014-06-11 Thread Andres Freund
On 2014-06-09 11:24:22 -0700, Kevin Grittner wrote: Andres Freund and...@2ndquadrant.com wrote: On 2014-06-09 09:45:12 -0700, Kevin Grittner wrote: I am not sure, given predicate.c's coding, how HEAPTUPLE_DELETE_IN_PROGRESS could cause problems. Could you elaborate, since that's the

Re: [HACKERS] replication commands and log_statements

2014-06-11 Thread Andres Freund
On 2014-06-11 14:50:34 +0200, Magnus Hagander wrote: On Wed, Jun 11, 2014 at 2:35 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-06-11 14:22:43 +0200, Magnus Hagander wrote: Yes, but how would you specify for example i want DDL and all replication commands (which is quite a

Re: [HACKERS] pg_receivexlog add synchronous mode

2014-06-11 Thread Fujii Masao
On Tue, Jun 10, 2014 at 5:01 PM, furu...@pm.nttdata.co.jp wrote: No. IIUC walreceiver does flush *less* frequently than what you implemented on pg_receivexlog. Your version of pg_receivexlog tries to do flush every time when it receives one WAL chunk. OTOH, walreceiver does flush only when

Re: [HACKERS] Inaccuracy in VACUUM's tuple count estimates

2014-06-11 Thread Kevin Grittner
Andres Freund and...@2ndquadrant.com wrote: On 2014-06-09 11:24:22 -0700, Kevin Grittner wrote: The only way that it could be a problem is if the DELETE is in a subtransaction which might get rolled back without rolling back the INSERT. The way I understand the code in that case the subxid

Re: [HACKERS] [GENERAL] Question about partial functional indexes and the query planner

2014-06-11 Thread Robert Haas
On Tue, Jun 10, 2014 at 7:19 PM, Tom Lane t...@sss.pgh.pa.us wrote: Given the lack of previous complaints, I'm not sure this amounts to a back-patchable bug, but it does seem like something worth fixing going forward. Agreed, although I'd be willing to see us slip it into 9.4. It's doubtful

Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-06-11 Thread Tom Lane
Marti Raudsepp ma...@juffo.org writes: On Wed, Jun 11, 2014 at 11:53 AM, David Rowley dgrowle...@gmail.com wrote: as long as the transaction id is taken before we start planning in session 1 then it should not matter if another session drops the constraint and inserts a NULL value as we won't

Re: [HACKERS] API change advice: Passing plan invalidation info from the rewriter into the planner?

2014-06-11 Thread Robert Haas
On Tue, Jun 10, 2014 at 7:18 PM, Craig Ringer cr...@2ndquadrant.com wrote: On 06/11/2014 02:19 AM, Tom Lane wrote: Hm ... I'm not following why we'd need a special case for superusers and not anyone else? Seems like any useful RLS scheme is going to require more privilege levels than just

Re: [HACKERS] replication commands and log_statements

2014-06-11 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: Your wish just seems like a separate feature to me. Including replication commands in 'all' seems correct independent of the desire for a more granular control. No, I think I've got to vote with the other side on that. The reason we can have

Re: [HACKERS] Proposing pg_hibernate

2014-06-11 Thread Robert Haas
On Tue, Jun 10, 2014 at 10:03 PM, Gurjeet Singh gurj...@singh.im wrote: And it's probably accepted by now that such a bahviour is not catastrophic, merely inconvenient. I think the whole argument for having pg_hibernator is that getting the block cache properly initialized is important. If

Re: [HACKERS] API change advice: Passing plan invalidation info from the rewriter into the planner?

2014-06-11 Thread Stephen Frost
* Craig Ringer (cr...@2ndquadrant.com) wrote: On 06/11/2014 02:19 AM, Tom Lane wrote: Hm ... I'm not following why we'd need a special case for superusers and not anyone else? Seems like any useful RLS scheme is going to require more privilege levels than just superuser and not-superuser.

Re: [HACKERS] API change advice: Passing plan invalidation info from the rewriter into the planner?

2014-06-11 Thread Stephen Frost
* Craig Ringer (cr...@2ndquadrant.com) wrote: On 06/11/2014 07:24 AM, Tom Lane wrote: Is the point of that that the table owner might have put trojan-horse functions into the RLS qual? If so, why are we only concerned about defending the superuser and not other users? Seems like the right

[HACKERS] [GSoC2014] Patch ALTER TABLE ... SET LOGGED

2014-06-11 Thread Fabrízio de Royes Mello
Hi all, As part of GSoC2014 I'm sending a patch to add the capability of change an unlogged table to logged [1]. I'll add it to the 9.5CF1. Regards, [1] https://wiki.postgresql.org/wiki/Allow_an_unlogged_table_to_be_changed_to_logged_GSoC_2014 -- Fabrízio de Royes Mello Consultoria/Coaching

Re: [HACKERS] API change advice: Passing plan invalidation info from the rewriter into the planner?

2014-06-11 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: I'm really concerned about the security implications of this patch. I think we're setting ourselves up for a whole lot of hurt for somewhat unclear gain. I'm certainly of a different opinion and, for the most part, I feel that if there are security

Re: [HACKERS] Allowing NOT IN to use ANTI joins

2014-06-11 Thread Greg Stark
On Wed, Jun 11, 2014 at 3:26 PM, Tom Lane t...@sss.pgh.pa.us wrote: If we didn't have mechanisms like this, we'd have far worse hazards from ALTER TABLE than whether the planner made an incorrect join optimization. Consider ALTER COLUMN TYPE for instance. Obviously not general cases of ALTER

Re: [HACKERS] API change advice: Passing plan invalidation info from the rewriter into the planner?

2014-06-11 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: * Robert Haas (robertmh...@gmail.com) wrote: I'm really concerned about the security implications of this patch. I think we're setting ourselves up for a whole lot of hurt for somewhat unclear gain. I'm certainly of a different opinion and, for the

Re: [HACKERS] wrapping in extended mode doesn't work well with default pager

2014-06-11 Thread Peter Eisentraut
On 6/8/14, 11:29 PM, Noah Misch wrote: The patch did not restore 9.3 behavior for that one. Starting with commit 6513633, the first line of letters is space-padded on the right to the width of the second line of letters. To illustrate, I have attached raw psql output from both commit 6513633

Re: [HACKERS] API change advice: Passing plan invalidation info from the rewriter into the planner?

2014-06-11 Thread Robert Haas
On Wed, Jun 11, 2014 at 12:23 PM, Stephen Frost sfr...@snowman.net wrote: In my view, commit 842faa714c0454d67e523f5a0b6df6500e9bc1a5 basically *is* row-level security: instead of applying a row-level security policy to a table, just create a security-barrier view over the table and grant

[HACKERS] view reloptions

2014-06-11 Thread Alvaro Herrera
I just noticed by chance that view relations are using StdRdOptions to allocate their reloptions. I can't find any reason for this, other than someone failed to realize that they should instead have a struct defined of its own, just like (say) GIN indexes do. Views using StdRdOptions is quite

Re: [HACKERS] wrapping in extended mode doesn't work well with default pager

2014-06-11 Thread Greg Stark
On Wed, Jun 11, 2014 at 7:52 PM, Peter Eisentraut pete...@gmx.net wrote: On 6/8/14, 11:29 PM, Noah Misch wrote: The patch did not restore 9.3 behavior for that one. Starting with commit 6513633, the first line of letters is space-padded on the right to the width of the second line of letters.

Re: [HACKERS] wrapping in extended mode doesn't work well with default pager

2014-06-11 Thread Greg Stark
And Gmail has thoroughly mangled that email. Let me see if I can resend it from Emacs more clearly. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] view reloptions

2014-06-11 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: Is it too late to redefine the check_option_offset stuff before 9.4 ships, so that it is in its own struct? We have a forced initdb already for beta2, so I'd say not. 3) I don't have time to do the legwork before CF1 anyway ... but if nobody

Re: [HACKERS] view reloptions

2014-06-11 Thread Fabrízio de Royes Mello
On Wed, Jun 11, 2014 at 4:46 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: I just noticed by chance that view relations are using StdRdOptions to allocate their reloptions. I can't find any reason for this, other than someone failed to realize that they should instead have a struct

Re: [HACKERS] [GENERAL] Question about partial functional indexes and the query planner

2014-06-11 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Jun 10, 2014 at 7:19 PM, Tom Lane t...@sss.pgh.pa.us wrote: Given the lack of previous complaints, I'm not sure this amounts to a back-patchable bug, but it does seem like something worth fixing going forward. Agreed, although I'd be willing

Re: [HACKERS] API change advice: Passing plan invalidation info from the rewriter into the planner?

2014-06-11 Thread Stephen Frost
Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: Stephen Frost sfr...@snowman.net writes: * Robert Haas (robertmh...@gmail.com) wrote: I'm really concerned about the security implications of this patch. I think we're setting ourselves up for a whole lot of hurt for somewhat unclear gain.

Re: [HACKERS] [patch] build issues on Win32

2014-06-11 Thread Noah Misch
On Wed, Mar 10, 2010 at 10:51:23AM -0500, Tom Lane wrote: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= d...@des.no writes: -LDFLAGS=-Wl,--allow-multiple-definition +LDFLAGS=${LDFLAGS} -Wl,--allow-multiple-definition That bit seems sane. I've committed that part. Thanks. -- Noah Misch

Re: [HACKERS] API change advice: Passing plan invalidation info from the rewriter into the planner?

2014-06-11 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Wed, Jun 11, 2014 at 12:23 PM, Stephen Frost sfr...@snowman.net wrote: This argument could have been made for column-level privileges also, no? Not really. First of all, we didn't have security_barrier views at that time, let alone security

Re: [HACKERS] wrapping in extended mode doesn't work well with default pager

2014-06-11 Thread Noah Misch
On Wed, Jun 11, 2014 at 08:59:34PM +0100, Greg Stark wrote: The leading space that you (ie Peter) are complaining about in: col | 1 +-- col | 2 Is there because if the cell wrapped it would get an ellipsis (ie '...' but it's a single unicode character) in that column to indicate

Re: [HACKERS] wrapping in extended mode doesn't work well with default pager

2014-06-11 Thread Tom Lane
Noah Misch n...@leadboat.com writes: Based on the commit message and procedural history, I thought commit 6513633 was changing behavior solely for the combination of \pset expanded and \pset format wrapped. Peter's and my test cases show that it also changed behavior for \pset expanded alone.

Re: [HACKERS] API change advice: Passing plan invalidation info from the rewriter into the planner?

2014-06-11 Thread Robert Haas
On Wed, Jun 11, 2014 at 8:59 PM, Stephen Frost sfr...@snowman.net wrote: Row-level security is not a chance for the system to deny access; it's a chance for user-defined code to take control and perform arbitrary operations. So the scope of what we're contemplating for row-level security is

[HACKERS] Few observations in replication slots related code

2014-06-11 Thread Amit Kapila
Few observations in Replication slots related code: 1. In function StartupReplicationSlots(XLogRecPtr checkPointRedo), parameter checkPointRedo is not used. 2. Few check are in different order in functions pg_create_physical_replication_slot() and pg_create_logical_replication_slot(). if

Re: [HACKERS] postgresql.auto.conf read from wrong directory

2014-06-11 Thread Amit Kapila
On Tue, May 27, 2014 at 10:35 AM, Amit Kapila amit.kapil...@gmail.com wrote: On Sun, May 11, 2014 at 11:23 PM, Tom Lane t...@sss.pgh.pa.us wrote: I think it's clearly *necessary* to forbid setting data_directory in postgresql.auto.conf. The file is defined to be found in the data

[HACKERS] lo_create(oid, bytea) breaks every extant release of libpq

2014-06-11 Thread Tom Lane
While investigating a different issue, I was astonished to find that pg_restore in HEAD is incapable of restoring dumps containing large objects: it fails with messages like pg_restore: [archiver] could not create large object 10: ERROR: function call message contains 1 arguments but

Re: [HACKERS] Proposing pg_hibernate

2014-06-11 Thread Gurjeet Singh
On Wed, Jun 11, 2014 at 12:25 AM, Amit Kapila amit.kapil...@gmail.com wrote: On Wed, Jun 11, 2014 at 7:59 AM, Gurjeet Singh gurj...@singh.im wrote: On Sun, Jun 8, 2014 at 3:24 AM, Amit Kapila amit.kapil...@gmail.com wrote: Yeap, but if it crashes before writing checkpoint record, it will lead

Re: [HACKERS] Proposing pg_hibernate

2014-06-11 Thread Gurjeet Singh
On Wed, Jun 11, 2014 at 10:56 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, Jun 10, 2014 at 10:03 PM, Gurjeet Singh gurj...@singh.im wrote: And it's probably accepted by now that such a bahviour is not catastrophic, merely inconvenient. I think the whole argument for having

Re: [HACKERS] lo_create(oid, bytea) breaks every extant release of libpq

2014-06-11 Thread Tatsuo Ishii
Meanwhile, we have to either revert the addition of lo_create(oid, bytea) altogether, or choose a different name for it. Suggestions? I wonder if there's any use case where we need to store bytea into large objects. Don't we already have bytea data type? If the use case is for large data which

Re: [HACKERS] Something flaky in the relfilenode mapping infrastructure

2014-06-11 Thread Noah Misch
On Tue, Apr 15, 2014 at 03:28:41PM -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-03-27 08:02:35 -0400, Tom Lane wrote: Buildfarm member prairiedog thinks there's something unreliable about commit f01d1ae3a104019d6d68aeff85c4816a275130b3: So I had made a

Re: [HACKERS] lo_create(oid, bytea) breaks every extant release of libpq

2014-06-11 Thread Noah Misch
On Wed, Jun 11, 2014 at 11:57:20PM -0400, Tom Lane wrote: While there's certainly a good argument to be made for making lo_initialize do that query differently, there's no way that we can fix every copy of libpq that's in the field. I think we have to consider that there can be only one

Re: [HACKERS] lo_create(oid, bytea) breaks every extant release of libpq

2014-06-11 Thread Pavel Stehule
2014-06-12 6:22 GMT+02:00 Tatsuo Ishii is...@postgresql.org: Meanwhile, we have to either revert the addition of lo_create(oid, bytea) altogether, or choose a different name for it. Suggestions? I wonder if there's any use case where we need to store bytea into large objects. Don't we

Re: [HACKERS] lo_create(oid, bytea) breaks every extant release of libpq

2014-06-11 Thread Tom Lane
Noah Misch n...@leadboat.com writes: On Wed, Jun 11, 2014 at 11:57:20PM -0400, Tom Lane wrote: Meanwhile, we have to either revert the addition of lo_create(oid, bytea) altogether, or choose a different name for it. Suggestions? lo_new() or lo_make()? An earlier draft of the patch that

Re: [HACKERS] lo_create(oid, bytea) breaks every extant release of libpq

2014-06-11 Thread Pavel Stehule
2014-06-12 6:54 GMT+02:00 Noah Misch n...@leadboat.com: On Wed, Jun 11, 2014 at 11:57:20PM -0400, Tom Lane wrote: While there's certainly a good argument to be made for making lo_initialize do that query differently, there's no way that we can fix every copy of libpq that's in the field.

Re: [HACKERS] Something flaky in the relfilenode mapping infrastructure

2014-06-11 Thread Tom Lane
Noah Misch n...@leadboat.com writes: The improved test just tripped: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=prairiedogdt=2014-06-12%2000%3A17%3A07 Ugh. If the MTBF is circa three months, how will we catch this before we're dead? A quick look around the machine's buildfarm

Re: [HACKERS] lo_create(oid, bytea) breaks every extant release of libpq

2014-06-11 Thread Pavel Stehule
2014-06-12 7:08 GMT+02:00 Tom Lane t...@sss.pgh.pa.us: Noah Misch n...@leadboat.com writes: On Wed, Jun 11, 2014 at 11:57:20PM -0400, Tom Lane wrote: Meanwhile, we have to either revert the addition of lo_create(oid, bytea) altogether, or choose a different name for it. Suggestions?