Re: [HACKERS] pgrowlocks relkind check

2017-06-13 Thread Amit Langote
On 2017/06/13 22:53, Peter Eisentraut wrote: > On 6/12/17 21:10, Amit Langote wrote: >> On 2017/06/13 0:29, Peter Eisentraut wrote: >>> On 4/24/17 21:22, Amit Langote wrote: >> create extension pgrowlocks; >> create view one as select 1; >> select pgrowlocks('one'); >> -- ERROR:

Re: [HACKERS] Dropping partitioned table drops a previously detached partition

2017-06-13 Thread Ashutosh Bapat
On Wed, Jun 14, 2017 at 10:21 AM, Ashutosh Bapat wrote: > On Tue, Jun 13, 2017 at 7:14 PM, Rahila Syed wrote: >> I reviewed and tested >> 0001-Dependency-between-partitioned-table-and-partition_v1.patch >> It applies cleanly on master and

Re: [HACKERS] Dropping partitioned table drops a previously detached partition

2017-06-13 Thread Ashutosh Bapat
On Tue, Jun 13, 2017 at 9:23 PM, Robert Haas wrote: > On Tue, Jun 13, 2017 at 9:44 AM, Rahila Syed wrote: >> I have added tests to the >> 0001-Dependency-between-partitioned-table-and-partition_v1.patch. Please >> find attached the v2 patch. > >

[HACKERS] type of release note of PG10

2017-06-13 Thread Yugo Nagata
Hi, I found a typo in the PG10 release note and attached is a patch to fix it. Regards, -- Yugo Nagata diff --git a/doc/src/sgml/release-10.sgml b/doc/src/sgml/release-10.sgml index b10086bd..f3e4a70 100644 --- a/doc/src/sgml/release-10.sgml +++

Re: [HACKERS] Dropping partitioned table drops a previously detached partition

2017-06-13 Thread Ashutosh Bapat
On Tue, Jun 13, 2017 at 7:14 PM, Rahila Syed wrote: > I reviewed and tested > 0001-Dependency-between-partitioned-table-and-partition_v1.patch > It applies cleanly on master and make check passes. > > Following are few comments: > >>/* >> * Drop the dependency created by

Re: [HACKERS] v10beta pg_catalog diagrams

2017-06-13 Thread Pavel Stehule
2017-06-14 5:53 GMT+02:00 Peter Eisentraut : > On 6/13/17 17:08, Andres Freund wrote: > > I wondered before if we shouldn't introduce "information only" > > unenforced foreign key constraints for the catalogs. We kind of > > manually do that via oidjoins, it'd

Re: [HACKERS] pg_receivewal and messages printed in non-verbose mode

2017-06-13 Thread Michael Paquier
On Tue, Jun 13, 2017 at 4:50 PM, Craig Ringer wrote: > On 13 June 2017 at 14:33, Michael Paquier wrote: >> Those come from stop_streaming in pg_receivewal.c. Shouldn't those >> messages only show up to the user if --verbose is used? It seems >>

Re: [HACKERS] outfuncs.c utility statement support

2017-06-13 Thread Amit Langote
On 2017/06/14 12:49, Peter Eisentraut wrote: > On 6/13/17 11:25, Peter Eisentraut wrote: >> Running with --debug-print-parse=on, executing >> >> create table test1 (a int, b text); >> >> gives output that is truncated somewhere in the middle (possibly a null >> byte) > > So this seems to be a

Re: [HACKERS] v10beta pg_catalog diagrams

2017-06-13 Thread Peter Eisentraut
On 6/13/17 17:08, Andres Freund wrote: > I wondered before if we shouldn't introduce "information only" > unenforced foreign key constraints for the catalogs. We kind of > manually do that via oidjoins, it'd be nicer if we'd a function > rechecking fkeys, and the fkeys were in the catalog... I

Re: [HACKERS] A bug in mapping attributes in ATExecAttachPartition()

2017-06-13 Thread Amit Langote
On 2017/06/14 5:36, Robert Haas wrote: > On Tue, Jun 13, 2017 at 10:24 AM, Robert Haas wrote: >> I think that's going to come as an unpleasant surprise to more than >> one user. I'm not sure exactly how we need to restructure things here >> so that this works properly, and

Re: [HACKERS] outfuncs.c utility statement support

2017-06-13 Thread Peter Eisentraut
On 6/13/17 11:25, Peter Eisentraut wrote: > Running with --debug-print-parse=on, executing > > create table test1 (a int, b text); > > gives output that is truncated somewhere in the middle (possibly a null > byte) So this seems to be a pretty basic bug. Some node fields of type char may be

Re: [HACKERS] Detection of IPC::Run presence in SSL TAP tests

2017-06-13 Thread Michael Paquier
On Tue, Jun 13, 2017 at 11:14 PM, Tom Lane wrote: > Certainly, it's pointless to have a defense only here. And I know very > well that make check falls over in an ugly, hard-to-interpret-if-you've- > not-seen-it-before fashion if you do --enable-tap-tests and don't have >

Re: [HACKERS] ICU support on Windows

2017-06-13 Thread Ashutosh Sharma
On Tue, Jun 13, 2017 at 6:45 PM, Peter Eisentraut wrote: > On 6/12/17 14:03, Ashutosh Sharma wrote: >>> I noticed that this only works if you use the "Win32" download of ICU, >>> because the "Win64" download uses "lib64" paths. I'm not sure what the >>> impact

Re: [HACKERS] Document bug regarding read only transactions

2017-06-13 Thread Tatsuo Ishii
> Your modification does not look completely correct to me either. > Temporary sequences can be updated in read-only transactions. Not sure. Temporary sequences are meaningless on standby because "create temporary sequence" command cannot be executed on standby anyway (and temporary sequence are

Re: [HACKERS] Document bug regarding read only transactions

2017-06-13 Thread Michael Paquier
On Wed, Jun 14, 2017 at 11:08 AM, Tatsuo Ishii wrote: > > In normal operation, read-only transactions are allowed to > -update sequences and to use LISTEN, UNLISTEN, and > +use LISTEN, UNLISTEN, and > NOTIFY, so Hot Standby sessions operate under

Re: [HACKERS] ICU support on Windows

2017-06-13 Thread Craig Ringer
On 13 June 2017 at 05:47, Alvaro Herrera wrote: > Michael Paquier wrote: >> On Tue, Jun 13, 2017 at 3:13 AM, Alvaro Herrera >> wrote: >> > Ashutosh Sharma wrote: > >> >> Yes, that's right, Win64 download uses lib64 path and in my case i had >>

Re: [HACKERS] A bug in mapping attributes in ATExecAttachPartition()

2017-06-13 Thread Amit Langote
On 2017/06/13 23:24, Robert Haas wrote: > On Mon, Jun 12, 2017 at 4:09 AM, Amit Langote > wrote: >> On 2017/06/09 20:49, Ashutosh Bapat wrote: >>> May be we should pass a flag to predicate_implied_by() to handle NULL >>> behaviour for CHECK constraints. Partitioning

[HACKERS] Document bug regarding read only transactions

2017-06-13 Thread Tatsuo Ishii
In https://www.postgresql.org/docs/10/static/hot-standby.html#hot-standby-users It is explained that read only transactions (not in standby) allow to update sequences. In normal operation, read-only transactions are allowed to update sequences and to use LISTEN, UNLISTEN, and NOTIFY,

Re: [HACKERS] A bug in mapping attributes in ATExecAttachPartition()

2017-06-13 Thread Robert Haas
On Tue, Jun 13, 2017 at 5:28 PM, Tom Lane wrote: > Robert Haas writes: >> OK, I think I see the problem here. predicate_implied_by() and >> predicate_refuted_by() differ in what they assume about the predicate >> evaluating to NULL, but both of them

Re: [HACKERS] PG10 Partitioned tables and relation_is_updatable()

2017-06-13 Thread Amit Langote
Hi Dean, On 2017/06/14 2:29, Dean Rasheed wrote: > On 13 June 2017 at 05:50, Ashutosh Bapat > wrote: >> On Tue, Jun 13, 2017 at 12:03 AM, Dean Rasheed >> wrote: >>> Barring objections, I'll push my original patch and work up patches

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jun 13, 2017 at 03:20:12PM -0400, Stephen Frost wrote: > > > OK, so let's go back. You are saying there are no security benefits to > > > this vs. file system encryption. > > > > I'm not sure that I can see any, myself.. Perhaps I'm

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Peter, * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > On 6/13/17 15:20, Stephen Frost wrote: > > And then you would need openssl on the other system to decrypt it. > > Or make the USB file system encrypted as well? If you're in that kind > of environment, that would surely be

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Peter, * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > On 6/13/17 15:20, Stephen Frost wrote: > > No, the benefit is that the database administrator can configure it and > > set it up and not have to get an OS-level administrator involved. There > > may also be other reasons why

Re: [HACKERS] A bug in mapping attributes in ATExecAttachPartition()

2017-06-13 Thread Tom Lane
Robert Haas writes: > OK, I think I see the problem here. predicate_implied_by() and > predicate_refuted_by() differ in what they assume about the predicate > evaluating to NULL, but both of them assume that if the clause > evaluates to NULL, that's equivalent to false.

Re: pgindent (was Re: [HACKERS] [COMMITTERS] pgsql: Preventive maintenance in advance of pgindent run.)

2017-06-13 Thread Tom Lane
Piotr Stefaniak writes: > On 2017-06-13 22:23, Tom Lane wrote: >> I could not find any places where reverting this change made the >> results worse, so I'm unclear on why you made it. > I must admit I'm a bit confused about why it's not fixed yet, but I'll > have to

Re: [HACKERS] fix possible optimizations in ATExecAttachPartition()

2017-06-13 Thread Jeevan Ladhe
On Wed, Jun 14, 2017 at 2:12 AM, Robert Haas wrote: > On Tue, Jun 13, 2017 at 5:22 AM, Amit Langote > wrote: > > Yeah, I was thinking the same while writing the patch posted on the > thread > > "A bug in mapping attributes in

Re: pgindent (was Re: [HACKERS] [COMMITTERS] pgsql: Preventive maintenance in advance of pgindent run.)

2017-06-13 Thread Piotr Stefaniak
On 2017-06-13 22:23, Tom Lane wrote: > I could not find any places where reverting this change made the > results worse, so I'm unclear on why you made it. I must admit I'm a bit confused about why it's not fixed yet, but I'll have to analyze that later this week. But the idea was to convince

Re: pgindent (was Re: [HACKERS] [COMMITTERS] pgsql: Preventive maintenance in advance of pgindent run.)

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 05:00:31PM -0400, Tom Lane wrote: > Anyway, it is now time to fish or cut bait. I don't think we can wait > much longer to decide whether we're going to adopt this new indent > version for PG 10. I think we should. The floor is open for votes. Works for me. -- Bruce

Re: [HACKERS] v10beta pg_catalog diagrams

2017-06-13 Thread Andres Freund
Hi, On 2017-06-12 11:28:39 -0400, Neil Anderson wrote: > I'm cross posting from general. I did some work to diagram the relationships > in pg_catalog for v10. I would like to add it to the developer FAQ here >

Re: [HACKERS] RTE_NAMEDTUPLESTORE, enrtuples and comments

2017-06-13 Thread Thomas Munro
On Wed, Jun 14, 2017 at 4:22 AM, Robert Haas wrote: > I'm just trying to understand your comments so that I can have an > intelligent opinion about what we should do from here. Given that the > replan wouldn't happen anyway, there seems to be no reason to tinker > with the

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread David Fetter
On Tue, Jun 13, 2017 at 10:28:14AM -0400, Peter Eisentraut wrote: > On 6/13/17 09:24, Stephen Frost wrote: > > but there are use-cases where it'd be really nice to be able to > > have PG doing the encryption instead of the filesystem because > > then you can do things like backup the database,

Re: pgindent (was Re: [HACKERS] [COMMITTERS] pgsql: Preventive maintenance in advance of pgindent run.)

2017-06-13 Thread Piotr Stefaniak
On 2017-06-13 18:22, Tom Lane wrote: > The Makefile is still BSD-ish of course, but I think > we'll just agree to disagree there. For compiling indent under Linux I use bmake(1). I have no problem with including a Makefile for GNU Make in my repository. As I understand it, there will be a copy

Re: [HACKERS] fix possible optimizations in ATExecAttachPartition()

2017-06-13 Thread Robert Haas
On Tue, Jun 13, 2017 at 5:22 AM, Amit Langote wrote: > Yeah, I was thinking the same while writing the patch posted on the thread > "A bug in mapping attributes in ATExecAttachPartition()" [1]. That patch > adds the break you mention in 2, but didn't do anything

Re: [HACKERS] v10beta pg_catalog diagrams

2017-06-13 Thread Robert Haas
On Tue, Jun 13, 2017 at 3:14 PM, Neil Anderson wrote: > There were a few relationships that I couldn't capture. Like where in > pg_extension extconfig is an array of oids that refer to pg_class or where > pg_depends could refer to basically any other system catalog, but

Re: [HACKERS] A bug in mapping attributes in ATExecAttachPartition()

2017-06-13 Thread Robert Haas
On Tue, Jun 13, 2017 at 10:24 AM, Robert Haas wrote: > I think that's going to come as an unpleasant surprise to more than > one user. I'm not sure exactly how we need to restructure things here > so that this works properly, and maybe modifying > predicate_implied_by()

Re: [HACKERS] GSoC 2017 weekly progress reports (week 2)

2017-06-13 Thread Kevin Grittner
On Tue, Jun 13, 2017 at 1:02 PM, Andrew Borodin wrote: > 2017-06-13 18:00 GMT+05:00 Shubham Barai : > Good job! +1! :-) > So, in current HEAD test predicate_gist_2.spec generate false > positives, but with your patch, it does not? Keep in mind,

Re: pgindent (was Re: [HACKERS] [COMMITTERS] pgsql: Preventive maintenance in advance of pgindent run.)

2017-06-13 Thread Tom Lane
I've now done a round of comparisons of results of our old indent with your current version. There's still one serious bug in the latter: it continues to misformat enum typedefs, for instance *** PG_FUNCTION_INFO_V1(pg_prewarm); *** 33,40 typedef enum {

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 04:08:29PM -0400, Peter Eisentraut wrote: > On 6/13/17 15:51, Bruce Momjian wrote: > > Isn't the leakage controlled by OS permissions, so is it really leakage, > > i.e., if you can see the leakage, you probably have bypassed the OS > > permissions and see the key and data

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Peter Eisentraut
On 6/13/17 15:51, Bruce Momjian wrote: > Isn't the leakage controlled by OS permissions, so is it really leakage, > i.e., if you can see the leakage, you probably have bypassed the OS > permissions and see the key and data anyway. One scenario (among many) is when you're done with the disk. If

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 03:20:12PM -0400, Stephen Frost wrote: > Bruce, > > * Bruce Momjian (br...@momjian.us) wrote: > > On Tue, Jun 13, 2017 at 02:38:58PM -0400, Stephen Frost wrote: > > > It's good to discuss what the feature would bring and what cases it > > > doesn't cover, as well as

Re: [HACKERS] Get stuck when dropping a subscription during synchronizing table

2017-06-13 Thread Peter Eisentraut
On 6/13/17 02:33, Noah Misch wrote: >> Steps to reproduce - >> X cluster -> create 100 tables , publish all tables (create publication pub >> for all tables); >> Y Cluster -> create 100 tables ,create subscription(create subscription sub >> connection 'user=centos host=localhost' publication pub;

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Peter Eisentraut
On 6/13/17 15:20, Stephen Frost wrote: > For example, you could simply do: > > cp -a /path/to/PG /mnt/usb > > and you're done. If you're using filesystem level encryption then you'd > have to re-encrypt the data, using something like: > > tar -cf - /path/to/PG | openssl -key private.key > >

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Peter Eisentraut
On 6/13/17 15:20, Stephen Frost wrote: > No, the benefit is that the database administrator can configure it and > set it up and not have to get an OS-level administrator involved. There > may also be other reasons why filesystem-level encryption is difficult > to set up or use in a certain

Re: [HACKERS] Broken hint bits (freeze)

2017-06-13 Thread Bruce Momjian
On Mon, Jun 12, 2017 at 06:31:11PM +0300, Vladimir Borodin wrote: > What about the following sequence? > > 1. Run pg_upgrade on master, > 2. Start it in single-user mode and stop (to get right wal_level in > pg_control), > 3. Copy pg_control somewhere, > 4. Start master, run analyze and stop. >

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jun 13, 2017 at 02:38:58PM -0400, Stephen Frost wrote: > > It's good to discuss what the feature would bring and what cases it > > doesn't cover, as well as discussing how it can be designed to make sure > > that later improvements are

Re: [HACKERS] v10beta pg_catalog diagrams

2017-06-13 Thread Neil Anderson
On 2017-06-13 1:22 PM, Bruce Momjian wrote: On Mon, Jun 12, 2017 at 04:07:35PM -0400, Peter Eisentraut wrote: On 6/12/17 11:28, Neil Anderson wrote: I'm cross posting from general. I did some work to diagram the relationships in pg_catalog for v10. I would like to add it to the developer FAQ

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 02:38:58PM -0400, Stephen Frost wrote: > It's good to discuss what the feature would bring and what cases it > doesn't cover, as well as discussing how it can be designed to make sure > that later improvements are able to be done without having to change it > around. I do

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jun 13, 2017 at 02:23:39PM -0400, Stephen Frost wrote: > > I'm not trying to shut down discussion, I'm simply pointing out where > > this feature will be helpful and where it won't be. If there's a way to > > make it better and able to

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 02:23:39PM -0400, Stephen Frost wrote: > I'm not trying to shut down discussion, I'm simply pointing out where > this feature will be helpful and where it won't be. If there's a way to > make it better and able to address an attack where the OS permission > system is

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jun 13, 2017 at 01:25:00PM -0400, Stephen Frost wrote: > > > I think the big win of Postgres doing the encryption is that the > > > user-visible file system is no longer a target (assuming OS permissions > > > are bypassed), while for

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jun 13, 2017 at 01:44:51PM -0400, Stephen Frost wrote: > > Just to be clear, I don't have any issue with discussing the idea that > > we want to get to a point where we can work with multiple keys and > > encrypt different tables with

Re: [HACKERS] GSoC 2017 weekly progress reports (week 2)

2017-06-13 Thread Andrew Borodin
2017-06-13 18:00 GMT+05:00 Shubham Barai : > > Project: Explicitly support predicate locks in index AMs besides b-tree > Hi, Shubham Good job! So, in current HEAD test predicate_gist_2.spec generate false positives, but with your patch, it does not? I'd suggest keeping

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 01:44:51PM -0400, Stephen Frost wrote: > Just to be clear, I don't have any issue with discussing the idea that > we want to get to a point where we can work with multiple keys and > encrypt different tables with different keys (or not encrypt certain > tables, et al) with

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Joe, * Joe Conway (m...@joeconway.com) wrote: > On 06/13/2017 10:20 AM, Stephen Frost wrote: > > * Joe Conway (m...@joeconway.com) wrote: > >> Except shell escaping issues, etc, etc > > > > That's not an issue- we're talking about reading the stdout of some > > other process, there's no shell

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 01:25:00PM -0400, Stephen Frost wrote: > > I think the big win of Postgres doing the encryption is that the > > user-visible file system is no longer a target (assuming OS permissions > > are bypassed), while for file system encryption it is the storage device > > that is

Re: [HACKERS] PG10 Partitioned tables and relation_is_updatable()

2017-06-13 Thread Dean Rasheed
On 13 June 2017 at 05:50, Ashutosh Bapat wrote: > On Tue, Jun 13, 2017 at 12:03 AM, Dean Rasheed > wrote: >> Barring objections, I'll push my original patch and work up patches >> for the other couple of issues I found. > > No

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Joe Conway
On 06/13/2017 10:20 AM, Stephen Frost wrote: > * Joe Conway (m...@joeconway.com) wrote: >> Except shell escaping issues, etc, etc > > That's not an issue- we're talking about reading the stdout of some > other process, there's no shell escaping that has to be done there. It could be an issue

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jun 13, 2017 at 01:01:32PM -0400, Stephen Frost wrote: > > > Well, usually the symetric key is stored using RSA and a symetric > > > cipher is used to encrypt/decrypt the data. I was thinking of a case > > > where you encrypt a row using

Re: [HACKERS] v10beta pg_catalog diagrams

2017-06-13 Thread Bruce Momjian
On Mon, Jun 12, 2017 at 04:07:35PM -0400, Peter Eisentraut wrote: > On 6/12/17 11:28, Neil Anderson wrote: > > I'm cross posting from general. I did some work to diagram the > > relationships in pg_catalog for v10. I would like to add it to the > > developer FAQ here > >

Re: [HACKERS] Why are we restricting exported snapshots in subtransactions?

2017-06-13 Thread Andres Freund
On 2017-06-13 13:15:57 -0400, Robert Haas wrote: > On Mon, Jun 12, 2017 at 11:04 PM, Andres Freund wrote: > > ExportSnapshot() has, right at the beginning, the following block: > > > > /* > > * We cannot export a snapshot from a subtransaction because there's no > >

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Joe, * Joe Conway (m...@joeconway.com) wrote: > Except shell escaping issues, etc, etc That's not an issue- we're talking about reading the stdout of some other process, there's no shell escaping that has to be done there. > > Let us, please, stop stressing over the right way to do key

[HACKERS] Announcing Release 5 of the PostgreSQL Buildfarm Client

2017-06-13 Thread Andrew Dunstan
Release 5 of the PostgreSQL Buildfarm Client has been released and can be downloaded from In a similar move to PostgreSQL version numbering, with this release we move to a one part numbering system. In addition to a number

Re: [HACKERS] Why are we restricting exported snapshots in subtransactions?

2017-06-13 Thread Robert Haas
On Mon, Jun 12, 2017 at 11:04 PM, Andres Freund wrote: > ExportSnapshot() has, right at the beginning, the following block: > > /* > * We cannot export a snapshot from a subtransaction because there's no > * easy way for importers to verify that the same

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 01:01:32PM -0400, Stephen Frost wrote: > > Well, usually the symetric key is stored using RSA and a symetric > > cipher is used to encrypt/decrypt the data. I was thinking of a case > > where you encrypt a row using a symetric key, then store RSA-encrypted > > versions of

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Joe Conway
On 06/13/2017 10:05 AM, Stephen Frost wrote: > Bruce, Joe, > > * Bruce Momjian (br...@momjian.us) wrote: >> On Tue, Jun 13, 2017 at 09:55:10AM -0700, Joe Conway wrote: >> > > That way, if the user wants to store the key in an unencrypted text >> > > file, they can set the encryption_key_command =

Re: [HACKERS] Transactional sequence stuff breaks pg_upgrade

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 11:14:02AM -0400, Robert Haas wrote: > Also, I think that if we did it that way, it would be significantly > harder to debug. Right now, if something goes boom, you can look at > the old and new clusters and figure out what doesn't match, but if > pg_upgrade renumbered

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Bruce, Joe, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jun 13, 2017 at 09:55:10AM -0700, Joe Conway wrote: > > > That way, if the user wants to store the key in an unencrypted text > > > file, they can set the encryption_key_command = 'cat /not/very/secure' > > > and call it a day. If

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jun 13, 2017 at 12:23:01PM -0400, Stephen Frost wrote: > > > Of course, if the > > > key stored in the database is visible to someone using the operating > > > system, we really haven't added much/any security --- I guess my point > > >

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 09:55:10AM -0700, Joe Conway wrote: > > That way, if the user wants to store the key in an unencrypted text > > file, they can set the encryption_key_command = 'cat /not/very/secure' > > and call it a day. If they want to prompt the user on the console or > > request the

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Joe Conway
On 06/13/2017 09:28 AM, Robert Haas wrote: > On Tue, Jun 13, 2017 at 12:23 PM, Stephen Frost wrote: >> Key management is an entirely independent discussion from this and the >> proposal from Ants, as I understand it, is that the key would *not* be >> in the database but could

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 12:23:01PM -0400, Stephen Frost wrote: > > As I understand it, having encryption in the database means the key is > > stored in the database, while having encryption in the file system means > > the key is stored in the operating system somewhere. > > Key management is

Re: [HACKERS] remove unnecessary flag has_null from PartitionBoundInfoData

2017-06-13 Thread Robert Haas
On Mon, Jun 12, 2017 at 9:03 AM, Ashutosh Bapat wrote: > On Mon, Jun 12, 2017 at 3:50 PM, amul sul wrote: >> On Wed, May 17, 2017 at 10:22 PM, Robert Haas wrote: >> [...] >>> I committed this with fixes for those issues,

Re: [HACKERS] Refreshing subscription relation state inside a transaction block

2017-06-13 Thread Masahiko Sawada
On Wed, Jun 14, 2017 at 1:02 AM, Masahiko Sawada wrote: > On Tue, Jun 13, 2017 at 4:53 PM, Petr Jelinek > wrote: >> On 13/06/17 09:06, Masahiko Sawada wrote: >>> Hi, >>> >>> The commit ddd7b22b225ae41d16ceb218b387645cb9becfdc makes table sync

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Robert Haas
On Tue, Jun 13, 2017 at 12:23 PM, Stephen Frost wrote: > Key management is an entirely independent discussion from this and the > proposal from Ants, as I understand it, is that the key would *not* be > in the database but could be anywhere that a shell command could get it >

Re: pgindent (was Re: [HACKERS] [COMMITTERS] pgsql: Preventive maintenance in advance of pgindent run.)

2017-06-13 Thread Tom Lane
Piotr Stefaniak writes: >> There's also the portability issues: __FBSDID() and bcopy() and >> [and err.h]. > I think that's fixed as well. I just finished some preliminary portability testing and things look much improved. The Makefile is still BSD-ish of course,

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jun 13, 2017 at 11:04:21AM -0400, Stephen Frost wrote: > > > Also, in the use case you describe, if you use pg_basebackup to make a > > > direct encrypted copy of a data directory, I think that would mean you'd > > > have to keep using

Re: [HACKERS] RTE_NAMEDTUPLESTORE, enrtuples and comments

2017-06-13 Thread Robert Haas
On Tue, Jun 13, 2017 at 12:04 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Jun 13, 2017 at 11:53 AM, Tom Lane wrote: >>> But it needs to be changeable, unless you like the proposition that we >>> can never replan a query

Re: [HACKERS] memory fields from getrusage()

2017-06-13 Thread Robert Haas
On Sat, Jun 10, 2017 at 9:31 PM, Tom Lane wrote: > We already do call getrusage(). The point of that comment is that the > contents of the resulting struct rusage are not very well standardized. > POSIX says only > > The header defines the rusage structure that includes

Re: [HACKERS] Typo in BRIN documentation

2017-06-13 Thread Julien Rouhaud
On Tue, Jun 13, 2017 at 11:29:30AM -0400, Peter Eisentraut wrote: > On 6/13/17 07:53, Julien Rouhaud wrote: > > I just found this typo while doing french translation, patch attached. > > fixed > Thanks ! -- Julien Rouhaud http://dalibo.com - http://dalibo.org -- Sent via pgsql-hackers mailing

Re: [HACKERS] RTE_NAMEDTUPLESTORE, enrtuples and comments

2017-06-13 Thread Tom Lane
Robert Haas writes: > On Tue, Jun 13, 2017 at 11:53 AM, Tom Lane wrote: >> But it needs to be changeable, unless you like the proposition that we >> can never replan a query inside a trigger on the basis of new information >> about how big the

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 11:04:21AM -0400, Stephen Frost wrote: > > Also, in the use case you describe, if you use pg_basebackup to make a > > direct encrypted copy of a data directory, I think that would mean you'd > > have to keep using the same key for all copies. > > That's true, but that

Re: [HACKERS] Refreshing subscription relation state inside a transaction block

2017-06-13 Thread Masahiko Sawada
On Tue, Jun 13, 2017 at 4:53 PM, Petr Jelinek wrote: > On 13/06/17 09:06, Masahiko Sawada wrote: >> Hi, >> >> The commit ddd7b22b225ae41d16ceb218b387645cb9becfdc makes table sync >> workers stop when subscription relation entry is removed. It doesn't >> work fine

Re: [HACKERS] RTE_NAMEDTUPLESTORE, enrtuples and comments

2017-06-13 Thread Robert Haas
On Tue, Jun 13, 2017 at 11:53 AM, Tom Lane wrote: >> How does it break those properties? I don't think enrtuples is being >> modified by planning or execution as things stand. > > But it needs to be changeable, unless you like the proposition that we > can never replan a

Re: [HACKERS] Dropping partitioned table drops a previously detached partition

2017-06-13 Thread Robert Haas
On Tue, Jun 13, 2017 at 9:44 AM, Rahila Syed wrote: > I have added tests to the > 0001-Dependency-between-partitioned-table-and-partition_v1.patch. Please > find attached the v2 patch. Thanks. Committed. I don't think the 0002 patch is an improvement - sure, it keeps

Re: [HACKERS] RTE_NAMEDTUPLESTORE, enrtuples and comments

2017-06-13 Thread Tom Lane
Robert Haas writes: > On Tue, Jun 13, 2017 at 10:05 AM, Tom Lane wrote: >> Well, the fundamental problem is that the RTE is a lousy place to keep >> rowcount estimates. That breaks assorted desirable properties like >> querytrees being readonly to

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 11:35:03AM -0400, Robert Haas wrote: > I anticipate that one of the trickier problems here will be handling > encryption of the write-ahead log. Suppose you encrypt WAL a block at > a time. In the current system, once you've written and flushed a > block, you can consider

Re: [HACKERS] RTE_NAMEDTUPLESTORE, enrtuples and comments

2017-06-13 Thread Robert Haas
On Tue, Jun 13, 2017 at 10:05 AM, Tom Lane wrote: > Robert Haas writes: >> Perhaps this is a silly question, but I don't particularly see what's >> wrong with: > >> 3. Do nothing. > > Well, the fundamental problem is that the RTE is a lousy place to

Re: [HACKERS] outfuncs.c utility statement support

2017-06-13 Thread Tom Lane
Peter Eisentraut writes: > While debugging some other stuff, I was wondering to what extent node > types supporting utility statements should be supported in outfuncs.c. We've largely not tried too hard in that department. From a debugging standpoint it'd be

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Robert Haas
On Mon, Jun 12, 2017 at 5:11 PM, Ants Aasma wrote: > Fundamentally there doesn't seem to be a big benefit of implementing > the encryption at PostgreSQL level instead of the filesystem. The > patch doesn't take any real advantage from the higher level knowledge > of the

Re: [HACKERS] Typo in BRIN documentation

2017-06-13 Thread Peter Eisentraut
On 6/13/17 07:53, Julien Rouhaud wrote: > I just found this typo while doing french translation, patch attached. fixed -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list

Re: [HACKERS] macOS Sierra & System Integrity Protection

2017-06-13 Thread Simon Riggs
On 13 June 2017 at 04:25, Robert Haas wrote: > I have a new MacBook Pro running Sierra. Congratulations. > 'make check' was failing: 'psql' repeatedly died with an abort > trap. Binaries worked fine when I ran them from the command line > (sometimes with

[HACKERS] outfuncs.c utility statement support

2017-06-13 Thread Peter Eisentraut
While debugging some other stuff, I was wondering to what extent node types supporting utility statements should be supported in outfuncs.c. Running with --debug-print-parse=on, executing create table test1 (a int, b text); gives output that is truncated somewhere in the middle (possibly a null

Re: [HACKERS] macOS Sierra & System Integrity Protection

2017-06-13 Thread Tom Lane
Simon Riggs writes: > On 13 June 2017 at 04:25, Robert Haas wrote: >> 'make check' was failing: 'psql' repeatedly died with an abort >> trap. Binaries worked fine when I ran them from the command line >> (sometimes with DYLD_LIBRARY_PATH, if needed)

Re: [HACKERS] Transactional sequence stuff breaks pg_upgrade

2017-06-13 Thread Robert Haas
On Tue, Jun 13, 2017 at 9:37 AM, Tom Lane wrote: > In the long run, it would certainly be cleaner if pg_upgrade dropped > the force-the-relfilenode-assignment approach and instead remapped > relfilenodes from old cluster to new. But I think it's just for > cleanliness rather

Re: [HACKERS] macOS Sierra & System Integrity Protection

2017-06-13 Thread Peter Eisentraut
On 6/12/17 23:38, Tom Lane wrote: > https://www.postgresql.org/message-id/26098.1446697...@sss.pgh.pa.us > >> My main purpose in writing this email is to pass along what I learned >> in the hopes of sparing somebody else some trouble, but perhaps there >> is a way to modify our regression test

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Peter, * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > I wonder what the proper extent of "encryption at rest" should be. If > you encrypt just on a file or block level, then someone looking at the > data directory or a backup can still learn a number of things about the > number

Re: [HACKERS] Fix a typo in shm_mq.c

2017-06-13 Thread Peter Eisentraut
On 6/12/17 21:32, Masahiko Sawada wrote: > Attached the patch for $subject. > > s/Whem/When/ committed -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list

Re: [HACKERS] tablesync.c - comment improvements

2017-06-13 Thread Peter Eisentraut
On 6/10/17 04:52, Erik Rijkers wrote: > tablesync.c - comment improvements Committed, thanks! -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Peter Eisentraut
On 6/13/17 09:24, Stephen Frost wrote: > but there are > use-cases where it'd be really nice to be able to have PG doing the > encryption instead of the filesystem because then you can do things like > backup the database, copy it somewhere else directly, and then restore > it using the regular PG

Re: [HACKERS] A bug in mapping attributes in ATExecAttachPartition()

2017-06-13 Thread Robert Haas
On Mon, Jun 12, 2017 at 4:09 AM, Amit Langote wrote: > On 2017/06/09 20:49, Ashutosh Bapat wrote: >> May be we should pass a flag to predicate_implied_by() to handle NULL >> behaviour for CHECK constraints. Partitioning has shown that it needs >> to use

  1   2   >