Re: [HACKERS] WIP: Data at rest encryption

2017-06-19 Thread Robert Haas
On Mon, Jun 19, 2017 at 12:30 PM, Robert Haas wrote: > On Thu, Jun 15, 2017 at 7:51 PM, Alvaro Herrera > wrote: >> I thought we called it "incremental development". From the opposite >> point of view, would you say we should ban use of

Re: [HACKERS] WIP: Data at rest encryption

2017-06-19 Thread Robert Haas
On Thu, Jun 15, 2017 at 7:51 PM, Alvaro Herrera wrote: > I thought we called it "incremental development". From the opposite > point of view, would you say we should ban use of passphrase-protected > SSL key files because the current user interface for them is bad? I

Re: [HACKERS] WIP: Data at rest encryption

2017-06-18 Thread Tomas Vondra
Hi all, I've noticed this thread got resurrected a few days ago, but I haven't managed to read all the messages until today. I do have a bunch of comments, but let me share them as a single consistent message instead of sending a thousand responses to individual messages. 1) Threat model

Re: [HACKERS] WIP: Data at rest encryption

2017-06-16 Thread Bruce Momjian
On Fri, Jun 16, 2017 at 11:06:39AM +0300, Konstantin Knizhnik wrote: > Encryption is much easier to implement than compression, because it is not > changing page size. So I do not see any "complexity and flexibility > challenges" here. > Just for reference I attached to this mail our own

Re: [HACKERS] WIP: Data at rest encryption

2017-06-16 Thread Bruce Momjian
On Thu, Jun 15, 2017 at 08:08:05PM -0400, Bruce Momjian wrote: > On Thu, Jun 15, 2017 at 04:56:36PM -0700, Andres Freund wrote: > > how few concerns about this feature's complexity / maintainability > > impact have been raised. > > Yeah, I guess we will just have to wait to see it since other

Re: [HACKERS] WIP: Data at rest encryption

2017-06-16 Thread Konstantin Knizhnik
On 16.06.2017 03:08, Bruce Momjian wrote: Yeah, I guess we will just have to wait to see it since other people are excited about it. My concern is code complexity and usability challenges, vs punting the problem to the operating system, though admittedly there are some cases where that is

Re: [HACKERS] WIP: Data at rest encryption

2017-06-15 Thread Tatsuo Ishii
> Yeah, I guess we will just have to wait to see it since other people are > excited about it. My concern is code complexity and usability > challenges, vs punting the problem to the operating system, though > admittedly there are some cases where that is not possible. Oracle sells this feature

Re: [HACKERS] WIP: Data at rest encryption

2017-06-15 Thread Bruce Momjian
On Thu, Jun 15, 2017 at 04:56:36PM -0700, Andres Freund wrote: > On 2017-06-15 19:44:43 -0400, Bruce Momjian wrote: > > Understood, but now you are promoting a feature with an admittedly-poor > > API, duplication of an OS feature, and perhaps an invasive change to the > > code. > > *Perhaps* an

Re: [HACKERS] WIP: Data at rest encryption

2017-06-15 Thread Andres Freund
On 2017-06-15 19:44:43 -0400, Bruce Momjian wrote: > Understood, but now you are promoting a feature with an admittedly-poor > API, duplication of an OS feature, and perhaps an invasive change to the > code. *Perhaps* an invasive change to the code? To me it's pretty evident that this'll be a

Re: [HACKERS] WIP: Data at rest encryption

2017-06-15 Thread Bruce Momjian
On Thu, Jun 15, 2017 at 07:51:36PM -0400, Alvaro Herrera wrote: > Bruce Momjian wrote: > > On Thu, Jun 15, 2017 at 07:27:55PM -0400, Stephen Frost wrote: > > > I expect the same would happen with the shell-command approach suggested > > > up-thread and the prompt-on-stdin approach too, they aren't

Re: [HACKERS] WIP: Data at rest encryption

2017-06-15 Thread Stephen Frost
Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Thu, Jun 15, 2017 at 07:27:55PM -0400, Stephen Frost wrote: > > I expect the same would happen with the shell-command approach suggested > > up-thread and the prompt-on-stdin approach too, they aren't great but I > > expect users would still

Re: [HACKERS] WIP: Data at rest encryption

2017-06-15 Thread Alvaro Herrera
Bruce Momjian wrote: > On Thu, Jun 15, 2017 at 07:27:55PM -0400, Stephen Frost wrote: > > I expect the same would happen with the shell-command approach suggested > > up-thread and the prompt-on-stdin approach too, they aren't great but I > > expect users would still use the feature. As Robert

Re: [HACKERS] WIP: Data at rest encryption

2017-06-15 Thread Bruce Momjian
On Thu, Jun 15, 2017 at 07:27:55PM -0400, Stephen Frost wrote: > I expect the same would happen with the shell-command approach suggested > up-thread and the prompt-on-stdin approach too, they aren't great but I > expect users would still use the feature. As Robert and I have > mentioned, there

Re: [HACKERS] WIP: Data at rest encryption

2017-06-15 Thread Stephen Frost
Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Thu, Jun 15, 2017 at 06:41:08PM -0400, Stephen Frost wrote: > > > > > One serious difference between in-database-encryption and SSH keys is > > > > > that the use of passwords for SSH is well understood and reasonable to > > > > > use, while I

Re: [HACKERS] WIP: Data at rest encryption

2017-06-15 Thread Bruce Momjian
On Thu, Jun 15, 2017 at 06:41:08PM -0400, Stephen Frost wrote: > > > > One serious difference between in-database-encryption and SSH keys is > > > > that the use of passwords for SSH is well understood and reasonable to > > > > use, while I think we all admit that use of passwords for database > >

Re: [HACKERS] WIP: Data at rest encryption

2017-06-15 Thread Stephen Frost
Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Thu, Jun 15, 2017 at 05:04:17PM -0400, Robert Haas wrote: > > > Also, there is the sense that security requires > > > trust of the root user, while using Postgres doesn't require the root > > > user to also use Postgres. > > > > I don't

Re: [HACKERS] WIP: Data at rest encryption

2017-06-15 Thread Bruce Momjian
On Thu, Jun 15, 2017 at 05:04:17PM -0400, Robert Haas wrote: > > Also, there is the sense that security requires > > trust of the root user, while using Postgres doesn't require the root > > user to also use Postgres. > > I don't understand this. It is certainly true that you're running >

Re: [HACKERS] WIP: Data at rest encryption

2017-06-15 Thread Robert Haas
On Thu, Jun 15, 2017 at 4:29 PM, Bruce Momjian wrote: > I think the big win for having OS features in the database is > selectivity --- the ability to selectively apply a feature to part of > the database. This is what you are doing by putting a password on your > SSH key, and

Re: [HACKERS] WIP: Data at rest encryption

2017-06-15 Thread Bruce Momjian
On Thu, Jun 15, 2017 at 03:09:32PM -0400, Robert Haas wrote: > To be honest, I find the hostility toward this feature a bit baffling. > The argument seems to be essentially that we shouldn't have this > feature because we'd have to maintain the code and many of the same > goals could be

Re: [HACKERS] WIP: Data at rest encryption

2017-06-15 Thread Robert Haas
On Thu, Jun 15, 2017 at 12:06 PM, Peter Eisentraut wrote: > Making this work well would be a major part of the usability story that > this is being sold on. If the proposed solution is that you can cobble > together a few bits of shell, then not only is that not

Re: [HACKERS] WIP: Data at rest encryption

2017-06-15 Thread Peter Eisentraut
On 6/14/17 17:41, Stephen Frost wrote: >> Relying on environment variables is clearly pretty crappy. So if that's >> the proposal, then I think it needs to be better. > I don't believe that was ever intended to be the final solution, I was > just pointing out that it's what the WIP patch did. >

Re: [HACKERS] WIP: Data at rest encryption

2017-06-15 Thread Robert Haas
On Wed, Jun 14, 2017 at 5:41 PM, Stephen Frost wrote: > I don't believe that was ever intended to be the final solution, I was > just pointing out that it's what the WIP patch did. > > The discussion had moved into having a command called which provided the > key on stdout, as

Re: [HACKERS] WIP: Data at rest encryption

2017-06-14 Thread Stephen Frost
Peter, * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > On 6/13/17 18:11, Stephen Frost wrote: > >> Let's see a proposal in those terms then. How easy can you make it, > >> compared to existing OS-level solutions, and will that justify the > >> maintenance overhead? > > From the

Re: [HACKERS] WIP: Data at rest encryption

2017-06-14 Thread Peter Eisentraut
On 6/13/17 18:11, Stephen Frost wrote: >> Let's see a proposal in those terms then. How easy can you make it, >> compared to existing OS-level solutions, and will that justify the >> maintenance overhead? > From the original post on this thread, which included a WIP patch: > >

Re: [HACKERS] WIP: Data at rest encryption

2017-06-14 Thread Peter Eisentraut
On 6/14/17 05:04, Aleksander Alekseev wrote: > A few companies that hired system administrators that are too > lazy to read two or three man pages is not a reason to re-implement file > system encryption (or compression, or mirroring if that matters) in any > open source RDBMS. To be fair, we did

Re: [HACKERS] WIP: Data at rest encryption

2017-06-14 Thread Bruce Momjian
On Wed, Jun 14, 2017 at 06:41:43PM +0300, Ants Aasma wrote: > On Wed, Jun 14, 2017 at 6:26 PM, Bruce Momjian wrote: > > Are you checking the CPU type or if AES instructions are enabled on the > > CPU? I ask this because I just realized in researching my new TLS talk > > that my

Re: [HACKERS] WIP: Data at rest encryption

2017-06-14 Thread Ants Aasma
On Wed, Jun 14, 2017 at 6:26 PM, Bruce Momjian wrote: > Are you checking the CPU type or if AES instructions are enabled on the > CPU? I ask this because I just realized in researching my new TLS talk > that my BIOS defaults to AES instructions disabled, and I had to > manually

Re: [HACKERS] WIP: Data at rest encryption

2017-06-14 Thread Bruce Momjian
On Wed, Jun 14, 2017 at 06:10:32PM +0300, Ants Aasma wrote: > On Tue, Jun 13, 2017 at 6:35 PM, Robert Haas wrote: > > Performance is likely to be poor on large databases, > > because every time a page transits between shared_buffers and the > > buffer cache we've got to

Re: [HACKERS] WIP: Data at rest encryption

2017-06-14 Thread Ants Aasma
On Tue, Jun 13, 2017 at 6:35 PM, Robert Haas wrote: > Of course, what would be even more useful is fine-grained encryption - > encrypt these tables (and the corresponding indexes, toast tables, and > WAL records related to any of that) with this key, encrypt these other >

Re: [HACKERS] WIP: Data at rest encryption

2017-06-14 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 06:29:20PM -0400, Stephen Frost wrote: > > Isn't the leakage controlled by OS permissions, so is it really leakage, > > i.e., if you can see the leakage, you probably have bypassed the OS > > permissions and see the key and data anyway. > > The case I'm mainly considering

Re: [HACKERS] WIP: Data at rest encryption

2017-06-14 Thread Bruce Momjian
On Wed, Jun 14, 2017 at 04:13:57PM +0300, Aleksander Alekseev wrote: > > > While I agree that configuring full disk encryption is not technically > > > difficult, it requires much more privileged access to the system and > > > basically requires the support of a system administrator. In addition,

Re: [HACKERS] WIP: Data at rest encryption

2017-06-14 Thread Aleksander Alekseev
> > While I agree that configuring full disk encryption is not technically > > difficult, it requires much more privileged access to the system and > > basically requires the support of a system administrator. In addition, > > if a volume is not available for encryption, PostgreSQL support for > >

Re: [HACKERS] WIP: Data at rest encryption

2017-06-14 Thread Aleksander Alekseev
Hi Kenneth, > > > File system encryption already exists and is well-tested. I don't see > > > any big advantages in re-implementing all of this one level up. You > > > would have to touch every single place in PostgreSQL backend and tool > > > code where a file is being read or written. Yikes.

Re: [HACKERS] WIP: Data at rest encryption

2017-06-14 Thread Kenneth Marshall
On Wed, Jun 14, 2017 at 12:04:26PM +0300, Aleksander Alekseev wrote: > Hi Ants, > > On Tue, Jun 13, 2017 at 09:07:49AM -0400, Peter Eisentraut wrote: > > On 6/12/17 17:11, Ants Aasma wrote: > > > I'm curious if the community thinks this is a feature worth having? > > > Even considering that

Re: [HACKERS] WIP: Data at rest encryption

2017-06-14 Thread Aleksander Alekseev
Hi Ants, On Tue, Jun 13, 2017 at 09:07:49AM -0400, Peter Eisentraut wrote: > On 6/12/17 17:11, Ants Aasma wrote: > > I'm curious if the community thinks this is a feature worth having? > > Even considering that security experts would classify this kind of > > encryption as a checkbox feature. >

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jun 13, 2017 at 03:20:12PM -0400, Stephen Frost wrote: > > > OK, so let's go back. You are saying there are no security benefits to > > > this vs. file system encryption. > > > > I'm not sure that I can see any, myself.. Perhaps I'm

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Peter, * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > On 6/13/17 15:20, Stephen Frost wrote: > > And then you would need openssl on the other system to decrypt it. > > Or make the USB file system encrypted as well? If you're in that kind > of environment, that would surely be

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Peter, * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > On 6/13/17 15:20, Stephen Frost wrote: > > No, the benefit is that the database administrator can configure it and > > set it up and not have to get an OS-level administrator involved. There > > may also be other reasons why

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread David Fetter
On Tue, Jun 13, 2017 at 10:28:14AM -0400, Peter Eisentraut wrote: > On 6/13/17 09:24, Stephen Frost wrote: > > but there are use-cases where it'd be really nice to be able to > > have PG doing the encryption instead of the filesystem because > > then you can do things like backup the database,

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 04:08:29PM -0400, Peter Eisentraut wrote: > On 6/13/17 15:51, Bruce Momjian wrote: > > Isn't the leakage controlled by OS permissions, so is it really leakage, > > i.e., if you can see the leakage, you probably have bypassed the OS > > permissions and see the key and data

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Peter Eisentraut
On 6/13/17 15:51, Bruce Momjian wrote: > Isn't the leakage controlled by OS permissions, so is it really leakage, > i.e., if you can see the leakage, you probably have bypassed the OS > permissions and see the key and data anyway. One scenario (among many) is when you're done with the disk. If

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 03:20:12PM -0400, Stephen Frost wrote: > Bruce, > > * Bruce Momjian (br...@momjian.us) wrote: > > On Tue, Jun 13, 2017 at 02:38:58PM -0400, Stephen Frost wrote: > > > It's good to discuss what the feature would bring and what cases it > > > doesn't cover, as well as

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Peter Eisentraut
On 6/13/17 15:20, Stephen Frost wrote: > For example, you could simply do: > > cp -a /path/to/PG /mnt/usb > > and you're done. If you're using filesystem level encryption then you'd > have to re-encrypt the data, using something like: > > tar -cf - /path/to/PG | openssl -key private.key > >

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Peter Eisentraut
On 6/13/17 15:20, Stephen Frost wrote: > No, the benefit is that the database administrator can configure it and > set it up and not have to get an OS-level administrator involved. There > may also be other reasons why filesystem-level encryption is difficult > to set up or use in a certain

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jun 13, 2017 at 02:38:58PM -0400, Stephen Frost wrote: > > It's good to discuss what the feature would bring and what cases it > > doesn't cover, as well as discussing how it can be designed to make sure > > that later improvements are

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 02:38:58PM -0400, Stephen Frost wrote: > It's good to discuss what the feature would bring and what cases it > doesn't cover, as well as discussing how it can be designed to make sure > that later improvements are able to be done without having to change it > around. I do

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jun 13, 2017 at 02:23:39PM -0400, Stephen Frost wrote: > > I'm not trying to shut down discussion, I'm simply pointing out where > > this feature will be helpful and where it won't be. If there's a way to > > make it better and able to

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 02:23:39PM -0400, Stephen Frost wrote: > I'm not trying to shut down discussion, I'm simply pointing out where > this feature will be helpful and where it won't be. If there's a way to > make it better and able to address an attack where the OS permission > system is

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jun 13, 2017 at 01:25:00PM -0400, Stephen Frost wrote: > > > I think the big win of Postgres doing the encryption is that the > > > user-visible file system is no longer a target (assuming OS permissions > > > are bypassed), while for

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jun 13, 2017 at 01:44:51PM -0400, Stephen Frost wrote: > > Just to be clear, I don't have any issue with discussing the idea that > > we want to get to a point where we can work with multiple keys and > > encrypt different tables with

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 01:44:51PM -0400, Stephen Frost wrote: > Just to be clear, I don't have any issue with discussing the idea that > we want to get to a point where we can work with multiple keys and > encrypt different tables with different keys (or not encrypt certain > tables, et al) with

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Joe, * Joe Conway (m...@joeconway.com) wrote: > On 06/13/2017 10:20 AM, Stephen Frost wrote: > > * Joe Conway (m...@joeconway.com) wrote: > >> Except shell escaping issues, etc, etc > > > > That's not an issue- we're talking about reading the stdout of some > > other process, there's no shell

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 01:25:00PM -0400, Stephen Frost wrote: > > I think the big win of Postgres doing the encryption is that the > > user-visible file system is no longer a target (assuming OS permissions > > are bypassed), while for file system encryption it is the storage device > > that is

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Joe Conway
On 06/13/2017 10:20 AM, Stephen Frost wrote: > * Joe Conway (m...@joeconway.com) wrote: >> Except shell escaping issues, etc, etc > > That's not an issue- we're talking about reading the stdout of some > other process, there's no shell escaping that has to be done there. It could be an issue

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jun 13, 2017 at 01:01:32PM -0400, Stephen Frost wrote: > > > Well, usually the symetric key is stored using RSA and a symetric > > > cipher is used to encrypt/decrypt the data. I was thinking of a case > > > where you encrypt a row using

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Joe, * Joe Conway (m...@joeconway.com) wrote: > Except shell escaping issues, etc, etc That's not an issue- we're talking about reading the stdout of some other process, there's no shell escaping that has to be done there. > > Let us, please, stop stressing over the right way to do key

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 01:01:32PM -0400, Stephen Frost wrote: > > Well, usually the symetric key is stored using RSA and a symetric > > cipher is used to encrypt/decrypt the data. I was thinking of a case > > where you encrypt a row using a symetric key, then store RSA-encrypted > > versions of

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Joe Conway
On 06/13/2017 10:05 AM, Stephen Frost wrote: > Bruce, Joe, > > * Bruce Momjian (br...@momjian.us) wrote: >> On Tue, Jun 13, 2017 at 09:55:10AM -0700, Joe Conway wrote: >> > > That way, if the user wants to store the key in an unencrypted text >> > > file, they can set the encryption_key_command =

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Bruce, Joe, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jun 13, 2017 at 09:55:10AM -0700, Joe Conway wrote: > > > That way, if the user wants to store the key in an unencrypted text > > > file, they can set the encryption_key_command = 'cat /not/very/secure' > > > and call it a day. If

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jun 13, 2017 at 12:23:01PM -0400, Stephen Frost wrote: > > > Of course, if the > > > key stored in the database is visible to someone using the operating > > > system, we really haven't added much/any security --- I guess my point > > >

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 09:55:10AM -0700, Joe Conway wrote: > > That way, if the user wants to store the key in an unencrypted text > > file, they can set the encryption_key_command = 'cat /not/very/secure' > > and call it a day. If they want to prompt the user on the console or > > request the

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Joe Conway
On 06/13/2017 09:28 AM, Robert Haas wrote: > On Tue, Jun 13, 2017 at 12:23 PM, Stephen Frost wrote: >> Key management is an entirely independent discussion from this and the >> proposal from Ants, as I understand it, is that the key would *not* be >> in the database but could

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 12:23:01PM -0400, Stephen Frost wrote: > > As I understand it, having encryption in the database means the key is > > stored in the database, while having encryption in the file system means > > the key is stored in the operating system somewhere. > > Key management is

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Robert Haas
On Tue, Jun 13, 2017 at 12:23 PM, Stephen Frost wrote: > Key management is an entirely independent discussion from this and the > proposal from Ants, as I understand it, is that the key would *not* be > in the database but could be anywhere that a shell command could get it >

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jun 13, 2017 at 11:04:21AM -0400, Stephen Frost wrote: > > > Also, in the use case you describe, if you use pg_basebackup to make a > > > direct encrypted copy of a data directory, I think that would mean you'd > > > have to keep using

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 11:04:21AM -0400, Stephen Frost wrote: > > Also, in the use case you describe, if you use pg_basebackup to make a > > direct encrypted copy of a data directory, I think that would mean you'd > > have to keep using the same key for all copies. > > That's true, but that

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Bruce Momjian
On Tue, Jun 13, 2017 at 11:35:03AM -0400, Robert Haas wrote: > I anticipate that one of the trickier problems here will be handling > encryption of the write-ahead log. Suppose you encrypt WAL a block at > a time. In the current system, once you've written and flushed a > block, you can consider

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Robert Haas
On Mon, Jun 12, 2017 at 5:11 PM, Ants Aasma wrote: > Fundamentally there doesn't seem to be a big benefit of implementing > the encryption at PostgreSQL level instead of the filesystem. The > patch doesn't take any real advantage from the higher level knowledge > of the

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Peter, * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > I wonder what the proper extent of "encryption at rest" should be. If > you encrypt just on a file or block level, then someone looking at the > data directory or a backup can still learn a number of things about the > number

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Peter Eisentraut
On 6/13/17 09:24, Stephen Frost wrote: > but there are > use-cases where it'd be really nice to be able to have PG doing the > encryption instead of the filesystem because then you can do things like > backup the database, copy it somewhere else directly, and then restore > it using the regular PG

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Stephen Frost
Ants, all, * Ants Aasma (ants.aa...@eesti.ee) wrote: > Yes, the plan is to pick it up again, Real Soon Now(tm). There are a > couple of loose ends for stuff that should be encrypted, but in the > current state of the patch aren't yet (from the top of my head, > logical decoding and

Re: [HACKERS] WIP: Data at rest encryption

2017-06-13 Thread Peter Eisentraut
On 6/12/17 17:11, Ants Aasma wrote: > I'm curious if the community thinks this is a feature worth having? > Even considering that security experts would classify this kind of > encryption as a checkbox feature. File system encryption already exists and is well-tested. I don't see any big

Re: [HACKERS] WIP: Data at rest encryption

2017-06-12 Thread Ants Aasma
On Mon, Jun 12, 2017 at 10:38 PM, Robert Haas wrote: > On Mon, Jun 13, 2016 at 11:07 AM, Peter Eisentraut > wrote: >> On 6/7/16 9:56 AM, Ants Aasma wrote: >>> >>> Similar things can be achieved with filesystem level encryption. >>> However

Re: [HACKERS] WIP: Data at rest encryption

2017-06-12 Thread Robert Haas
On Mon, Jun 13, 2016 at 11:07 AM, Peter Eisentraut wrote: > On 6/7/16 9:56 AM, Ants Aasma wrote: >> >> Similar things can be achieved with filesystem level encryption. >> However this is not always optimal for various reasons. One of the >> better reasons is the

Re: [HACKERS] WIP: Data at rest encryption

2016-06-15 Thread PostgreSQL - Hans-Jürgen Schönig
On 06/14/2016 09:59 PM, Jim Nasby wrote: On 6/12/16 2:13 AM, Ants Aasma wrote: On Fri, Jun 10, 2016 at 5:23 AM, Haribabu Kommi wrote: > 1. Instead of doing the entire database files encryption, how about > providing user an option to protect only some particular

Re: [HACKERS] WIP: Data at rest encryption

2016-06-14 Thread Jim Nasby
On 6/12/16 2:13 AM, Ants Aasma wrote: On Fri, Jun 10, 2016 at 5:23 AM, Haribabu Kommi wrote: > 1. Instead of doing the entire database files encryption, how about > providing user an option to protect only some particular tables that > wants the encryption at

Re: [HACKERS] WIP: Data at rest encryption

2016-06-13 Thread Haribabu Kommi
On Sun, Jun 12, 2016 at 5:13 PM, Ants Aasma wrote: > On Fri, Jun 10, 2016 at 5:23 AM, Haribabu Kommi > wrote: > >> 2. Instead of depending on a contrib module for the encryption, how >> about integrating pgcrypto contrib in to the core and add that

Re: [HACKERS] WIP: Data at rest encryption

2016-06-13 Thread Peter Eisentraut
On 6/12/16 3:13 AM, Ants Aasma wrote: 5. Instead of providing passphrase through environmental variable, > better to provide some options to pg_ctl etc. That looks like it would be worse from a security perspective. Integrating a passphrase prompt would be an option, but a way for scripts to

Re: [HACKERS] WIP: Data at rest encryption

2016-06-13 Thread Peter Eisentraut
On 6/7/16 9:56 AM, Ants Aasma wrote: Similar things can be achieved with filesystem level encryption. However this is not always optimal for various reasons. One of the better reasons is the desire for HSM based encryption in a storage area network based setup. Could you explain this in more

Re: [HACKERS] WIP: Data at rest encryption

2016-06-13 Thread Ants Aasma
On Mon, Jun 13, 2016 at 5:17 AM, Michael Paquier wrote: > On Sun, Jun 12, 2016 at 4:13 PM, Ants Aasma wrote: >>> I feel separate file is better to include the key data instead of pg_control >>> file. >> >> I guess that would be more flexible.

Re: [HACKERS] WIP: Data at rest encryption

2016-06-12 Thread Michael Paquier
On Sun, Jun 12, 2016 at 4:13 PM, Ants Aasma wrote: >> I feel separate file is better to include the key data instead of pg_control >> file. > > I guess that would be more flexible. However I think at least the fact > that the database is encrypted should remain in the

Re: [HACKERS] WIP: Data at rest encryption

2016-06-12 Thread Ants Aasma
On Fri, Jun 10, 2016 at 5:23 AM, Haribabu Kommi wrote: > 1. Instead of doing the entire database files encryption, how about > providing user an option to protect only some particular tables that > wants the encryption at table/tablespace level. This not only provides >

Re: [HACKERS] WIP: Data at rest encryption

2016-06-09 Thread Haribabu Kommi
On Tue, Jun 7, 2016 at 11:56 PM, Ants Aasma wrote: > Hi all, > > I have been working on data-at-rest encryption support for PostgreSQL. > In my experience this is a common request that customers make. The > short of the feature is that all PostgreSQL data files are encrypted

[HACKERS] WIP: Data at rest encryption

2016-06-07 Thread Ants Aasma
Hi all, I have been working on data-at-rest encryption support for PostgreSQL. In my experience this is a common request that customers make. The short of the feature is that all PostgreSQL data files are encrypted with a single master key and are decrypted when read from the OS. It does not