Re: Parallel copy

2020-02-26 Thread Alastair Turner
On Wed, 26 Feb 2020 at 10:54, Amit Kapila wrote: > > On Tue, Feb 25, 2020 at 9:30 PM Tomas Vondra > wrote: > > ... > > > > Perhaps. I guess it'll depend on the CSV file (number of fields, ...), > > so I still think we need to do some measurements first. > > > > Agreed. > > > I'm willing to > >

Re: Parallel copy

2020-02-14 Thread Alastair Turner
On Fri, 14 Feb 2020 at 11:57, Amit Kapila wrote: > On Fri, Feb 14, 2020 at 3:36 PM Thomas Munro > wrote: > > > > On Fri, Feb 14, 2020 at 9:12 PM Amit Kapila > wrote: ... > > > Another approach that came up during an offlist discussion with Robert > > > is that we have one dedicated worker

Re: Parallel copy

2020-02-15 Thread Alastair Turner
On Sat, 15 Feb 2020 at 04:55, Amit Kapila wrote: > > On Fri, Feb 14, 2020 at 7:16 PM Alastair Turner wrote: > > ... > > > > Parsing rows from the raw input (the work done by CopyReadLine()) in a > > single process would accommodate line returns in quoted fields. I

Re: Proposed patch for key managment

2020-12-19 Thread Alastair Turner
Hi Stephen On Fri, 18 Dec 2020 at 21:36, Stephen Frost wrote: > > Greetings Alastair, > > * Alastair Turner (min...@decodable.me) wrote: > > On Wed, 16 Dec 2020 at 22:43, Stephen Frost wrote: ... > > passphrase key wrapper, the secret store and the cloud/HW KMS. &g

Re: Proposed patch for key managment

2020-12-19 Thread Alastair Turner
Hi Bruce On Sat, 19 Dec 2020 at 02:38, Bruce Momjian wrote: > > I am not going be as kind. Our workflow is: > > Desirability -> Design -> Implement -> Test -> Review -> Commit > https://wiki.postgresql.org/wiki/Todo#Development_Process > > I have already asked about the first

Re: Proposed patch for key managment

2020-12-20 Thread Alastair Turner
Thanks Stephen, On Mon, 21 Dec 2020 at 00:33, Stephen Frost wrote: > > Greetings, > > * Alastair Turner (min...@decodable.me) wrote: ... > > > > What I'd like specifically is to have the option of an external > > keyring as a first class key store, where the k

Re: Proposed patch for key managment

2020-12-16 Thread Alastair Turner
Hi Bruce On Wed, 16 Dec 2020 at 00:12, Bruce Momjian wrote: > ... > > The second approach is to make a new API for what you want I am trying to motivate for an alternate API. Specifically, an API which allows any potential adopter of Postgres and Cluster File Encryption to adopt them

Re: Proposed patch for key managment

2020-12-15 Thread Alastair Turner
Hi Bruce et al Firstly, thanks for shaping the patch, getting it down to a manageable scope of cluster file encryption. I think this is a great feature and it matters to a lot of the customers I talk to at VMware about adopting Postgres. Since it's exciting stuff, I've been trying to lash

Re: Proposed patch for key managment

2020-12-16 Thread Alastair Turner
On Wed, 16 Dec 2020 at 21:32, Stephen Frost wrote: > > Greetings, > > * Alastair Turner (min...@decodable.me) wrote: > > On Wed, 16 Dec 2020 at 00:12, Bruce Momjian wrote: > > > The second approach is to make a new API for what you want > > > > I am

Re: Proposed patch for key managment

2020-12-16 Thread Alastair Turner
On Wed, 16 Dec 2020 at 22:43, Stephen Frost wrote: > > Greetings, ... > > If I'm following, you're suggesting something like: > > cluster_passphrase_command = 'aws get %q' > > and then '%q' gets replaced with "Please provide the WAL DEK: ", or > something like that? Prompting the user for each

Re: Proposed patch for key managment

2020-12-22 Thread Alastair Turner
Hi Bruce In ckey_passphrase.sh.sample + +echo "$PASS" | sha256sum | cut -d' ' -f1 + Under the threat model discussed, a copy of the keyfile could be attacked offline. So getting from passphrase to DEKs should be as resource intensive as possible to slow down brute-force attempts. Instead of

Re: Proposed patch for key managment

2020-12-20 Thread Alastair Turner
Thanks, Bruce On Sat, 19 Dec 2020 at 16:58, Bruce Momjian wrote: > ... > > To enable the direct injection of keys into the server, we would need a > new command for this, since trying to make the passphrase command do > this will lead to unnecessary complexity. The passphrase command should >

Re: Proposed patch for key management

2021-01-05 Thread Alastair Turner
Hi Bruce On Mon, 4 Jan 2021 at 18:23, Bruce Momjian wrote: > > On Fri, Jan 1, 2021 at 06:26:36PM +, Alastair Turner wrote: > > After the long intro, my question - If using a standard format, > > managed by a library, for the internal keystore does not result in a >

Re: Proposed patch for key management

2021-01-05 Thread Alastair Turner
On Mon, 4 Jan 2021 at 17:56, Bruce Momjian wrote: > > On Sat, Jan 2, 2021 at 12:47:19PM +, Alastair Turner wrote: > > > > There is also a further validation task - probably beyond the scope of > > the key management patch and into the encryption patch[es]

Re: Proposed patch for key management

2021-01-01 Thread Alastair Turner
s that a pg account could be > > > compromised, and the "master key" (if any, that is just one particular > > > cryptographic design) should not be accessible in that case. The first > > > barrier would be pg admin account. > > > > Let's unp

Re: Proposed patch for key management

2021-01-02 Thread Alastair Turner
Hi Fabien On Sat, 2 Jan 2021 at 09:50, Fabien COELHO wrote: > ... > ISTM that pg at the core level should (only) directly provide: > > (1) a per-file encryption scheme, with loadable (hook-replaceable > functions??) to manage pages, maybe: > >encrypt(page_id, *key, *clear_page,