Re: XTS cipher mode for cluster file encryption

2022-02-02 Thread Sasasu
On 2022/2/2 01:50, Bruce Momjian wrote: > Well, I sent an email a week ago asking if people want to advance this > feature forward, and so far you are the only person to reply, which I > think means there isn't enough interest in this feature to advance it. I am still focus on this thread. and I

Re: XTS cipher mode for cluster file encryption

2022-02-01 Thread Michael Banck
Hi, On Tue, Feb 01, 2022 at 01:07:36PM -0500, Stephen Frost wrote: > On Tue, Feb 1, 2022 at 12:50 Bruce Momjian wrote: > > On Tue, Feb 1, 2022 at 07:45:06AM +0100, Antonin Houska wrote: > > > > With pg_upgrade modified to preserve the relfilenode, tablespace > > > > oid, and database oid, we

Re: XTS cipher mode for cluster file encryption

2022-02-01 Thread Andres Freund
Hi, On 2022-02-01 13:27:03 -0500, Bruce Momjian wrote: > On Tue, Feb 1, 2022 at 01:07:36PM -0500, Stephen Frost wrote: > > Well, I sent an email a week ago asking if people want to advance this > > feature forward, and so far you are the only person to reply, which I > > think means

Re: XTS cipher mode for cluster file encryption

2022-02-01 Thread Bruce Momjian
On Tue, Feb 1, 2022 at 01:07:36PM -0500, Stephen Frost wrote: > Well, I sent an email a week ago asking if people want to advance this > feature forward, and so far you are the only person to reply, which I > think means there isn't enough interest in this feature to advance it. > >

Re: XTS cipher mode for cluster file encryption

2022-02-01 Thread Stephen Frost
Greetings, On Tue, Feb 1, 2022 at 12:50 Bruce Momjian wrote: > On Tue, Feb 1, 2022 at 07:45:06AM +0100, Antonin Houska wrote: > > > With pg_upgrade modified to preserve the relfilenode, tablespace oid, > and > > > database oid, we are now closer to implementing cluster file encryption > > >

Re: XTS cipher mode for cluster file encryption

2022-02-01 Thread Bruce Momjian
On Tue, Feb 1, 2022 at 07:45:06AM +0100, Antonin Houska wrote: > > With pg_upgrade modified to preserve the relfilenode, tablespace oid, and > > database oid, we are now closer to implementing cluster file encryption > > using XTS. I think we have a few steps left: > > > > 1. modify temporary

Re: XTS cipher mode for cluster file encryption

2022-01-31 Thread Antonin Houska
Bruce Momjian wrote: > On Mon, Nov 29, 2021 at 08:37:31AM +0100, Antonin Houska wrote: > > The changes to buffile.c are not trivial, but we haven't really changed the > > API, as long as you mean BufFileCreateTemp(), BufFileWrite(), BufFileRead(). > > > > What our patch affects on the caller

Re: XTS cipher mode for cluster file encryption

2022-01-24 Thread Bruce Momjian
On Mon, Nov 29, 2021 at 08:37:31AM +0100, Antonin Houska wrote: > The changes to buffile.c are not trivial, but we haven't really changed the > API, as long as you mean BufFileCreateTemp(), BufFileWrite(), BufFileRead(). > > What our patch affects on the caller side is that

Re: XTS cipher mode for cluster file encryption

2021-11-28 Thread Antonin Houska
Stephen Frost wrote: > Greetings, > > * Bruce Momjian (br...@momjian.us) wrote: > > On Tue, Oct 19, 2021 at 02:54:56PM -0400, Stephen Frost wrote: > > > * Sasasu (i...@sasa.su) wrote: > > > > A unified block-based I/O API sounds great. Has anyone tried to do this > > > > before? It would be

Re: XTS cipher mode for cluster file encryption

2021-11-12 Thread Bruce Momjian
On Mon, Nov 1, 2021 at 02:24:36PM -0400, Stephen Frost wrote: > I can understand the general idea that we should be sure to engineer > this in a way that multiple methods can be used, as surely one day folks > will say that AES128 isn't acceptable any more. In terms of what we'll > do from the

Re: XTS cipher mode for cluster file encryption

2021-11-03 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Mon, Nov 1, 2021 at 02:24:36PM -0400, Stephen Frost wrote: > > I can understand the general idea that we should be sure to engineer > > this in a way that multiple methods can be used, as surely one day folks > > will say that AES128

Re: XTS cipher mode for cluster file encryption

2021-11-02 Thread Bruce Momjian
On Mon, Nov 1, 2021 at 02:24:36PM -0400, Stephen Frost wrote: > I can understand the general idea that we should be sure to engineer > this in a way that multiple methods can be used, as surely one day folks > will say that AES128 isn't acceptable any more. In terms of what we'll > do from the

Re: XTS cipher mode for cluster file encryption

2021-11-02 Thread Sasasu
On 2021/11/2 02:24, Stephen Frost wrote: I can understand the general idea that we should be sure to engineer this in a way that multiple methods can be used, as surely one day folks will say that AES128 isn't acceptable any more. Cheers! OpenPGP_0x4E72AF09097DAE2E.asc Description: OpenPGP

Re: XTS cipher mode for cluster file encryption

2021-11-01 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Oct 26, 2021 at 11:11:39PM +0200, Tomas Vondra wrote: > > BTW I'm not sure what the existing patches do, but I wonder if we should > > calculate the checksum before or after encryption. I'd say it should be > > after encryption,

Re: XTS cipher mode for cluster file encryption

2021-10-26 Thread Bruce Momjian
On Tue, Oct 26, 2021 at 11:11:39PM +0200, Tomas Vondra wrote: > BTW I'm not sure what the existing patches do, but I wonder if we should > calculate the checksum before or after encryption. I'd say it should be > after encryption, because checksums were meant as a protection against > issues at

Re: XTS cipher mode for cluster file encryption

2021-10-26 Thread Tomas Vondra
On 10/26/21 21:43, Stephen Frost wrote: Greetings, * Yura Sokolov (y.soko...@postgrespro.ru) wrote: ... >> Integrity could be based on simple non-cryptographic checksum, and it could be checked after decryption. It would be imposible to intentionally change encrypted page in a way it will

Re: XTS cipher mode for cluster file encryption

2021-10-26 Thread Stephen Frost
Greetings, * Yura Sokolov (y.soko...@postgrespro.ru) wrote: > В Пн, 25/10/2021 в 12:12 -0400, Stephen Frost пишет: > > * Yura Sokolov (y.soko...@postgrespro.ru) wrote: > > > В Чт, 21/10/2021 в 13:28 -0400, Stephen Frost пишет: > > > > I really don't think this is necessary. Similar to

Re: XTS cipher mode for cluster file encryption

2021-10-26 Thread Sasasu
On 2021/10/26 17:33, Yura Sokolov wrote: Yes, AES with AES-NI is fastest. But not so much. And, AES-CTR could be easily used instead of ChaCha12 in Adiantum. Adiantum uses ChaCha12 as a stream cipher, and any other stream cipher will be ok as well with minor modifications to algorithm. not so

Re: XTS cipher mode for cluster file encryption

2021-10-26 Thread Yura Sokolov
В Вт, 26/10/2021 в 11:08 +0800, Sasasu пишет: > On 2021/10/26 04:32, Yura Sokolov wrote: > > And among others Adiantum looks best: it is fast even without hardware > > acceleration, > > No, AES is fast on modern high-end hardware. > > on X86 AMD 3700X > type 1024 bytes 8192 bytes

Re: XTS cipher mode for cluster file encryption

2021-10-25 Thread Sasasu
On 2021/10/26 04:32, Yura Sokolov wrote: And among others Adiantum looks best: it is fast even without hardware acceleration, No, AES is fast on modern high-end hardware. on X86 AMD 3700X type 1024 bytes 8192 bytes 16384 bytes aes-128-ctr 8963982.50k 11124613.88k

Re: XTS cipher mode for cluster file encryption

2021-10-25 Thread Yura Sokolov
В Пн, 25/10/2021 в 12:12 -0400, Stephen Frost пишет: > Greetings, > > * Yura Sokolov (y.soko...@postgrespro.ru) wrote: > > В Чт, 21/10/2021 в 13:28 -0400, Stephen Frost пишет: > > > I really don't think this is necessary. Similar to PageSetChecksumCopy > > > and PageSetChecksumInplace, I'm sure

Re: XTS cipher mode for cluster file encryption

2021-10-25 Thread Bruce Momjian
On Mon, Oct 25, 2021 at 11:58:14AM -0400, Stephen Frost wrote: > As for the specific encryption method to use, using CTR would be simpler > as it doesn't require access to be block-based, though we would need to > make sure to not re-use the IV across any of the temporary files being > created

Re: XTS cipher mode for cluster file encryption

2021-10-25 Thread Stephen Frost
Greetings, * Yura Sokolov (y.soko...@postgrespro.ru) wrote: > В Чт, 21/10/2021 в 13:28 -0400, Stephen Frost пишет: > > I really don't think this is necessary. Similar to PageSetChecksumCopy > > and PageSetChecksumInplace, I'm sure we would have functions which are > > called in the appropriate

Re: XTS cipher mode for cluster file encryption

2021-10-25 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Oct 19, 2021 at 02:54:56PM -0400, Stephen Frost wrote: > > * Sasasu (i...@sasa.su) wrote: > > > A unified block-based I/O API sounds great. Has anyone tried to do this > > > before? It would be nice if the front-end tools could also

Re: XTS cipher mode for cluster file encryption

2021-10-25 Thread Yura Sokolov
В Чт, 21/10/2021 в 13:28 -0400, Stephen Frost пишет: > Greetings, > > I really don't think this is necessary. Similar to PageSetChecksumCopy > and PageSetChecksumInplace, I'm sure we would have functions which are > called in the appropriate spots to do encryption (such as 'encrypt_page' > and

Re: XTS cipher mode for cluster file encryption

2021-10-23 Thread Bruce Momjian
On Tue, Oct 19, 2021 at 02:54:56PM -0400, Stephen Frost wrote: > * Sasasu (i...@sasa.su) wrote: > > A unified block-based I/O API sounds great. Has anyone tried to do this > > before? It would be nice if the front-end tools could also use these API. > > The TDE patch from Cybertec did go down

Re: XTS cipher mode for cluster file encryption

2021-10-23 Thread Bruce Momjian
On Mon, Oct 18, 2021 at 12:37:39PM -0400, Robert Haas wrote: > I do really like the idea of using AES-GCM-SIV not because I know > anything about it, but because the integrity checking seems cool, and -- > storing the nonce seems like

Re: XTS cipher mode for cluster file encryption

2021-10-23 Thread Bruce Momjian
On Tue, Oct 19, 2021 at 02:44:26PM -0400, Stephen Frost wrote: > Another threat model to consider is if the attacker has read-only access > to the data directory through, say, unix group read privileges or maybe > the ability to monitor the traffic on the SAN, or the ability to > read-only mount

Re: XTS cipher mode for cluster file encryption

2021-10-22 Thread Bruce Momjian
On Fri, Oct 22, 2021 at 11:36:37AM -0400, Stephen Frost wrote: > > I am not re-discuss using CTR for heap table. I mean use some CTR-like > > algorithm *only* for WAL encryption. My idea is exactly the same when you > > are typing "we hopefully aren't going to write different WAL records at the >

Re: XTS cipher mode for cluster file encryption

2021-10-22 Thread Bruce Momjian
On Mon, Oct 18, 2021 at 12:37:39PM -0400, Robert Haas wrote: > > Thus, to me, it doesn't seem worth going down the XTS route, just to > > temporarily save a bit of implementation effort. We'll have to endure that > > pain anyway. > > I agree up to a point, but I do also kind of feel like we

Re: XTS cipher mode for cluster file encryption

2021-10-22 Thread Bruce Momjian
On Tue, Oct 19, 2021 at 02:44:26PM -0400, Stephen Frost wrote: > There are ways around it. There likely always will be. We need to be > clear about what it provides and what it doesn't. We need to stop > telling ourselves that the only answer is a 100% solution and therefore > it's impossible

Re: XTS cipher mode for cluster file encryption

2021-10-22 Thread Bruce Momjian
On Mon, Oct 18, 2021 at 11:56:03AM -0400, Stephen Frost wrote: > Greetings, > > * Tomas Vondra (tomas.von...@enterprisedb.com) wrote: > > On 10/15/21 21:22, Stephen Frost wrote: > > >Now, to address the concern around re-encrypting a block with the same > > >key+IV but different data and leaking

Re: XTS cipher mode for cluster file encryption

2021-10-22 Thread Stephen Frost
Greetings, * Sasasu (i...@sasa.su) wrote: > On 2021/10/22 01:28, Stephen Frost wrote: > >None of these are actually working with or changing the data though, > >they're just copying it. I don't think we'd actually want these to > >decrypt and reencrypt the data. > > OK, but why ALTER TABLE SET

Re: XTS cipher mode for cluster file encryption

2021-10-21 Thread Sasasu
On 2021/10/22 01:28, Stephen Frost wrote: None of these are actually working with or changing the data though, they're just copying it. I don't think we'd actually want these to decrypt and reencrypt the data. OK, but why ALTER TABLE SET TABLESPACE use smgrread() and smgrextend() instead of

Re: XTS cipher mode for cluster file encryption

2021-10-21 Thread Stephen Frost
Greetings, * Sasasu (i...@sasa.su) wrote: > On 2021/10/20 20:24, Stephen Frost wrote: > > PG does have a block-based IO API, it's just not exposed as hooks. In > > particular, take a look at md.c, though perhaps you'd be more interested > > in the higher level bufmgr.c routines. For the

Re: XTS cipher mode for cluster file encryption

2021-10-20 Thread Sasasu
On 2021/10/20 20:24, Stephen Frost wrote: > PG does have a block-based IO API, it's just not exposed as hooks. In > particular, take a look at md.c, though perhaps you'd be more interested > in the higher level bufmgr.c routines. For the specific places where > encryption may hook in, looking

Re: XTS cipher mode for cluster file encryption

2021-10-20 Thread Stephen Frost
Greetings, * Sasasu (i...@sasa.su) wrote: > But If PG has a clear block-based IO API, TDE is much easier to understand. PG does have a block-based IO API, it's just not exposed as hooks. In particular, take a look at md.c, though perhaps you'd be more interested in the higher level bufmgr.c

Re: XTS cipher mode for cluster file encryption

2021-10-20 Thread Sasasu
On 2021/10/20 02:54, Stephen Frost wrote: > I agree with Robert- using hooks for this really isn't realistic. OK, I agree use hooks is too invasive. Just a whim, never mind. But If PG has a clear block-based IO API, TDE is much easier to understand. security people may lack database knowledge

Re: XTS cipher mode for cluster file encryption

2021-10-19 Thread Stephen Frost
Greetings, * Sasasu (i...@sasa.su) wrote: > On 2021/10/19 00:37, Robert Haas wrote: > >I think what we ought to be doing at > >this point is combining our efforts to try to get some things > >committed which make future work in this area committed - like that > >patch to preserve relfilenode and

Re: XTS cipher mode for cluster file encryption

2021-10-19 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@enterprisedb.com) wrote: > On 10/18/21 17:56, Stephen Frost wrote: > >* Tomas Vondra (tomas.von...@enterprisedb.com) wrote: > >>On 10/15/21 21:22, Stephen Frost wrote: > >>>Now, to address the concern around re-encrypting a block with the same > >>>key+IV

Re: XTS cipher mode for cluster file encryption

2021-10-19 Thread Robert Haas
On Tue, Oct 19, 2021 at 11:46 AM Sasasu wrote: > As there are so many threat models, I propose to do the TDE feature by a > set of hooks. This is too invasive to do using hooks. We are inevitably going to need to make significant changes in core. -- Robert Haas EDB: http://www.enterprisedb.com

Re: XTS cipher mode for cluster file encryption

2021-10-19 Thread Sasasu
On 2021/10/19 00:37, Robert Haas wrote: I think what we ought to be doing at this point is combining our efforts to try to get some things committed which make future work in this area committed - like that patch to preserve relfilenode and database OID, or maybe some patches to drive all of our

Re: XTS cipher mode for cluster file encryption

2021-10-18 Thread Tomas Vondra
On 10/18/21 17:56, Stephen Frost wrote: >> ... I've argued for storing the nonce, but I don't quite see why would we need integrity guarantees? AFAICS the threat model the patch aims to address is an attacker who can observe the data (e.g. a low-privileged OS user), but can't modify the files.

Re: XTS cipher mode for cluster file encryption

2021-10-18 Thread Tomas Vondra
On 10/18/21 17:56, Stephen Frost wrote: Greetings, * Tomas Vondra (tomas.von...@enterprisedb.com) wrote: On 10/15/21 21:22, Stephen Frost wrote: Now, to address the concern around re-encrypting a block with the same key+IV but different data and leaking what parts of the page changed, I do

Re: XTS cipher mode for cluster file encryption

2021-10-18 Thread Tomas Vondra
On 10/18/21 04:19, Sasasu wrote: Just a mention. the HMAC (or AE/AD) can be disabled in AES-GCM. HMAC in AES-GCM is an encrypt-then-hash MAC. CRC-32 is not a crypto-safe hash (technically CRC-32 is not a hash function). Cryptographers may unhappy with CRC-32. True. If you can flip

Re: XTS cipher mode for cluster file encryption

2021-10-18 Thread Robert Haas
On Fri, Oct 15, 2021 at 5:21 PM Andres Freund wrote: > I don't find that line of argument *that* convincing. The reason XTS is the > de-facto standard is that for generic block layer encryption is that you can't > add additional data for each block without very significant overhead > (basically

Re: XTS cipher mode for cluster file encryption

2021-10-18 Thread Stephen Frost
Greetings, * Tomas Vondra (tomas.von...@enterprisedb.com) wrote: > On 10/15/21 21:22, Stephen Frost wrote: > >Now, to address the concern around re-encrypting a block with the same > >key+IV but different data and leaking what parts of the page changed, I > >do think we should use the LSN and

Re: XTS cipher mode for cluster file encryption

2021-10-17 Thread Sasasu
On 2021/10/16 04:57, Tomas Vondra wrote: > > Seems reasonable, on the assumption the threat models are the same. On 2021/10/16 03:22, Stephen Frost wrote: plain64: the initial vector is the 64-bit little-endian version of the sector number, padded with zeros if necessary That is, the default

Re: XTS cipher mode for cluster file encryption

2021-10-17 Thread Sasasu
Just a mention. the HMAC (or AE/AD) can be disabled in AES-GCM. HMAC in AES-GCM is an encrypt-then-hash MAC. CRC-32 is not a crypto-safe hash (technically CRC-32 is not a hash function). Cryptographers may unhappy with CRC-32. I think CRC or SHA is not such important. If IV can be stored, I

Re: XTS cipher mode for cluster file encryption

2021-10-17 Thread Tomas Vondra
On 10/16/21 18:28, Bruce Momjian wrote: On Sat, Oct 16, 2021 at 09:15:05AM -0700, Andres Freund wrote: Hi, On 2021-10-16 10:16:25 -0400, Bruce Momjian wrote: As a final comment to Andres's email, adding a GCM has the problems above, plus it wouldn't detect changes to pg_xact, fsm, vm, etc,

Re: XTS cipher mode for cluster file encryption

2021-10-17 Thread Tomas Vondra
On 10/16/21 16:16, Bruce Momjian wrote: On Fri, Oct 15, 2021 at 10:57:03PM +0200, Tomas Vondra wrote: That said, I don't think that's really a huge issue or something that's a show stopper or a reason to hold off on using XTS. Note that what those bits actually *are* isn't leaked, just that

Re: XTS cipher mode for cluster file encryption

2021-10-16 Thread Bruce Momjian
On Sat, Oct 16, 2021 at 09:15:05AM -0700, Andres Freund wrote: > Hi, > > On 2021-10-16 10:16:25 -0400, Bruce Momjian wrote: > > As a final comment to Andres's email, adding a GCM has the problems > > above, plus it wouldn't detect changes to pg_xact, fsm, vm, etc, which > > could also affect the

Re: XTS cipher mode for cluster file encryption

2021-10-16 Thread Andres Freund
Hi, On 2021-10-16 10:16:25 -0400, Bruce Momjian wrote: > As a final comment to Andres's email, adding a GCM has the problems > above, plus it wouldn't detect changes to pg_xact, fsm, vm, etc, which > could also affect the integrity of the data. Someone could also restore > and old copy of a

Re: XTS cipher mode for cluster file encryption

2021-10-16 Thread Bruce Momjian
On Fri, Oct 15, 2021 at 10:57:03PM +0200, Tomas Vondra wrote: > > That said, I don't think that's really a huge issue or something that's > > a show stopper or a reason to hold off on using XTS. Note that what > > those bits actually *are* isn't leaked, just that they changed in some > > fashion

Re: XTS cipher mode for cluster file encryption

2021-10-15 Thread Tomas Vondra
On 10/15/21 23:02, Robert Haas wrote: On Fri, Oct 15, 2021 at 3:22 PM Stephen Frost wrote: Specifically: The default cipher for LUKS is nowadays aes-xts-plain64 and then this: https://gitlab.com/cryptsetup/cryptsetup/-/wikis/DMCrypt where plain64 is defined as: plain64: the initial

Re: XTS cipher mode for cluster file encryption

2021-10-15 Thread Andres Freund
Hi, On 2021-10-15 15:22:48 -0400, Stephen Frost wrote: > * Bruce Momjian (br...@momjian.us) wrote: > > Finally, there is an interesting web page about when not to use XTS: > > > > https://sockpuppet.org/blog/2014/04/30/you-dont-want-xts/ > > This particular article always struck me as more

Re: XTS cipher mode for cluster file encryption

2021-10-15 Thread Robert Haas
On Fri, Oct 15, 2021 at 3:22 PM Stephen Frost wrote: > Specifically: The default cipher for LUKS is nowadays aes-xts-plain64 > > and then this: > > https://gitlab.com/cryptsetup/cryptsetup/-/wikis/DMCrypt > > where plain64 is defined as: > > plain64: the initial vector is the 64-bit little-endian

Re: XTS cipher mode for cluster file encryption

2021-10-15 Thread Tomas Vondra
On 10/15/21 21:22, Stephen Frost wrote: Greetings, * Bruce Momjian (br...@momjian.us) wrote: As you might have seen from my email in another thread, thanks to Stephen and Cybertec staff, I am back working on cluster file encryption/TDE. Stephen was going to research if XTS cipher mode

Re: XTS cipher mode for cluster file encryption

2021-10-15 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > As you might have seen from my email in another thread, thanks to > Stephen and Cybertec staff, I am back working on cluster file > encryption/TDE. > > Stephen was going to research if XTS cipher mode would be a good fit for > this since it

XTS cipher mode for cluster file encryption

2021-10-13 Thread Bruce Momjian
As you might have seen from my email in another thread, thanks to Stephen and Cybertec staff, I am back working on cluster file encryption/TDE. Stephen was going to research if XTS cipher mode would be a good fit for this since it was determined that CTR cipher mode was too vulnerable to IV