Re: Temporary file access API

2022-10-11 Thread Michael Paquier
On Tue, Sep 27, 2022 at 04:22:39PM +, John Morris wrote: > I’m also wondering about code simplification vs a more general > encryption/compression framework. I started with the current > proposal, and I’m also looking at David Steele’s approach to > encryption/compression in pgbackrest. I’m

Re: Temporary file access API

2022-09-27 Thread John Morris
* So I think that code simplification and easy adoption of in-memory data changes (such as encryption or compression) are two rather distinct goals. admit that I'm running out of ideas how to develop a framework that'd be useful for both. I’m also wondering about code simplification vs a more

Re: Temporary file access API

2022-09-23 Thread Antonin Houska
be useful for both. [1] https://www.postgresql.org/message-id/CA%2BTgmoZWP8UtkNVLd75Qqoh9VGOVy_0xBK%2BSHZAdNvnfaikKsQ%40mail.gmail.com > From: Robert Haas > Date: Wednesday, September 21, 2022 at 11:54 AM > To: Antonin Houska > Cc: PostgreSQL Hackers , Peter Eisentraut > , S

Re: Temporary file access API

2022-09-22 Thread John Morris
ers , Peter Eisentraut , Stephen Frost Subject: Re: Temporary file access API On Mon, Aug 8, 2022 at 2:26 PM Antonin Houska wrote: > > I don't think that (3) is a separate advantage from (1) and (2), so I > > don't have anything separate to say about it. > > I thought that the uncon

Re: Temporary file access API

2022-08-09 Thread Robert Haas
On Mon, Aug 8, 2022 at 2:26 PM Antonin Houska wrote: > > I don't think that (3) is a separate advantage from (1) and (2), so I > > don't have anything separate to say about it. > > I thought that the uncontrollable buffer size is one of the things you > complaint about in > >

Re: Temporary file access API

2022-08-08 Thread Bruce Momjian
On Mon, Aug 8, 2022 at 08:27:27PM +0200, Antonin Houska wrote: > > For instance, to go back to my earlier comments, if the data is > > already block-structured, we do not need to insert a second layer of > > buffering; if it isn't, maybe we should, not just for TDE but for > > other reasons as

Re: Temporary file access API

2022-08-08 Thread Antonin Houska
Robert Haas wrote: > On Fri, Jul 29, 2022 at 11:36 AM Antonin Houska wrote: > > Attached is a new version. It allows the user to set "elevel" (i.e. ERROR is > > not necessarily thrown on I/O failure, if the user prefers to check the > > number > > of bytes read/written) and to specify the

Re: Temporary file access API

2022-07-29 Thread Robert Haas
On Fri, Jul 29, 2022 at 11:36 AM Antonin Houska wrote: > Attached is a new version. It allows the user to set "elevel" (i.e. ERROR is > not necessarily thrown on I/O failure, if the user prefers to check the number > of bytes read/written) and to specify the buffer size. Also, 0015 adds a >

Re: Temporary file access API

2022-07-29 Thread Antonin Houska
Antonin Houska wrote: > Peter Eisentraut wrote: > > > On 04.07.22 18:35, Antonin Houska wrote: > > >> Attached is a new version of the patch, to evaluate what the API use in > > >> the > > >> backend could look like. I haven't touched places where the file is > > >> accessed > > >> in a

Re: Temporary file access API

2022-07-05 Thread Antonin Houska
Peter Eisentraut wrote: > On 04.07.22 18:35, Antonin Houska wrote: > >> Attached is a new version of the patch, to evaluate what the API use in the > >> backend could look like. I haven't touched places where the file is > >> accessed > >> in a non-trivial way, e.g. lseek() / fseek() or

Re: Temporary file access API

2022-07-04 Thread Peter Eisentraut
On 04.07.22 18:35, Antonin Houska wrote: Attached is a new version of the patch, to evaluate what the API use in the backend could look like. I haven't touched places where the file is accessed in a non-trivial way, e.g. lseek() / fseek() or pg_pwrite() / pg_pread() is called. Rebased patch set

Re: Temporary file access API

2022-07-04 Thread Antonin Houska
Antonin Houska wrote: > Robert Haas wrote: > > > On Tue, Apr 12, 2022 at 5:30 AM Antonin Houska wrote: > > > Robert Haas wrote: > > > > On Mon, Apr 11, 2022 at 4:05 AM Antonin Houska wrote: > > > > > There are't really that many kinds of files to encrypt: > > > > > > > > > >

Re: Temporary file access API

2022-04-20 Thread Antonin Houska
Robert Haas wrote: > On Tue, Apr 12, 2022 at 5:30 AM Antonin Houska wrote: > > Robert Haas wrote: > > > On Mon, Apr 11, 2022 at 4:05 AM Antonin Houska wrote: > > > > There are't really that many kinds of files to encrypt: > > > > > > > >

Re: Temporary file access API

2022-04-13 Thread Bruce Momjian
On Wed, Apr 13, 2022 at 06:54:13PM -0400, Robert Haas wrote: > On Wed, Apr 13, 2022 at 6:25 PM Bruce Momjian wrote: > > I don't think we want to be encrypting pg_xact/, so they can get the > > transaction commit rate from there. > > I think it would be a good idea to eventually encrypt SLRU

Re: Temporary file access API

2022-04-13 Thread Stephen Frost
Greetings, On Wed, Apr 13, 2022 at 18:54 Robert Haas wrote: > On Wed, Apr 13, 2022 at 6:25 PM Bruce Momjian wrote: > > I don't think we want to be encrypting pg_xact/, so they can get the > > transaction commit rate from there. > > I think it would be a good idea to eventually encrypt SLRU

Re: Temporary file access API

2022-04-13 Thread Robert Haas
On Wed, Apr 13, 2022 at 6:25 PM Bruce Momjian wrote: > I don't think we want to be encrypting pg_xact/, so they can get the > transaction commit rate from there. I think it would be a good idea to eventually encrypt SLRU data, including pg_xact. Maybe not in the first version. -- Robert Haas

Re: Temporary file access API

2022-04-13 Thread Bruce Momjian
On Mon, Apr 11, 2022 at 04:34:18PM -0400, Robert Haas wrote: > On Mon, Apr 11, 2022 at 4:05 AM Antonin Houska wrote: > > There are't really that many kinds of files to encrypt: > > > > https://wiki.postgresql.org/wiki/Transparent_Data_Encryption#List_of_the_files_that_contain_user_data > > > >

Re: Temporary file access API

2022-04-13 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Tue, Apr 12, 2022 at 5:30 AM Antonin Houska wrote: > > Robert Haas wrote: > > > On Mon, Apr 11, 2022 at 4:05 AM Antonin Houska wrote: > > > > There are't really that many kinds of files to encrypt: > > > > > > > >

Re: Temporary file access API

2022-04-13 Thread Antonin Houska
Matthias van de Meent wrote: > On Mon, 11 Apr 2022 at 10:05, Antonin Houska wrote: > > > > There are't really that many kinds of files to encrypt: > > > > https://wiki.postgresql.org/wiki/Transparent_Data_Encryption#List_of_the_files_that_contain_user_data > > > > (And pg_stat/* files should be

Re: Temporary file access API

2022-04-12 Thread Matthias van de Meent
On Mon, 11 Apr 2022 at 10:05, Antonin Houska wrote: > > Robert Haas wrote: > > > On Fri, Apr 8, 2022 at 4:54 AM Antonin Houska wrote: > > > Do you think that the use of a system call is a problem itself (e.g. > > > because > > > the code looks less simple if read/write is used somewhere and >

Re: Temporary file access API

2022-04-12 Thread Robert Haas
On Tue, Apr 12, 2022 at 5:30 AM Antonin Houska wrote: > Robert Haas wrote: > > On Mon, Apr 11, 2022 at 4:05 AM Antonin Houska wrote: > > > There are't really that many kinds of files to encrypt: > > > > > >

Re: Temporary file access API

2022-04-12 Thread Antonin Houska
Robert Haas wrote: > On Mon, Apr 11, 2022 at 4:05 AM Antonin Houska wrote: > > There are't really that many kinds of files to encrypt: > > > > https://wiki.postgresql.org/wiki/Transparent_Data_Encryption#List_of_the_files_that_contain_user_data > > > > (And pg_stat/* files should be removed

Re: Temporary file access API

2022-04-11 Thread Robert Haas
On Mon, Apr 11, 2022 at 4:05 AM Antonin Houska wrote: > There are't really that many kinds of files to encrypt: > > https://wiki.postgresql.org/wiki/Transparent_Data_Encryption#List_of_the_files_that_contain_user_data > > (And pg_stat/* files should be removed from the list.) This kind of gets

Re: Temporary file access API

2022-04-11 Thread Antonin Houska
Robert Haas wrote: > On Fri, Apr 8, 2022 at 4:54 AM Antonin Houska wrote: > > Do you think that the use of a system call is a problem itself (e.g. because > > the code looks less simple if read/write is used somewhere and fread/fwrite > > elsewhere; of course of read/write is mandatory in

Re: Temporary file access API

2022-04-08 Thread Robert Haas
On Fri, Apr 8, 2022 at 4:54 AM Antonin Houska wrote: > Do you think that the use of a system call is a problem itself (e.g. because > the code looks less simple if read/write is used somewhere and fread/fwrite > elsewhere; of course of read/write is mandatory in special cases like WAL, > heap

Re: Temporary file access API

2022-04-08 Thread Antonin Houska
Robert Haas wrote: > On Tue, Mar 8, 2022 at 6:12 AM Antonin Houska wrote: > > Thanks for your comments, the initial version is attached here. > > I've been meaning to look at this thread for some time but have not > found enough time to do that until just now. And now I have > questions... >

Re: Temporary file access API

2022-04-05 Thread Robert Haas
On Tue, Mar 8, 2022 at 6:12 AM Antonin Houska wrote: > Thanks for your comments, the initial version is attached here. I've been meaning to look at this thread for some time but have not found enough time to do that until just now. And now I have questions... 1. Supposing we accepted this, how

Re: Temporary file access API

2022-03-08 Thread Antonin Houska
Stephen Frost wrote: > * Antonin Houska (a...@cybertec.at) wrote: > > Here I'm starting a new thread to discuss a topic that's related to the > > Transparent Data Encryption (TDE), but could be useful even without that. > > The > > problem has been addressed somehow in the Cybertec TDE fork,

Re: Temporary file access API

2022-03-01 Thread Stephen Frost
Greetings, * Antonin Houska (a...@cybertec.at) wrote: > Here I'm starting a new thread to discuss a topic that's related to the > Transparent Data Encryption (TDE), but could be useful even without that. The > problem has been addressed somehow in the Cybertec TDE fork, and I can post > the code

Temporary file access API

2022-02-08 Thread Antonin Houska
Here I'm starting a new thread to discuss a topic that's related to the Transparent Data Encryption (TDE), but could be useful even without that. The problem has been addressed somehow in the Cybertec TDE fork, and I can post the code here if it helps. However, after reading [1] (and the posts