Re: UniqueKey v2

2024-05-13 Thread Antonin Houska
Antonin Houska wrote: > Andy Fan wrote: > > > > > > * Combining the UKs > > > > > > IMO this is the most problematic part of the patch. You call > > > populate_joinrel_uniquekeys() for the same join multiple times, > > > > Why

Re: UniqueKey v2

2024-05-13 Thread Antonin Houska
s ECs meets at least one of the following conditions: > > > > a) contains a constant > > True. > > > > b) contains a column of a relation which has already been proven > > single-row. > > True, not sure if it is easy to tell. > > > b) is referenced by an

Join removal and attr_needed cleanup

2024-04-29 Thread Antonin Houska
removals. Do cases like this seem worth the effort and is the logic I use correct? -- Antonin Houska Web: https://www.cybertec-postgresql.com >From 279856bf97ce08c0c2e0c736a00831bf6324713b Mon Sep 17 00:00:00 2001 From: Antonin Houska Date: Mon, 29 Apr 2024 11:34:30 +0200 Subject: [PATCH] Clea

Use "unique keys" to enhance outer join removal

2024-04-29 Thread Antonin Houska
minor fix to [1]. [1] https://www.postgresql.org/message-id/7971.1713526758%40antos -- Antonin Houska Web: https://www.cybertec-postgresql.com >From a0be4ee7698ff03d6c22398f20fd2c7efadbff45 Mon Sep 17 00:00:00 2001 From: Antonin Houska Date: Mon, 29 Apr 2024 07:53:00 +0200 Subject: [PATCH 1/2]

Re: UniqueKey v2

2024-04-19 Thread Antonin Houska
s unique because its EC meets the condition a). (Of course you need to check em_jdomain before you use particular EM.) Are you going to submit the patch to the first CF of PG 18? Please let me know if I can contribute to the effort by reviewing or writing some code. -- Antonin Houska Web:

Re: why there is not VACUUM FULL CONCURRENTLY?

2024-02-16 Thread Antonin Houska
Alvaro Herrera wrote: > This is great to hear. > > On 2024-Jan-31, Antonin Houska wrote: > > > Is your plan to work on it soon or should I try to write a draft patch? (I > > assume this is for PG >= 18.) > > I don't have plans for it, so if you have reso

pg_language(langispl) column apparently not needed

2024-02-02 Thread Antonin Houska
I couldn't find a reference to the 'langispl' attribute, so I removed it (see the diff attached) and the master branch compiled cleanly. Is there yet a reason to keep it? -- Antonin Houska Web: https://www.cybertec-postgresql.com diff --git a/src/backend/commands/proclang.c b/src/backend

Re: why there is not VACUUM FULL CONCURRENTLY?

2024-01-31 Thread Antonin Houska
ume this is for PG >= 18.) [1] https://www.postgresql.org/docs/current/mvcc-caveats.html -- Antonin Houska Web: https://www.cybertec-postgresql.com

cost_incremental_sort() and limit_tuples

2023-12-03 Thread Antonin Houska
I think that cost_incremental_sort() does not account for the limit_tuples argument properly. Attached is my proposal to fix the problem. -- Antonin Houska Web: https://www.cybertec-postgresql.com diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c index

Stop the search once replication origin is found

2023-11-20 Thread Antonin Houska
Although it's not performance-critical, I think it just makes sense to break the loop in replorigin_session_setup() as soon as we've found the origin. -- Antonin Houska Web: https://www.cybertec-postgresql.com diff --git a/src/backend/replication/logical/origin.c b/src/backend/replication

Re: walsender "wakeup storm" on PG16, likely because of bc971f4025c (Optimize walsender wake up logic using condition variables)

2023-08-17 Thread Antonin Houska
Thomas Munro wrote: > On Wed, Aug 16, 2023 at 11:18 PM Antonin Houska wrote: > > I try to understand this patch (commit 5ffb7c7750) because I use condition > > variable in an extension. One particular problem occured to me, please > > consider: > > > > Conditio

Re: walsender "wakeup storm" on PG16, likely because of bc971f4025c (Optimize walsender wake up logic using condition variables)

2023-08-16 Thread Antonin Houska
was sent in between. Shouldn't at least AbortTransaction() and AbortSubTransaction() check the return value of ConditionVariableCancelSleep(), and re-send the signal if needed? Note that I'm just thinking about such a problem, did not try to reproduce it. -- Antonin Houska Web: https://www.cybertec-postgresql.com

Shouldn't cost_append() also scale the partial path's cost?

2023-06-14 Thread Antonin Houska
Like in cost_seqscan(), I'd expect the subpath cost to be divided among parallel workers. The patch below shows what I mean. Am I right? -- Antonin Houska Web: https://www.cybertec-postgresql.com diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c index

Re: Privileges on PUBLICATION

2023-03-15 Thread Antonin Houska
mprehensive description of what it is trying to do"? :) I tried to improve the documentation and commit messages in v05. v06 (just rebased) is attached. -- Antonin Houska Web: https://www.cybertec-postgresql.com >From d4490664ec80f52d23c4345eec5771764bcdbb17 Mon Sep 17 00:00:00 2001 From

Re: Parallelize correlated subqueries that execute within each worker

2023-03-08 Thread Antonin Houska
James Coleman wrote: > On Mon, Feb 6, 2023 at 11:39 AM Antonin Houska wrote: > Attached is v9. ok, I've changed the status to RfC -- Antonin Houska Web: https://www.cybertec-postgresql.com

Re: Parallelize correlated subqueries that execute within each worker

2023-02-06 Thread Antonin Houska
y contain more relations than required_outer, and therefore it's safer to check the first. Similar comments might apply to generate_useful_gather_paths(). Here I also suggest to move this test /* We can't pass params to workers. */ if (!bms_is_subset(PATH_REQ_OUTER(subpath), rel->relids)) continue; to the top of the loop because it's relatively cheap. -- Antonin Houska Web: https://www.cybertec-postgresql.com

Re: RLS makes COPY TO process child tables

2023-02-01 Thread Antonin Houska
Yugo NAGATA wrote: > On Wed, 01 Feb 2023 12:45:57 +0100 > Antonin Houska wrote: > > > While working on [1] I noticed that if RLS gets enabled, the COPY TO command > > includes the contents of child table into the result, although the > > documentation says it sho

RLS makes COPY TO process child tables

2023-02-01 Thread Antonin Houska
l) as well as fix proposal (copy_rls_no_inh.diff). [1] https://commitfest.postgresql.org/41/3641/ -- Antonin Houska Web: https://www.cybertec-postgresql.com create table a(i int); insert into a values (1); create table a1() inherits(a); insert into a1 values (1); -- Only the parent table

Re: Cross-partition UPDATE and foreign table partitions

2023-01-17 Thread Antonin Houska
Antonin Houska wrote: > I was wondering why ExecCrossPartitionUpdateForeignKey() has an unused > argument "oldslot" and wanted to suggest its removal. However, before I did, > it occurred to me that callers may want to pass the whole slot when the > partition is a fo

Cross-partition UPDATE and foreign table partitions

2023-01-17 Thread Antonin Houska
tead, I'd expect it to delete the tuple from "a1" and insert it into "a2". That looks like a bug. -- Antonin Houska Web: https://www.cybertec-postgresql.com CREATE EXTENSION IF NOT EXISTS postgres_fdw WITH SCHEMA public; CREATE SERVER s1 FOREIGN DATA WRAPPER postgres_fdw OPTION

Re: grouping pushdown

2023-01-04 Thread Antonin Houska
If both sides should be grouped, finalization of the partial aggregates would be more difficult, and I'm not sure it'd be worth the effort. > [1] https://commitfest.postgresql.org/41/3764/ -- Antonin Houska Web: https://www.cybertec-postgresql.com

Re: Privileges on PUBLICATION

2022-12-16 Thread Antonin Houska
Antonin Houska wrote: > Antonin Houska wrote: > > > Peter Eisentraut wrote: > > > > > On 04.11.22 08:28, Antonin Houska wrote: > > > > I thought about the whole concept a bit more and I doubt if the > > > > PUBLICATION > > &g

Re: refactor ExecGrant_*() functions

2022-12-13 Thread Antonin Houska
Peter Eisentraut wrote: > On 12.12.22 10:44, Antonin Houska wrote: > > Peter Eisentraut wrote: > > > >> On 06.12.22 09:41, Antonin Houska wrote: > >>> Attached are my proposals for improvements. One is to avoid memory leak, > >>> the >

Re: refactor ExecGrant_*() functions

2022-12-12 Thread Antonin Houska
Peter Eisentraut wrote: > On 06.12.22 09:41, Antonin Houska wrote: > > Attached are my proposals for improvements. One is to avoid memory leak, the > > other tries to improve readability a little bit. > > I added the readability improvement to my v2 patch. The pfree() cal

sendFileWithContent() does not advance the source pointer

2022-12-08 Thread Antonin Houska
generates scripts to create many tablespaces as well as the underlying directories. Fix is attached here as well. -- Antonin Houska Web: https://www.cybertec-postgresql.com #!/bin/bash TBSPDIR=/mnt/ramdisk/tbspcs TBSPCOUNT=2048 SCRIPT_SH=create.sh SCRIPT_SQL=create.sql echo "#!/bin

Re: refactor ExecGrant_*() functions

2022-12-06 Thread Antonin Houska
ww.postgresql.org/message-id/flat/20330.1652105397@antos -- Antonin Houska Web: https://www.cybertec-postgresql.com diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c index d3121a469f..ac4490c0b8 100644 --- a/src/backend/catalog/aclchk.c +++ b/src/backend/catalog/aclchk.c @@ -22

Re: Privileges on PUBLICATION

2022-11-29 Thread Antonin Houska
Antonin Houska wrote: > Peter Eisentraut wrote: > > > On 04.11.22 08:28, Antonin Houska wrote: > > > I thought about the whole concept a bit more and I doubt if the > > > PUBLICATION > > > privilege is the best approach. In particular, the user specif

Re: Privileges on PUBLICATION

2022-11-14 Thread Antonin Houska
Peter Eisentraut wrote: > On 04.11.22 08:28, Antonin Houska wrote: > > I thought about the whole concept a bit more and I doubt if the PUBLICATION > > privilege is the best approach. In particular, the user specified in CREATE > > SUBSCRIPTION ... CONNECTION ... (say "

Re: refactor ownercheck and aclcheck functions

2022-11-07 Thread Antonin Houska
the amount of (almost) copy & pasted code. I haven't found any problem in your patch that would be worth mentioning, except that the 0001 part does not apply to the current master branch. -- Antonin Houska Web: https://www.cybertec-postgresql.com

Re: Privileges on PUBLICATION

2022-11-04 Thread Antonin Houska
Mark Dilger wrote: > > On Nov 4, 2022, at 12:28 AM, Antonin Houska wrote: > > > > I thought about the whole concept a bit more and I doubt if the PUBLICATION > > privilege is the best approach. In particular, the user specified in CREATE > > SUBSCRIPTION ... CON

Re: Privileges on PUBLICATION

2022-11-04 Thread Antonin Houska
Amit Kapila wrote: > On Thu, Nov 3, 2022 at 11:12 AM Antonin Houska wrote: > > > > Peter Eisentraut wrote: > > > > > The CF entry is about privileges on publications. Please rebase that > > > patch > > > and repost it so that the CF app an

Re: Privileges on PUBLICATION

2022-11-04 Thread Antonin Houska
Peter Eisentraut wrote: > On 03.11.22 01:43, Antonin Houska wrote: > > Peter Eisentraut wrote: > > > >> The CF entry is about privileges on publications. Please rebase that patch > >> and repost it so that the CF app and the CF bot are up to date. > > T

Re: Privileges on PUBLICATION

2022-11-02 Thread Antonin Houska
ion. I haven't added a note about this problem to the documentation yet, in case someone has better idea how to approach the problem. -- Antonin Houska Web: https://www.cybertec-postgresql.com diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml index 03c0193709..d510220a07 100644 --- a/doc/sr

Re: Temporary file access API

2022-09-23 Thread Antonin Houska
be useful for both. [1] https://www.postgresql.org/message-id/CA%2BTgmoZWP8UtkNVLd75Qqoh9VGOVy_0xBK%2BSHZAdNvnfaikKsQ%40mail.gmail.com > From: Robert Haas > Date: Wednesday, September 21, 2022 at 11:54 AM > To: Antonin Houska > Cc: PostgreSQL Hackers , Peter Eisentraut > , S

Return value of PathNameOpenFile()

2022-09-06 Thread Antonin Houska
call, I assume that zero is a valid result, isn't it? -- Antonin Houska Web: https://www.cybertec-postgresql.com

Re: Temporary file access API

2022-08-08 Thread Antonin Houska
Robert Haas wrote: > On Fri, Jul 29, 2022 at 11:36 AM Antonin Houska wrote: > > Attached is a new version. It allows the user to set "elevel" (i.e. ERROR is > > not necessarily thrown on I/O failure, if the user prefers to check the > > number > >

Re: Temporary file access API

2022-07-29 Thread Antonin Houska
Antonin Houska wrote: > Peter Eisentraut wrote: > > > On 04.07.22 18:35, Antonin Houska wrote: > > >> Attached is a new version of the patch, to evaluate what the API use in > > >> the > > >> backend could look like. I haven't touched places wh

Re: Proposal: allow database-specific role memberships

2022-07-19 Thread Antonin Houska
query in check_memberships() only needs an outer JOIN for the pg_database table, while the other joins can be inner. * In this part SET SESSION AUTHORIZATION role_read_12_noinherit; SELECT * FROM data; -- error SET ROLE role_read_12; -- error SELECT * FROM data; -- error I think you don't need to query the table again if the SET ROLE statement failed and the same query had been executed before the SET ROLE. -- Antonin Houska Web: https://www.cybertec-postgresql.com

Re: Problem about postponing gathering partial paths for topmost scan/join rel

2022-07-18 Thread Antonin Houska
Richard Guo wrote: > On Fri, Jul 15, 2022 at 5:00 PM Richard Guo wrote: > > On Fri, Jul 15, 2022 at 4:03 PM Richard Guo wrote: > > On Thu, Jul 14, 2022 at 10:02 PM Antonin Houska wrote: > > I'd prefer a test that demonstrates that the Gather node at the top of th

Re: Problem about postponing gathering partial paths for topmost scan/join rel

2022-07-14 Thread Antonin Houska
the parallel "Hash Full Join" gets implemented (I've noticed but not checked in detail [1]), it might break this test. I'd prefer a test that demonstrates that the Gather node at the top of the "subproblem plan" is useful purely from the *cost* perspective, rather than due to executor limitation. [1] https://commitfest.postgresql.org/38/2903/ -- Antonin Houska Web: https://www.cybertec-postgresql.com

Re: [PROPOSAL] Detecting plan changes with plan_id in pg_stat_activity

2022-07-14 Thread Antonin Houska
I don't see how rebuilding an extra hashtable in > entry_dealloc(), so when holding an exclusive lwlock, while not saving any > other work elsewhere could be free? > > Looking at the script, you have: > echo "log_min_messages=debug1" >> $PGDATA/postgresql.conf; \ > > Is that really necessary? Couldn't you upgrade the gc message to a higher > level for your benchmark need, or expose some new counter in > pg_stat_statements_info maybe? Have you done the benchmark using a debug > build > or normal build? > > -- Antonin Houska Web: https://www.cybertec-postgresql.com

Re: Temporary file access API

2022-07-05 Thread Antonin Houska
Peter Eisentraut wrote: > On 04.07.22 18:35, Antonin Houska wrote: > >> Attached is a new version of the patch, to evaluate what the API use in the > >> backend could look like. I haven't touched places where the file is > >> accessed > >> in

Re: Temporary file access API

2022-07-04 Thread Antonin Houska
Antonin Houska wrote: > Robert Haas wrote: > > > On Tue, Apr 12, 2022 at 5:30 AM Antonin Houska wrote: > > > Robert Haas wrote: > > > > On Mon, Apr 11, 2022 at 4:05 AM Antonin Houska wrote: > > > > > There are't really that many kind

Re: Proposal: allow database-specific role memberships

2022-06-29 Thread Antonin Houska
ssions of "B" (e.g. "pg_write_all_data"), then I think the roles hierarchy is poorly designed. A counter-example might help me to understand. * Why do you think that "unsafe_tests" is the appropriate name for the directory that contains regression tests? I can spend more time on the review if the patch gets rebased. -- Antonin Houska Web: https://www.cybertec-postgresql.com

Re: Privileges on PUBLICATION

2022-06-20 Thread Antonin Houska
Euler Taveira wrote: > --eeab359ad6094efd84562cddd7fb9e89 > Content-Type: text/plain > > On Wed, May 18, 2022, at 6:44 AM, Antonin Houska wrote: > > ok, please see the next version. > The new paragraph looks good to me. I'm not sure if the CREATE PUBLICATION is > th

Re: Privileges on PUBLICATION

2022-05-19 Thread Antonin Houska
Euler Taveira wrote: > On Wed, May 18, 2022, at 6:16 AM, Antonin Houska wrote: > > The patch is attached to this message. > > Great. Add it to the next CF. I'll review it when I have some spare time. https://commitfest.postgresql.org/38/3641/ Thanks! -- Antonin Ho

Re: Privileges on PUBLICATION

2022-05-18 Thread Antonin Houska
Euler Taveira wrote: > On Fri, May 13, 2022, at 3:36 AM, Antonin Houska wrote: > > Attached is my proposal. It tries to be more specific and does not mention > the > absence of the privileges explicitly. > > You explained the current issue but say nothing

Re: Privileges on PUBLICATION

2022-05-18 Thread Antonin Houska
Antonin Houska wrote: > Euler Taveira wrote: > > > On Tue, May 10, 2022, at 5:37 AM, Antonin Houska wrote: > > > > My understanding is that the rows/columns filtering is a way for the > > *publisher* to control which data is available to particular replica

Re: Privileges on PUBLICATION

2022-05-13 Thread Antonin Houska
Peter Eisentraut wrote: > On 10.05.22 10:37, Antonin Houska wrote: > > My understanding is that the rows/columns filtering is a way for the > > *publisher* to control which data is available to particular replica. From > > this point of view, the publication privi

Re: Privileges on PUBLICATION

2022-05-13 Thread Antonin Houska
Euler Taveira wrote: > On Tue, May 10, 2022, at 5:37 AM, Antonin Houska wrote: > > My understanding is that the rows/columns filtering is a way for the > *publisher* to control which data is available to particular replica. From > this point of view, the publication privil

Re: Privileges on PUBLICATION

2022-05-10 Thread Antonin Houska
Amit Kapila wrote: > On Tue, May 10, 2022 at 12:16 AM Euler Taveira wrote: > > > > On Mon, May 9, 2022, at 11:09 AM, Antonin Houska wrote: > > > > Now that the user can specify rows and columns to be omitted from the > > logical > > replication [1],

Re: Privileges on PUBLICATION

2022-05-10 Thread Antonin Houska
Euler Taveira wrote: > On Mon, May 9, 2022, at 11:09 AM, Antonin Houska wrote: > > Now that the user can specify rows and columns to be omitted from the logical > replication [1], I suppose hiding rows and columns from the subscriber is an > important use case.

Privileges on PUBLICATION

2022-05-09 Thread Antonin Houska
] https://www.postgresql.org/docs/devel/sql-createpublication.html -- Antonin Houska Web: https://www.cybertec-postgresql.com

Re: Temporary file access API

2022-04-20 Thread Antonin Houska
Robert Haas wrote: > On Tue, Apr 12, 2022 at 5:30 AM Antonin Houska wrote: > > Robert Haas wrote: > > > On Mon, Apr 11, 2022 at 4:05 AM Antonin Houska wrote: > > > > There are't really that many kinds of files to encrypt: > > > >

Re: Temporary file access API

2022-04-13 Thread Antonin Houska
Matthias van de Meent wrote: > On Mon, 11 Apr 2022 at 10:05, Antonin Houska wrote: > > > > There are't really that many kinds of files to encrypt: > > > > https://wiki.postgresql.org/wiki/Transparent_Data_Encryption#List_of_the_files_that_contain_user_data > >

Re: Temporary file access API

2022-04-12 Thread Antonin Houska
Robert Haas wrote: > On Mon, Apr 11, 2022 at 4:05 AM Antonin Houska wrote: > > There are't really that many kinds of files to encrypt: > > > > https://wiki.postgresql.org/wiki/Transparent_Data_Encryption#List_of_the_files_that_contain_user_data > > > > (And

Re: 回复:POC: Cleaning up orphaned files using undo logs

2022-04-11 Thread Antonin Houska
uZyQQ9h%3DoRiy%3DZQV5%2BxHQXxMWmep4Ygg8Dg%40mail.gmail.com -- Antonin Houska Web: https://www.cybertec-postgresql.com

Re: Temporary file access API

2022-04-11 Thread Antonin Houska
Robert Haas wrote: > On Fri, Apr 8, 2022 at 4:54 AM Antonin Houska wrote: > > Do you think that the use of a system call is a problem itself (e.g. because > > the code looks less simple if read/write is used somewhere and fread/fwrite > > elsewhere; of course of re

Re: Temporary file access API

2022-04-08 Thread Antonin Houska
Robert Haas wrote: > On Tue, Mar 8, 2022 at 6:12 AM Antonin Houska wrote: > > Thanks for your comments, the initial version is attached here. > > I've been meaning to look at this thread for some time but have not > found enough time to do that until just now. And now

Re: Logical replication row filtering and TOAST

2022-04-05 Thread Antonin Houska
Amit Kapila wrote: > > Antonin Houska wrote: > > > > Nevertheless, a comment in pgoutput_row_filter(), saying that TOASTed values > > are not expected if old_slot is NULL, might be useful. > > > > How about something like the attached? Yes, that'd be suf

Re: Logical replication row filtering and TOAST

2022-04-05 Thread Antonin Houska
Antonin Houska wrote: > I spent some time thinking about a special case of evaluation of the row > filter and wrote a comment that might be useful (see the attachment). However > now I think that it's not perfect if the code really relies on the fact that > value of an indexed c

Logical replication row filtering and TOAST

2022-04-05 Thread Antonin Houska
n_tuple(oldtup, desc); heap_freetuple(oldtup); } Do I miss anything? -- Antonin Houska Web: https://www.cybertec-postgresql.com diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c index 20d0b1e1253..2be5aaa18c4 100644 --- a/src/backend/replication

Re: Temporary file access API

2022-03-08 Thread Antonin Houska
Stephen Frost wrote: > * Antonin Houska (a...@cybertec.at) wrote: > > Here I'm starting a new thread to discuss a topic that's related to the > > Transparent Data Encryption (TDE), but could be useful even without that. > > The > > problem has been addressed someh

Re: storing an explicit nonce

2022-02-08 Thread Antonin Houska
effort that, sure, may make > some things better too but if we could do that with the same API then it > could also be done later if someone's interested in that. My initial proposal is in this new thread: https://www.postgresql.org/message-id/4987.1644323098%40antos -- Antonin Houska Web: https://www.cybertec-postgresql.com

Temporary file access API

2022-02-08 Thread Antonin Houska
of cipher, in case we support more than one. I think that's enough to start the discussion. Thanks for feedback in advance. [1] https://www.postgresql.org/message-id/CA%2BTgmoYGjN_f%3DFCErX49bzjhNG%2BGoctY%2Ba%2BXhNRWCVvDY8U74w%40mail.gmail.com -- Antonin Houska Web: https://www.cybertec-postgresql.com

Re: Doc: CREATE_REPLICATION_SLOT command requires the plugin name

2022-02-01 Thread Antonin Houska
Amit Kapila wrote: > On Tue, Feb 1, 2022 at 5:43 PM Antonin Houska wrote: > > > > Amit Kapila wrote: > > > > > On Tue, Feb 1, 2022 at 3:44 PM Antonin Houska wrote: > > > > > > > > I got a syntax error when using the command according to

Re: Doc: CREATE_REPLICATION_SLOT command requires the plugin name

2022-02-01 Thread Antonin Houska
Amit Kapila wrote: > On Tue, Feb 1, 2022 at 3:44 PM Antonin Houska wrote: > > > > I got a syntax error when using the command according to the existing > > documentation. The output_plugin parameter needs to be passed too. > > > > Why do we need it f

Doc: CREATE_REPLICATION_SLOT command requires the plugin name

2022-02-01 Thread Antonin Houska
I got a syntax error when using the command according to the existing documentation. The output_plugin parameter needs to be passed too. -- Antonin Houska Web: https://www.cybertec-postgresql.com diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml index 24e93f9b28..6065c18fdf

Re: XTS cipher mode for cluster file encryption

2022-01-31 Thread Antonin Houska
Bruce Momjian wrote: > On Mon, Nov 29, 2021 at 08:37:31AM +0100, Antonin Houska wrote: > > The changes to buffile.c are not trivial, but we haven't really changed the > > API, as long as you mean BufFileCreateTemp(), BufFileWrite(), BufFileRead(). > > > > What our

Re: [PATCH] buffile: ensure start offset is aligned with BLCKSZ

2021-11-29 Thread Antonin Houska
nitialization vector (IV) is BLCKSZ bytes. Another problem is that you might want to store the IV somewhere in between the data. In short, the encryption makes the buffered IO rather different and the specific code should be kept aside, although the same API is used to invoke it. -- Antonin Houska

Re: XTS cipher mode for cluster file encryption

2021-11-28 Thread Antonin Houska
Transient() replace OpenTransientFile(), CloseTransientFile(), write()/fwrite() and read()/fread() respectively in reorderbuffer.c and in pgstat.c. These changes become a little bit less invasive in TDE 1.1 than they were in 1.0, see [1], see the diffs attached. (I expect that [2] will get committed somed

Re: Reuse of State value in Aggregates

2021-11-09 Thread Antonin Houska
likely be different. Maybe you need to post an example where you think such reusing would be useful. -- Antonin Houska Web: https://www.cybertec-postgresql.com

Re: [PATCH] Full support for index LP_DEAD hint bits on standby

2021-11-09 Thread Antonin Houska
wed()? > > Yes, this way it is looks better. Done. Also, I have added some checks > for “maybe” LSN-related logic to the test. Attached is a proposal for a minor addition that would make sense to me, add it if you think it's appropriate. I think I've said enough, changing the status

Re: [PATCH] Full support for index LP_DEAD hint bits on standby

2021-11-03 Thread Antonin Houska
uot;. * I can see no test for the INDEX_LP_DEAD_OK_MIN_LSN value of the IndexLpDeadAllowedResult enumeration. Shouldn't there be only two values, e.g. INDEX_LP_DEAD_OK and INDEX_LP_DEAD_MAYBE_OK ? Or a boolean variable (in index_fetch_heap()) of the appropriate name, e.g. kill_maybe_allowed, and

Re: storing an explicit nonce

2021-11-01 Thread Antonin Houska
ipher function to an unique value. Can you please try to explain once again what you consider a requirement (uniqueness, randomness, etc.) on the IV for the XTS mode? Thanks. -- Antonin Houska Web: https://www.cybertec-postgresql.com

Re: storing an explicit nonce

2021-10-08 Thread Antonin Houska
ar case isn't something that > XTS is particularly good at and that's generally accepted, yet lots of > folks use XTS anyway because the concern isn't "someone has root access > on the box and is watching all block writes" but rather "laptop was > stolen" where the attacker doesn't get to see multiple writes where the > same key+tweak has been used, and the latter is really the attack vector > we're looking to address with XTS too. I've heard a few times that database running in a cloud is also a valid use case for the TDE. In that case I think it should be expected that "someone has root access on the box and is watching all block writes". -- Antonin Houska Web: https://www.cybertec-postgresql.com

Re: storing an explicit nonce

2021-10-07 Thread Antonin Houska
. It would be an important difference if there was a guarantee that the 16-byte blocks are consitent even on torn page - does e.g. proper alignment of pages guarantee that? Nevertheless, the absence of the chaining may be a reason to prefer CBC to XTS anyway. -- Antonin Houska Web: https://www.cybertec-postgresql.com

Re: storing an explicit nonce

2021-10-05 Thread Antonin Houska
But I think ultimately we do want to encrypt and decrypt in blocks, so > if we create such a layer, it should expose byte-oriented APIs but > combine the actual I/Os somehow. That's also good for cutting down the > number of system calls, which is a benefit unto itself. -- Antonin Houska Web: https://www.cybertec-postgresql.com

Re: POC: Cleaning up orphaned files using undo logs

2021-09-29 Thread Antonin Houska
Thomas Munro wrote: > On Wed, Sep 29, 2021 at 8:18 AM Antonin Houska wrote: > > I'm just trying to use the existing infrastructure: the effect of DROP TABLE > > also appear to be performed by the checkpointer. However I don't know why > > the > > u

Re: POC: Cleaning up orphaned files using undo logs

2021-09-28 Thread Antonin Houska
Dmitry Dolgov <9erthali...@gmail.com> wrote: > > On Tue, Sep 21, 2021 at 10:07:55AM +0200, Dmitry Dolgov wrote: > > On Tue, 21 Sep 2021 09:00 Antonin Houska, wrote: > > > > > Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > > > &

Re: POC: Cleaning up orphaned files using undo logs

2021-09-28 Thread Antonin Houska
Amit Kapila wrote: > On Mon, Sep 20, 2021 at 10:55 AM Antonin Houska wrote: > > > > Amit Kapila wrote: > > > > > On Thu, Sep 9, 2021 at 8:33 PM Antonin Houska wrote: > > > > > > The problem of the temporary undo log is that it's loaded into loc

Re: POC: Cleaning up orphaned files using undo logs

2021-09-27 Thread Antonin Houska
Amit Kapila wrote: > On Fri, Sep 24, 2021 at 4:44 PM Antonin Houska wrote: > > > > Amit Kapila wrote: > > > > > On Mon, Sep 20, 2021 at 10:24 AM Antonin Houska wrote: > > > > > > > > Amit Kapila wrote: > > > > > > &g

Re: POC: Cleaning up orphaned files using undo logs

2021-09-24 Thread Antonin Houska
Amit Kapila wrote: > On Mon, Sep 20, 2021 at 10:24 AM Antonin Houska wrote: > > > > Amit Kapila wrote: > > > > > On Fri, Sep 17, 2021 at 9:50 PM Dmitry Dolgov <9erthali...@gmail.com> > > > wrote: > > > > > > >

Re: POC: Cleaning up orphaned files using undo logs

2021-09-21 Thread Antonin Houska
Dmitry Dolgov <9erthali...@gmail.com> wrote: > > On Tue, Sep 14, 2021 at 10:51:42AM +0200, Antonin Houska wrote: > > Antonin Houska wrote: > > > > > Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > > > > * By throwing at the patchset `

Re: [PATCH] Full support for index LP_DEAD hint bits on standby

2021-09-20 Thread Antonin Houska
learing of the "LP-safe-on-standby" bits, but, as I said above, I'm not sure if the tests ever let standby to set those bits before the promotion. -- Antonin Houska Web: https://www.cybertec-postgresql.com

Re: POC: Cleaning up orphaned files using undo logs

2021-09-19 Thread Antonin Houska
Amit Kapila wrote: > On Thu, Sep 9, 2021 at 8:33 PM Antonin Houska wrote: > > The problem of the temporary undo log is that it's loaded into local buffers > > and that backend can exit w/o flushing local buffers to disk, and thus we > > are > > not guaranteed to f

Re: POC: Cleaning up orphaned files using undo logs

2021-09-19 Thread Antonin Houska
Amit Kapila wrote: > On Fri, Sep 17, 2021 at 9:50 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > > > On Tue, Sep 14, 2021 at 10:51:42AM +0200, Antonin Houska wrote: > > > > * What happened with the idea of abandoning discard worker for the sake

Re: POC: Cleaning up orphaned files using undo logs

2021-06-30 Thread Antonin Houska
Antonin Houska wrote: > tsunakawa.ta...@fujitsu.com wrote: > > > I'm crawling like a snail to read the patch set. Below are my first set of > > review comments, which are all minor. > > Thanks. I've added the patch to the upcoming CF [1], so it possibly gets mo

Re: decoupling table and index vacuum

2021-06-23 Thread Antonin Houska
> different indexes could be removing different TID ranges. > > We kind of have this feature right now, due to parallel vacuum... -- Antonin Houska Web: https://www.cybertec-postgresql.com

Possible optimization of heap_lock_tuple()

2021-05-26 Thread Antonin Houska
It seems that a concurrent UPDATE can restart heap_lock_tuple() even if it's not necessary. Is the attached proposal correct and worth applying? -- Antonin Houska Web: https://www.cybertec-postgresql.com diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index

Re: storing an explicit nonce

2021-05-25 Thread Antonin Houska
is that the CTR mode encrypts each individual bit, independent from any other bit on the page. For non-hint bits data=data', so (data ^ data') is always zero, regardless the actual values of the data. So I agree with you that by reusing the nonce we only expose the hint bits. -- Antonin Houska Web: https://www.cybertec-postgresql.com

Re: [PATCH] Full support for index LP_DEAD hint bits on standby

2021-05-10 Thread Antonin Houska
" variables should always have the same value, so one variable should be enough. Assert(so->numKilled) would be appropriate in that case. The situation is similar for btree and hash indexes. doc.patch: "+applying the fill page write." [1] https://www.postgresql.org/message-id/61470.1620647290%40antos -- Antonin Houska Web: https://www.cybertec-postgresql.com

Re: [PATCH] Full support for index LP_DEAD hint bits on standby

2021-05-10 Thread Antonin Houska
ular” commit record is replayed without XLogFlush and, as > result, without UpdateMinRecoveryPoint. ok, I missed this. Thanks for explanation. -- Antonin Houska Web: https://www.cybertec-postgresql.com

Re: [PATCH] Full support for index LP_DEAD hint bits on standby

2021-05-10 Thread Antonin Houska
equently, see comments in UpdateMinRecoveryPoint(). I don't know if what you do in code_optional.patch would be a problem. Actually I think that a commit record should be replayed more often than XLOG_RUNNING_XACTS, shouldn't it? -- Antonin Houska Web: https://www.cybertec-postgresql.com

Re: [PATCH] Full support for index LP_DEAD hint bits on standby

2021-05-06 Thread Antonin Houska
REPLICA SCANS INDEX AND SET hint bits (index_lsn=10) >INDEX IS FLUSHED (minRecoveryPoint=index_lsn=10) >CRASH > > On crash recovery, a standby will be able to handle queries after > LSN=10. But the index page contains hints bits from the future >

Re: Built-in connection pooler

2021-04-28 Thread Antonin Houska
ssary. * Comment lines are often too long. * pgindent should be applied to the patch at some point. I can spend more time reviewing the patch during the next CF. -- Antonin Houska Web: https://www.cybertec-postgresql.com

Re: WIP: Aggregation push-down

2021-03-11 Thread Antonin Houska
al years, I don't believe another rebase(s) is anything but wasted effort. So I've withdrawn the patch, also to save CFMs from examining the history again and again uselessly. The code is free so anyone can continue if he thinks it makes sense. If it finally finds its way into the PG core and if meanin

Re: POC: Cleaning up orphaned files using undo logs

2021-03-10 Thread Antonin Houska
tsunakawa.ta...@fujitsu.com wrote: > I'm crawling like a snail to read the patch set. Below are my first set of > review comments, which are all minor. Thanks. I will check your comments when I'll be preparing the next version of the patch. -- Antonin Houska Web: https://www.cy

pg_config_h.in not up-to-date

2021-02-18 Thread Antonin Houska
When I run "autoreconf" on the master branch, git generates the diff below. Shouldn't it just be applied? I suppose someone changed configure.ac and forgot to update the generated file. -- Antonin Houska Web: https://www.cybertec-postgresql.com diff --git a/src/include/pg_config.

Re: POC: Cleaning up orphaned files using undo logs

2021-02-03 Thread Antonin Houska
Bruce Momjian wrote: > On Fri, Jan 29, 2021 at 06:30:15PM +0100, Antonin Houska wrote: > > Antonin Houska wrote: > > > Well, on repeated run of the test I could also hit the first one. I could > > > fix > > > it and will post a new version of th

  1   2   3   4   >