RE: [Proposal] Add foreign-server health checks infrastructure

2021-11-23 Thread kuroda.hay...@fujitsu.com
Dear Kato-san, I found the missing space, and I added a test. I'm very happy if you review this. Best Regards, Hayato Kuroda FUJITSU LIMITED v03_0001_add_checking_infrastracture.patch Description: v03_0001_add_checking_infrastracture.patch <>

Re: Post-CVE Wishlist

2021-11-23 Thread Peter Eisentraut
On 23.11.21 23:41, Heikki Linnakangas wrote: On 23/11/2021 23:44, Robert Haas wrote: On Tue, Nov 23, 2021 at 2:18 PM Tom Lane wrote: Jacob Champion writes: = Implicit TLS = Aside from security, one small benefit of skipping the Starttls-style negotiation is that you avoid one round-trip

Re: Mop-up from Test::More version change patch

2021-11-23 Thread Peter Eisentraut
On 23.11.21 18:03, Tom Lane wrote: 0002 is written to apply to v14 and earlier, and what it wants to do is back-patch the effects of 405f32fc4, so that the minimum Test::More version is 0.98 in all branches. The thought here is that (1) somebody is likely to want to back-patch a test involving

Re: Python 3.11 vs. Postgres

2021-11-23 Thread Peter Eisentraut
On 24.11.21 04:07, Tom Lane wrote: According to [1], we need to stop including Python's . I've not checked whether this creates any backwards-compatibility issues. regards, tom lane [1] https://bugzilla.redhat.com/show_bug.cgi?id=2023272 See attached patch. The

Re: Sequence's value can be rollback after a crashed recovery.

2021-11-23 Thread Laurenz Albe
On Tue, 2021-11-23 at 16:41 -0500, Tom Lane wrote: > I wrote: > > I wonder though if we shouldn't try to improve the existing text. > > The phrasing "never rolled back" seems like it's too easily > > misinterpreted.  Maybe rewrite the block like > > ... > > A bit of polishing later, maybe like

[BUG]Missing REPLICA IDENTITY check when DROP NOT NULL

2021-11-23 Thread tanghy.f...@fujitsu.com
Hi, I think I found a problem related to replica identity. According to PG doc at [1], replica identity includes only columns marked NOT NULL. But in fact users can accidentally break this rule as follows: create table tbl (a int not null unique); alter table tbl replica identity using INDEX

Re: parallel vacuum comments

2021-11-23 Thread Masahiko Sawada
On Wed, Nov 24, 2021 at 1:34 PM Amit Kapila wrote: > > On Wed, Nov 24, 2021 at 7:43 AM Masahiko Sawada wrote: > > > > On Mon, Nov 22, 2021 at 1:48 PM Amit Kapila wrote: > > > > > > On Fri, Nov 19, 2021 at 7:55 AM houzj.f...@fujitsu.com > > > wrote: > > > > > > > > > > > 4) > > > > > > > > Just

Re: pg_get_publication_tables() output duplicate relid

2021-11-23 Thread Amit Langote
On Tue, Nov 23, 2021 at 12:21 PM Amit Kapila wrote: > Isn't it better to document this case and explain what > users can expect instead of trying to design a solution around this? I thought about the problems you've described and it looks like I hadn't considered many of the details which

Re: parallel vacuum comments

2021-11-23 Thread Masahiko Sawada
On Wed, Nov 24, 2021 at 1:28 PM Amit Kapila wrote: > > On Wed, Nov 24, 2021 at 7:07 AM Masahiko Sawada wrote: > > > > On Mon, Nov 22, 2021 at 6:35 PM Amit Kapila wrote: > > > > > > On Tue, Nov 16, 2021 at 11:23 AM Masahiko Sawada > > > wrote: > > > > > > > > I've incorporated these comments

Re: Synchronizing slots from primary to standby

2021-11-23 Thread Masahiko Sawada
On Sun, Oct 31, 2021 at 7:08 PM Peter Eisentraut wrote: > > I want to reactivate $subject. I took Petr Jelinek's patch from [0], > rebased it, added a bit of testing. It basically works, but as > mentioned in [0], there are various issues to work out. Thank you for working on this feature! >

Re: Rename dead_tuples to dead_items in vacuumlazy.c

2021-11-23 Thread Dilip Kumar
On Wed, Nov 24, 2021 at 11:16 AM Peter Geoghegan wrote: > > Attached patch performs polishing within vacuumlazy.c, as follow-up > work to the refactoring work in Postgres 14. This mainly consists of > changing references of dead tuples to dead items, which reflects the > fact that VACUUM no

Re: row filtering for logical replication

2021-11-23 Thread Amit Kapila
On Wed, Nov 24, 2021 at 6:51 AM houzj.f...@fujitsu.com wrote: > > On Tues, Nov 23, 2021 6:16 PM Amit Kapila wrote: > > On Tue, Nov 23, 2021 at 1:29 PM houzj.f...@fujitsu.com > > wrote: > > > > > > On Tues, Nov 23, 2021 2:27 PM vignesh C wrote: > > > > On Thu, Nov 18, 2021 at 7:04 AM Peter

Rename dead_tuples to dead_items in vacuumlazy.c

2021-11-23 Thread Peter Geoghegan
Attached patch performs polishing within vacuumlazy.c, as follow-up work to the refactoring work in Postgres 14. This mainly consists of changing references of dead tuples to dead items, which reflects the fact that VACUUM no longer deals with TIDs that might point to remaining heap tuples with

Re: Teach pg_receivewal to use lz4 compression

2021-11-23 Thread Michael Paquier
On Mon, Nov 22, 2021 at 09:02:47AM -0500, Robert Haas wrote: > On Mon, Nov 22, 2021 at 12:46 AM Jeevan Ladhe > wrote: >> Fair enough. But, still I have a doubt in mind what benefit would that >> really bring to us here, because we are immediately also freeing the >> lz4buf without using it

Re: Post-CVE Wishlist

2021-11-23 Thread Michael Paquier
On Tue, Nov 23, 2021 at 02:18:30PM -0500, Tom Lane wrote: > Jacob Champion writes: >> = Client-Side Auth Selection = >> The second request is for the client to stop fully trusting the server >> during the authentication phase. If I tell libpq to use a client >> certificate, for example, I don't

RE: [Proposal] Add foreign-server health checks infrastructure

2021-11-23 Thread kuroda.hay...@fujitsu.com
Dear Kato-san, Thank you for reviewing! > Thank you for sending the patches! > I confirmed that they can be compiled and tested successfully on CentOS > 8. Thanks! > + { > + {"remote_servers_connection_check_interval", PGC_USERSET, > CONN_AUTH_SETTINGS, > +

Re: Mop-up from Test::More version change patch

2021-11-23 Thread Noah Misch
On Tue, Nov 23, 2021 at 12:03:05PM -0500, Tom Lane wrote: > Attached are a couple of patches I propose in the wake of commit > 405f32fc4 (Require version 0.98 of Test::More for TAP tests). > > 0001 responds to the failure we saw on buildfarm member wrasse [1] > where, despite configure having

Re: parallel vacuum comments

2021-11-23 Thread Amit Kapila
On Wed, Nov 24, 2021 at 7:43 AM Masahiko Sawada wrote: > > On Mon, Nov 22, 2021 at 1:48 PM Amit Kapila wrote: > > > > On Fri, Nov 19, 2021 at 7:55 AM houzj.f...@fujitsu.com > > wrote: > > > > > > > > 4) > > > > > > Just a personal suggestion for the parallel related function name. Since > > >

RE: Implementing Incremental View Maintenance

2021-11-23 Thread r.takahash...@fujitsu.com
Hi Nagata-san, > Ok. I'll fix _copyIntoClause() and _equalIntoClause() as well as > _readIntoClause() > and _outIntoClause(). OK. > > ivm=# create table t (c1 int, c2 int); > > CREATE TABLE > > ivm=# create incremental materialized view ivm_t as select distinct c1 from > > t; > > NOTICE:

Re: parallel vacuum comments

2021-11-23 Thread Amit Kapila
On Wed, Nov 24, 2021 at 7:07 AM Masahiko Sawada wrote: > > On Mon, Nov 22, 2021 at 6:35 PM Amit Kapila wrote: > > > > On Tue, Nov 16, 2021 at 11:23 AM Masahiko Sawada > > wrote: > > > > > > I've incorporated these comments and attached an updated patch. > > > > > > > Review comments: > >

RE: Implementing Incremental View Maintenance

2021-11-23 Thread r.takahash...@fujitsu.com
Hi Nagata-san, Sorry for late reply. > However, even if we create triggers recursively on the parents or children, > we would still > need more consideration. This is because we will have to convert the format > of tuple of > modified table to the format of the table specified in the view

Re: Some RELKIND macro refactoring

2021-11-23 Thread Michael Paquier
On Mon, Nov 22, 2021 at 11:21:52AM +0100, Peter Eisentraut wrote: > On 19.11.21 08:31, Michael Paquier wrote: >> Regarding 0001, I find the existing code a bit more self-documenting >> if we keep those checks flagInhAttrs() and guessConstraintInheritance(). >> So I would rather leave these. > >

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-11-23 Thread Dilip Kumar
On Tue, Nov 23, 2021 at 10:29 PM John Naylor wrote: > > Hi, > > I've looked over this patch set and email thread a couple times, and I don't > see anything amiss, but I'm also not terribly familiar with the subsystems > this part of the code relies on. I haven't yet tried to stress test with a

Re: VS2022: Support Visual Studio 2022 on Windows

2021-11-23 Thread Michael Paquier
On Sun, Nov 21, 2021 at 10:41:17AM +0900, Michael Paquier wrote: > I'll double-check your patch later, but that looks rather good to me. > Will try to apply and back-patch, and it would be better to check the > version numbers assigned in the patch, as well. I have spent a couple of hours on that

Re: Warning in geqo_main.c from clang 13

2021-11-23 Thread Tom Lane
Thomas Munro writes: > Clang 13 on my machine and peripatus (but not Apple clang 13 on eg > sifika, I'm still confused about Apple's versioning but I think that's > really llvm 12-based) warns: > geqo_main.c:86:8: warning: variable 'edge_failures' set but not used > [-Wunused-but-set-variable] >

Re: Skipping logical replication transactions on subscriber side

2021-11-23 Thread Amit Kapila
On Wed, Nov 24, 2021 at 7:51 AM Masahiko Sawada wrote: > > On Thu, Nov 18, 2021 at 12:52 PM houzj.f...@fujitsu.com > wrote: > > > > On Tuesday, November 16, 2021 2:31 PM Masahiko Sawada > > wrote: > > > Right. I've fixed this issue and attached an updated patch. > > > > > Hi, > > > > I have

Python 3.11 vs. Postgres

2021-11-23 Thread Tom Lane
According to [1], we need to stop including Python's . I've not checked whether this creates any backwards-compatibility issues. regards, tom lane [1] https://bugzilla.redhat.com/show_bug.cgi?id=2023272

Re: parallel vacuum comments

2021-11-23 Thread Amit Kapila
On Wed, Nov 24, 2021 at 7:48 AM Masahiko Sawada wrote: > > On Fri, Nov 19, 2021 at 11:25 AM houzj.f...@fujitsu.com > wrote: > > > > 3) > > > > + /* > > +* Reset all index status back to invalid (while checking that we > > have > > +* processed all indexes). > > +*/

Warning in geqo_main.c from clang 13

2021-11-23 Thread Thomas Munro
Hi, Clang 13 on my machine and peripatus (but not Apple clang 13 on eg sifika, I'm still confused about Apple's versioning but I think that's really llvm 12-based) warns: geqo_main.c:86:8: warning: variable 'edge_failures' set but not used [-Wunused-but-set-variable] int

Re: Skipping logical replication transactions on subscriber side

2021-11-23 Thread Masahiko Sawada
On Thu, Nov 18, 2021 at 12:52 PM houzj.f...@fujitsu.com wrote: > > On Tuesday, November 16, 2021 2:31 PM Masahiko Sawada > wrote: > > Right. I've fixed this issue and attached an updated patch. > > > Hi, > > I have few comments for the testcases. > > 1) > > +my $appname = 'tap_sub'; >

Re: parallel vacuum comments

2021-11-23 Thread Masahiko Sawada
On Fri, Nov 19, 2021 at 11:25 AM houzj.f...@fujitsu.com wrote: > > On Tues, Nov 16, 2021 1:53 PM Masahiko Sawada wrote: > > I've incorporated these comments and attached an updated patch. > > Thanks for updating the patch. > I read the latest patch and have few comments. Thank you for the

Re: Mop-up from Test::More version change patch

2021-11-23 Thread Tom Lane
Andrew Dunstan writes: > On 11/23/21 12:03, Tom Lane wrote: >> If you set PERL then it's highly likely that you want to use >> "prove" from the same installation. So 0001 modifies configure >> to first check for an executable "prove" in the same directory >> as $PERL. If that's not what you

Re: parallel vacuum comments

2021-11-23 Thread Masahiko Sawada
On Mon, Nov 22, 2021 at 1:48 PM Amit Kapila wrote: > > On Fri, Nov 19, 2021 at 7:55 AM houzj.f...@fujitsu.com > wrote: > > > > On Tues, Nov 16, 2021 1:53 PM Masahiko Sawada wrote: > > > I've incorporated these comments and attached an updated patch. > > > > > > 2) > > static void

Re: Mop-up from Test::More version change patch

2021-11-23 Thread Andrew Dunstan
On 11/23/21 12:03, Tom Lane wrote: > [ moving thread to -hackers for a bit more visibility ] > > Attached are a couple of patches I propose in the wake of commit > 405f32fc4 (Require version 0.98 of Test::More for TAP tests). > > 0001 responds to the failure we saw on buildfarm member wrasse [1]

Re: parallel vacuum comments

2021-11-23 Thread Masahiko Sawada
On Mon, Nov 22, 2021 at 6:35 PM Amit Kapila wrote: > > On Tue, Nov 16, 2021 at 11:23 AM Masahiko Sawada > wrote: > > > > I've incorporated these comments and attached an updated patch. > > > > Review comments: > > 1. > index_can_participate_parallel_vacuum() > { > .. > + /* > +

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2021-11-23 Thread Andres Freund
Hi, On 2021-11-23 17:01:20 -0800, Peter Geoghegan wrote: > > On reason for my doubt is the following: > > > > We can set all-visible on a page without a FPW image (well, as long as hint > > bits aren't logged). There's a significant difference between needing to WAL > > log FPIs for every heap

RE: row filtering for logical replication

2021-11-23 Thread houzj.f...@fujitsu.com
On Tues, Nov 23, 2021 6:16 PM Amit Kapila wrote: > On Tue, Nov 23, 2021 at 1:29 PM houzj.f...@fujitsu.com > wrote: > > > > On Tues, Nov 23, 2021 2:27 PM vignesh C wrote: > > > On Thu, Nov 18, 2021 at 7:04 AM Peter Smith > > > wrote: > > > > > > > > PSA new set of v40* patches. > > > > > > Few

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2021-11-23 Thread Peter Geoghegan
On Mon, Nov 22, 2021 at 9:49 PM Andres Freund wrote: > > For example, we can definitely afford to wait a few more milliseconds > > to get a cleanup lock just once > > We currently have no infrastructure to wait for an lwlock or pincount for a > limited time. And at least for the former it'd not

Re: XMAX_LOCK_ONLY and XMAX_COMMITTED (fk/multixact code)

2021-11-23 Thread Mark Dilger
> On Nov 23, 2021, at 4:51 PM, Bossart, Nathan wrote: > > This is a good point. Right now, you'd have to manually inspect the > infomask field to determine the exact form of the invalid combination. > My only worry with this is that we'd want to make sure these checks > stayed consistent

Re: XMAX_LOCK_ONLY and XMAX_COMMITTED (fk/multixact code)

2021-11-23 Thread Bossart, Nathan
On 11/23/21, 4:36 PM, "Mark Dilger" wrote: > I have to wonder if, when corruption is reported for conditions like this: > > + if ((ctx->tuphdr->t_infomask & HEAP_XMAX_COMMITTED) && > + HEAP_XMAX_IS_LOCKED_ONLY(ctx->tuphdr->t_infomask)) > > if the first thing we're going to

Re: Windows build warnings

2021-11-23 Thread Greg Nancarrow
On Wed, Nov 24, 2021 at 1:41 AM Alvaro Herrera wrote: > > On 2021-Nov-23, Juan José Santamaría Flecha wrote: > > > On Tue, Nov 23, 2021 at 2:11 PM Daniel Gustafsson wrote: > > > > It's supported in clang as well per the documentation [0] in at least some > > > configurations or distributions: >

Re: XMAX_LOCK_ONLY and XMAX_COMMITTED (fk/multixact code)

2021-11-23 Thread Mark Dilger
> On Nov 23, 2021, at 4:26 PM, Bossart, Nathan wrote: > > I've finally gotten started on this, and I've attached a work-in- > progress patch to gather some early feedback. I'm specifically > wondering if there are other places it'd be good to check for these > unsupported combinations and

Re: XMAX_LOCK_ONLY and XMAX_COMMITTED (fk/multixact code)

2021-11-23 Thread Bossart, Nathan
The archives seem unhappy with my attempt to revive this old thread, so here is a link to it in case anyone is looking for more context: https://www.postgresql.org/message-id/flat/3476708e-7919-20cb-ca45-6603470565f7%40amazon.com Nathan

Re: Sequence's value can be rollback after a crashed recovery.

2021-11-23 Thread Andy Fan
> > > You could say that that's the same rookie error as relying on the > persistence of any other uncommitted DML ... > IIUC, This is not the same as uncommitted DML exactly. For any uncommitted DML, it is a rollback for sure. But as for sequence, The xmax is not changed during sequence's

Re: Failed transaction statistics to measure the logical replication progress

2021-11-23 Thread Masahiko Sawada
On Tue, Nov 23, 2021 at 3:21 PM Greg Nancarrow wrote: > > On Sat, Nov 20, 2021 at 1:11 AM Masahiko Sawada wrote: > > > > I'm concerned that these new names will introduce confusion; if we > > have last_error_relid, last_error_command, last_error_message, > > last_error_time, and last_error_xid,

Re: Failed transaction statistics to measure the logical replication progress

2021-11-23 Thread Masahiko Sawada
On Sat, Nov 20, 2021 at 3:25 PM Amit Kapila wrote: > > On Fri, Nov 19, 2021 at 7:41 PM Masahiko Sawada wrote: > > > > On Thu, Nov 18, 2021 at 12:26 PM Amit Kapila > > wrote: > > > > > > On Wed, Nov 17, 2021 at 7:12 PM osumi.takami...@fujitsu.com > > > wrote: > > > > > > > > On Wednesday,

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2021-11-23 Thread Justin Pryzby
On Mon, Nov 22, 2021 at 07:17:01PM +, Bossart, Nathan wrote: > In an attempt to get this patch set off the ground again, I took a > look at the first 5 patches. > I haven't looked at the following patches too much, but I'm getting > the idea that they might address a lot of the feedback above

Re: Building postgresql armv7 on emulated x86_64

2021-11-23 Thread Marek Kulik
What is weird is that this issue is only present in Fedora Rawhide, older versions of fedora are not affected. I couldn't pinpoint what package update caused that issue. I made a regression for gcc and packages related to it with no luck. It seems to be an issue related to a bug in gcc. Here is

Re: Post-CVE Wishlist

2021-11-23 Thread Heikki Linnakangas
On 23/11/2021 23:44, Robert Haas wrote: On Tue, Nov 23, 2021 at 2:18 PM Tom Lane wrote: Jacob Champion writes: = Implicit TLS = Aside from security, one small benefit of skipping the Starttls-style negotiation is that you avoid one round-trip to the server. I think this idea is a

Re: Support for NSS as a libpq TLS backend

2021-11-23 Thread Joshua Brindle
On Tue, Nov 23, 2021 at 9:12 AM Daniel Gustafsson wrote: > > > On 17 Nov 2021, at 19:42, Joshua Brindle > > wrote: > > On Tue, Nov 16, 2021 at 1:26 PM Joshua Brindle > > wrote: > > >> I think there it a typo in the docs here that prevents them from > >> building (this diff seems to fix it): >

Re: Sequence's value can be rollback after a crashed recovery.

2021-11-23 Thread Bossart, Nathan
On 11/23/21, 1:41 PM, "Tom Lane" wrote: > I wrote: >> I wonder though if we shouldn't try to improve the existing text. >> The phrasing "never rolled back" seems like it's too easily >> misinterpreted. Maybe rewrite the block like >> ... > > A bit of polishing later, maybe like the attached.

Re: Post-CVE Wishlist

2021-11-23 Thread Tom Lane
Robert Haas writes: > I am not persuaded by this argument. Suppose we added a server option > like ssl_port which causes us to listen on an additional port and, on > that port, everything, from the first byte on this connection, is > encrypted using SSL. Right, a separate port number (much akin

Re: Post-CVE Wishlist

2021-11-23 Thread Robert Haas
On Tue, Nov 23, 2021 at 2:18 PM Tom Lane wrote: > Jacob Champion writes: > > = Implicit TLS = > > I think this idea is a nonstarter. It breaks backwards compatibility > at the protocol level in order to fix entirely-hypothetical bugs. > Nobody is going to like that tradeoff. Moreover, how

Re: pg_upgrade parallelism

2021-11-23 Thread Tom Lane
Jacob Champion writes: > Right. What I'm worried about is, if disk space or write performance on > the new cluster is a concern, then having a copy-mode upgrade silently > use copy-on-write could be a problem if the DBA needs copy mode to > actually copy. Particularly for the cross-filesystem

Re: Sequence's value can be rollback after a crashed recovery.

2021-11-23 Thread Tom Lane
I wrote: > I wonder though if we shouldn't try to improve the existing text. > The phrasing "never rolled back" seems like it's too easily > misinterpreted. Maybe rewrite the block like > ... A bit of polishing later, maybe like the attached. regards, tom lane diff

Re: pg_upgrade parallelism

2021-11-23 Thread Jacob Champion
On Tue, 2021-11-23 at 13:51 -0600, Justin Pryzby wrote: > > I guess you're concerned for someone who wants to be able to run pg_upgrade > and > preserve the ability to start the old cluster in addition to the new. Right. What I'm worried about is, if disk space or write performance on the new

Re: Sequence's value can be rollback after a crashed recovery.

2021-11-23 Thread Tom Lane
Tomas Vondra writes: > I see Tom speculated we may not flush WAL if a transaction only does > nextval() in that other thread, but I don't think that's true. AFAICS if > the nextval() call writes stuff to WAL, the RecordTransactionCommit will > have wrote_xlog=true and valid XID. And so it'll do

Re: Sequence's value can be rollback after a crashed recovery.

2021-11-23 Thread Jeremy Schneider
On 11/23/21 05:49, Andy Fan wrote: > > > I think at this thread[1], which claimed to get this issue even after > > commit, I haven't tried it myself though. > > > > [1] > >

Re: Avoiding smgrimmedsync() during nbtree index builds

2021-11-23 Thread Melanie Plageman
On Fri, Nov 19, 2021 at 3:11 PM Melanie Plageman wrote: > > On Mon, May 3, 2021 at 5:24 PM Melanie Plageman > wrote: > > > > So, I've written a patch which avoids doing the immediate fsync for > > index builds either by using shared buffers or by queueing sync requests > > for the checkpointer.

Re: prevent immature WAL streaming

2021-11-23 Thread Alvaro Herrera
On 2021-Nov-23, Tom Lane wrote: > We're *still* not out of the woods with 026_overwrite_contrecord.pl, > as we are continuing to see occasional "mismatching overwritten LSN" > failures, further down in the test where it tries to start up the > standby: Augh. > Looking at adjacent successful

Re: pg_upgrade parallelism

2021-11-23 Thread Justin Pryzby
On Tue, Nov 23, 2021 at 06:54:03PM +, Jacob Champion wrote: > On Wed, 2021-11-17 at 14:34 -0600, Justin Pryzby wrote: > > On Wed, Nov 17, 2021 at 02:44:52PM -0500, Jaime Casanova wrote: > > > > > > - why we read()/write() at all? is not a faster way of copying the file? > > > i'm asking

Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

2021-11-23 Thread Thomas Munro
Here's a patch for Linux and also FreeBSD. The latter OS decided to turn on ASLR by default recently, causing my workstation to fail like this quite reliably, which reminded me to follow up with this. It disables ASLR in pg_ctl and pg_regress, which is enough for check and check-world, but

Re: Post-CVE Wishlist

2021-11-23 Thread Tom Lane
Jacob Champion writes: > = Implicit TLS = I think this idea is a nonstarter. It breaks backwards compatibility at the protocol level in order to fix entirely-hypothetical bugs. Nobody is going to like that tradeoff. Moreover, how shall the server know whether an incoming connection is expected

Re: prevent immature WAL streaming

2021-11-23 Thread Tom Lane
We're *still* not out of the woods with 026_overwrite_contrecord.pl, as we are continuing to see occasional "mismatching overwritten LSN" failures, further down in the test where it tries to start up the standby: sysname |branch | snapshot | stage |

Re: pg_upgrade parallelism

2021-11-23 Thread Jacob Champion
On Wed, 2021-11-17 at 14:34 -0600, Justin Pryzby wrote: > On Wed, Nov 17, 2021 at 02:44:52PM -0500, Jaime Casanova wrote: > > > > - why we read()/write() at all? is not a faster way of copying the file? > > i'm asking that because i don't actually know. > > No portable way. Linux has this: >

Re: xlog.c: removing ReadRecPtr and EndRecPtr

2021-11-23 Thread Robert Haas
On Wed, Nov 17, 2021 at 5:01 PM Robert Haas wrote: > In rescanLatestTimeLine(), the problem is IMHO probably serious enough > to justify a separate commit with back-patching. On second thought, I think it's better not to back-patch this fix. It turns out that, while it's easy enough to

Post-CVE Wishlist

2021-11-23 Thread Jacob Champion
Hello all, Now that the MITM CVEs are published [1], I wanted to share my wishlist of things that would have made those attacks difficult/impossible to pull off. = Implicit TLS = The frontend/backend protocol uses a STARTTLS-style negotiation, which has had a fair number of implementation

Mop-up from Test::More version change patch

2021-11-23 Thread Tom Lane
[ moving thread to -hackers for a bit more visibility ] Attached are a couple of patches I propose in the wake of commit 405f32fc4 (Require version 0.98 of Test::More for TAP tests). 0001 responds to the failure we saw on buildfarm member wrasse [1] where, despite configure having carefully

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-11-23 Thread John Naylor
Hi, I've looked over this patch set and email thread a couple times, and I don't see anything amiss, but I'm also not terribly familiar with the subsystems this part of the code relies on. I haven't yet tried to stress test with a large database, but it seems like a good idea to do so. I have a

Re: row filtering for logical replication

2021-11-23 Thread vignesh C
On Tue, Nov 23, 2021 at 4:58 PM Ajin Cherian wrote: > > Attaching a new patchset v41 which includes changes by both Peter and myself. Few comments on v41-0002 patch: 1) Tab completion should be handled for completion of "WITH(" in "create publication pub1 for table t1 where (c1 > 10)": @@

Re: Reduce function call costs on ELF platforms

2021-11-23 Thread Peter Eisentraut
On 22.11.21 23:32, Tom Lane wrote: The easier approach for this class of issues is to use the linker option -Bsymbolic. I don't recall details, but we've previously rejected the idea of trying to use -Bsymbolic widely; apparently it has undesirable side-effects on some platforms. See commit

Re: [RFC] ASOF Join

2021-11-23 Thread Chapman Flack
On 11/23/21 10:41, Isaac Morland wrote: > Umm, it's definitely negative: > > odyssey=> select '1 month -31 days +12:00:00'::interval < '0 > months'::interval; > -- > t Well, what you've shown here is that it's "negative" according to an arbitrary total ordering imposed in

Re: Granting SET and ALTER SYSTE privileges for GUCs

2021-11-23 Thread Robert Haas
On Mon, Nov 22, 2021 at 7:21 PM Mark Dilger wrote: > There is a new information_schema.guc_privileges view, not present in v2. It's my impression that information_schema is a child of the SQL standard, and that inventions specific to PG go in pg_catalog. Also, I think the user-facing name for

Re: A test for replay of regression tests

2021-11-23 Thread Andrew Dunstan
On 11/23/21 04:07, Thomas Munro wrote: > On Wed, Oct 6, 2021 at 7:10 PM Thomas Munro wrote: >> I wonder if for Windows we'd want to switch to real symlinks, since, >> as far as I know from some light reading, reparse points are converted >> to absolute paths on creation, so the pgdata directory

Re: Feature Proposal: Connection Pool Optimization - Change the Connection User

2021-11-23 Thread Todd Hubers
Hi Jacob and Daniel, Thanks for your feedback. >@Daniel - I think thats conflating session_user and current_user, SET ROLE is not a login event. This is by design and discussed in the documentation.. Agreed, I am using those terms loosely. I have updated option 4 in the proposal document. I

Re: [RFC] ASOF Join

2021-11-23 Thread Isaac Morland
On Tue, 23 Nov 2021 at 09:44, Chapman Flack wrote: > On 11/23/21 02:29, Ilya Anfimov wrote: > > (*We > > strangely don't have an absolute value operator on interval, but > > I think you've got the point*). > > Although tangential to the topic, that might be because a PG interval > is a triple

Re: ResourceOwner refactoring

2021-11-23 Thread Aleksander Alekseev
Hi Heikki, > Attached is a newly rebased version. It includes your tweaks, and a few > more comment and indentation tweaks. v10-0002 rotted a little so I rebased it. The new patch set passed `make installcheck-world` locally, but let's see if cfbot has a second opinion. I'm a bit busy with

Re: Some RELKIND macro refactoring

2021-11-23 Thread Alvaro Herrera
On 2021-Nov-23, Alvaro Herrera wrote: > On 2021-Nov-22, Peter Eisentraut wrote: > > > Maybe > > > > else > > { > > Assert(RELKIND_HAS_STORAGE(rel->rd_rel->relkind); > > RelationCreateStorage(rel->rd_node, relpersistence); > > } > > > > create_storage is set earlier

Re: Some RELKIND macro refactoring

2021-11-23 Thread Alvaro Herrera
On 2021-Nov-22, Peter Eisentraut wrote: > Maybe > > else > { > Assert(RELKIND_HAS_STORAGE(rel->rd_rel->relkind); > RelationCreateStorage(rel->rd_node, relpersistence); > } > > create_storage is set earlier based on RELKIND_HAS_STORAGE(), so this would > be

Re: Some RELKIND macro refactoring

2021-11-23 Thread Alvaro Herrera
On 2021-Nov-19, Michael Paquier wrote: > I think that you should keep this block as it is now. The part where > a relkind does not support table AMs and does not require storage > would get uncovered, and this new code would just attempt to create > storage, so it seems to me that the existing

Re: Should rename "startup process" to something else?

2021-11-23 Thread Tom Lane
Alvaro Herrera writes: > On 2021-Nov-20, Andrew Dunstan wrote: >> Maybe something along those lines but using a dash/hyphen would work: >> e.g. wal-replayer > Yeah, the idea of a dash occurred to me too. Bad memories of uuid-ossp float up ... can we use underscore?

Re: Windows build warnings

2021-11-23 Thread Tom Lane
Alvaro Herrera writes: > Right ... the problem, as I understand, is that the syntax for > [[maybe_unused]] is different from what we can do with the current > pg_attribute_unused -- [[maybe_unused]] goes before the variable name. > We would need to define pg_attribute_unused macro (maybe have it

Re: [RFC] ASOF Join

2021-11-23 Thread Chapman Flack
On 11/23/21 02:29, Ilya Anfimov wrote: > (*We > strangely don't have an absolute value operator on interval, but > I think you've got the point*). Although tangential to the topic, that might be because a PG interval is a triple of independently-signed months/days/seconds components. An interval

Re: Should rename "startup process" to something else?

2021-11-23 Thread Alvaro Herrera
On 2021-Nov-20, Andrew Dunstan wrote: > Maybe something along those lines but using a dash/hyphen would work: > e.g. wal-replayer Yeah, the idea of a dash occurred to me too. -- Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/ "La grandeza es una experiencia

Re: Windows build warnings

2021-11-23 Thread Alvaro Herrera
On 2021-Nov-23, Juan José Santamaría Flecha wrote: > On Tue, Nov 23, 2021 at 2:11 PM Daniel Gustafsson wrote: > > It's supported in clang as well per the documentation [0] in at least some > > configurations or distributions: > [[maybe_unused]] is also recognized from Visual Studio 2017

Re: Windows build warnings

2021-11-23 Thread Juan José Santamaría Flecha
On Tue, Nov 23, 2021 at 2:11 PM Daniel Gustafsson wrote: > > On 22 Nov 2021, at 16:40, Tom Lane wrote: > > > I can't find anything that is providing a non-empty definition of > > PG_USED_FOR_ASSERTS_ONLY (a/k/a pg_attribute_unused) for anything > > except GCC. > > It's supported in clang as

Re: [Patch] ALTER SYSTEM READ ONLY

2021-11-23 Thread Amul Sul
On Wed, Nov 17, 2021 at 6:20 PM Amul Sul wrote: > > On Wed, Nov 17, 2021 at 4:07 PM Amul Sul wrote: > > > > On Wed, Nov 17, 2021 at 11:13 AM Amul Sul wrote: > > > > > > On Sat, Nov 13, 2021 at 2:18 AM Robert Haas wrote: > > > > > > > > On Mon, Nov 8, 2021 at 8:20 AM Amul Sul wrote: > > >

Re: Sequence's value can be rollback after a crashed recovery.

2021-11-23 Thread Andy Fan
> > > > I think at this thread[1], which claimed to get this issue even after > > commit, I haven't tried it myself though. > > > > [1] > https://www.postgresql.org/message-id/flat/ea6485e3-98d0-24a7-094c-87f9d5f9b18f%40amazon.com#4cfe7217c829419b769339465e8c2915 > > > > I did try, and I haven't

Re: logical decoding and replication of sequences

2021-11-23 Thread Tomas Vondra
On 11/23/21 02:01, Andres Freund wrote: Hi, On 2021-09-25 22:05:43 +0200, Hannu Krosing wrote: If our aim is just to make sure that all user-visible data in *transactional* tables is consistent with sequence state then one very much simplified approach to this could be to track the results

Re: removing global variable ThisTimeLineID

2021-11-23 Thread Robert Haas
On Tue, Nov 23, 2021 at 4:36 AM Drouvot, Bertrand wrote: > Indeed, I think the logical decoding on standby patch just needs to > change the Assert in GetWALInsertionTimeLine() to check > SharedRecoveryState is RECOVERY_STATE_DONE or RECOVERY_STATE_ARCHIVE. > > Would that make sense from your

Re: Windows build warnings

2021-11-23 Thread Daniel Gustafsson
> On 22 Nov 2021, at 16:40, Tom Lane wrote: > > Daniel Gustafsson writes: >> Fair enough. Looking at where we use PG_USED_FOR_ASSERTS_ONLY (and where it >> works), these two warnings are the only places where we apply it to a pointer >> typedef (apart from one place where the variable is

Re: row filtering for logical replication

2021-11-23 Thread Amit Kapila
On Tue, Nov 23, 2021 at 4:58 PM Ajin Cherian wrote: > > Attaching a new patchset v41 which includes changes by both Peter and myself. > In 0003 patch, why is below change required? --- a/src/backend/replication/pgoutput/pgoutput.c +++ b/src/backend/replication/pgoutput/pgoutput.c @@ -1,4 +1,4 @@

Re: logical decoding/replication: new functions pg_ls_logicaldir and pg_ls_replslotdir

2021-11-23 Thread Michael Paquier
On Sun, Nov 21, 2021 at 08:45:52AM +0530, Bharath Rupireddy wrote: > I don't think we need to go far to contrib/test_decoding/, even if we > add it there we can't test it for the outputs of these functions, so > I've added the tests in misc_functinos.sql itself. +SELECT COUNT(*) >= 0 AS OK FROM

Re: row filtering for logical replication

2021-11-23 Thread Amit Kapila
On Tue, Nov 23, 2021 at 1:29 PM houzj.f...@fujitsu.com wrote: > > On Tues, Nov 23, 2021 2:27 PM vignesh C wrote: > > On Thu, Nov 18, 2021 at 7:04 AM Peter Smith > > wrote: > > > > > > PSA new set of v40* patches. > > > > Few comments: > > 1) When a table is added to the publication, replica

Re: psql - add SHOW_ALL_RESULTS option

2021-11-23 Thread Daniel Gustafsson
> On 8 Oct 2021, at 14:15, Peter Eisentraut > wrote: > > On 02.10.21 16:31, Fabien COELHO wrote: >>> Attached v9 integrates your tests and makes them work. >> Attached v11 is a rebase. > > This patch still has a few of the problems reported earlier this year. The patch fails to apply and the

Re: A test for replay of regression tests

2021-11-23 Thread Thomas Munro
On Wed, Oct 6, 2021 at 7:10 PM Thomas Munro wrote: > I wonder if for Windows we'd want to switch to real symlinks, since, > as far as I know from some light reading, reparse points are converted > to absolute paths on creation, so the pgdata directory would be less > portable than it would be on

Re: row filtering for logical replication

2021-11-23 Thread Amit Kapila
On Thu, Nov 18, 2021 at 11:02 AM Peter Smith wrote: > > On Mon, Nov 15, 2021 at 9:31 PM Amit Kapila wrote: > > > > > 4. I think we should add some comments in pgoutput_row_filter() as to > > why we are caching the row_filter here instead of > > get_rel_sync_entry()? That has been discussed

RE: row filtering for logical replication

2021-11-23 Thread houzj.f...@fujitsu.com
On Tues, Nov 23, 2021 2:27 PM vignesh C wrote: > On Thu, Nov 18, 2021 at 7:04 AM Peter Smith > wrote: > > > > PSA new set of v40* patches. > > Few comments: > 1) When a table is added to the publication, replica identity is checked. But > while modifying the publish action to include