Re: Timeout failure in 019_replslot_limit.pl

On Fri, Sep 17, 2021 at 06:59:24PM -0300, Alvaro Herrera wrote: > On 2021-Sep-07, Kyotaro Horiguchi wrote: > > It seems like the "kill 'STOP'" in the script didn't suspend the > > processes before advancing WAL. The attached uses 'ps' command to > > check that since I didn't come up with the way

Re: psql: \dl+ to list large objects privileges

The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed Hi, I think this is an interesting patch. +1 I tested it for

Re: Access last_sqlstate from libpq

Daniel Frey writes: > In case of an error when I received a PGresult*, I can access the SQLSTATE by > calling >PGresult* pgresult = ...; >const char* sqlstate = PQresultErrorField( pgresult, PG_DIAG_SQLSTATE ); Right ... > However, this is not possible in a couple of other cases where

Re: improve pg_receivewal code

On Fri, Sep 17, 2021 at 11:46:33AM +0530, Bharath Rupireddy wrote: > Thanks. I changed the code that way. PSA v3 patch. Thanks. Done. -- Michael signature.asc Description: PGP signature

Re: Access last_sqlstate from libpq

On Friday, September 17, 2021, Daniel Frey wrote: > > > Are you sure or are you guessing? > Guessing regarding the implementations of these interfaces. David J.

Re: postgres.h included from relcache.h - but removing it breaks pg_upgrade

Hi, On 2021-09-18 02:51:09 +0300, Alexander Korotkov wrote: > On Tue, Sep 14, 2021 at 6:53 AM Tom Lane wrote: > > Without having looked at the details, I think using a forward-declare > > to avoid including relcache.h in visibilitymap.h might be a reasonably > > non-painful fix. > > I like that

Re: Access last_sqlstate from libpq

> On 18. Sep 2021, at 01:45, David G. Johnston > wrote: > > > > On Friday, September 17, 2021, Daniel Frey wrote: > > However, this is not possible in a couple of other cases where I don't have a > PGresult*, only the PGconn* is available: > > * PQconnectdb (and variants) > > *

Re: postgres.h included from relcache.h - but removing it breaks pg_upgrade

On Tue, Sep 14, 2021 at 6:53 AM Tom Lane wrote: > Andres Freund writes: > > On 2021-09-13 22:40:19 -0400, Tom Lane wrote: > >> As for the fix ... what in the world is pg_upgrade doing including > >> relcache.h? It seems like there's a more fundamental problem here: > >> either relcache.h is

Re: Access last_sqlstate from libpq

On Friday, September 17, 2021, Daniel Frey wrote: > > > However, this is not possible in a couple of other cases where I don't > have a PGresult*, only the PGconn* is available: > > * PQconnectdb (and variants) > > * PQputCopyData > * PQputCopyEnd > * PQgetCopyData > > * lo_* (large object

Access last_sqlstate from libpq

Hi, I am the author of a PostgreSQL C++ client library, taoPQ (https://github.com/taocpp/taopq), wrapping the C-API of libpq. In case of an error when I received a PGresult*, I can access the SQLSTATE by calling PGresult* pgresult = ...; const char* sqlstate = PQresultErrorField(

Re: postgres.h included from relcache.h - but removing it breaks pg_upgrade

On Tue, Sep 14, 2021 at 5:40 AM Tom Lane wrote: > Andres Freund writes: > > I noticed that postgres.h is included from relcache.h (starting in [1]) and > > wanted to fix that - it violates our usual policy against including > > postgres.h > > from within headers. > > Ugh, yeah, that's entirely

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

Hi, I manage a bunch of Postgres servers at Oslo University and we use real ssl certs on all our servers. I was actually really surprised to discover that the libpq default is sslmode=require and that the root cert defaults to a file under the user’s home directory. I have been planning to use

Re: Timeout failure in 019_replslot_limit.pl

On 2021-Sep-07, Kyotaro Horiguchi wrote: > It seems like the "kill 'STOP'" in the script didn't suspend the > processes before advancing WAL. The attached uses 'ps' command to > check that since I didn't come up with the way to do the same in Perl. Ah! so we tell the kernel to send the signal,

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

Hm. Let's Encrypt's FAQ tells me I'm on the right track with that question but the distinctinos are far more coarse than I was worried about: Does Let’s Encrypt issue certificates for anything other than SSL/TLS for websites? Let’s Encrypt certificates are standard Domain Validation

Re: prevent immature WAL streaming

On 2021-Sep-17, Bossart, Nathan wrote: > > That was the first implementation, a few versions of the patch ago. An > > added benefit of a separate WAL record is that you can carry additional > > data for validation, such as -- as suggested by Andres -- the CRC of the > > partial data contained in

Re: prevent immature WAL streaming

On 9/17/21, 1:32 PM, "Alvaro Herrera" wrote: > On 2021-Sep-17, Bossart, Nathan wrote: > >> I gave the patch a read-through. I'm wondering if the >> XLOG_OVERWRITE_CONTRECORD records are actually necessary. IIUC we >> will set XLP_FIRST_IS_ABORTED_PARTIAL on the next page, and >> xlp_pageaddr on

Re: Automatic notification of top transaction IDs

The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed Hello This patch applies fine to master branch and the

Re: Add jsonlog log_destination for JSON server logs

On Thu, Sep 16, 2021 at 9:36 PM Michael Paquier wrote: > I am not really impressed by 0001 and the introduction of > LOG_DESTINATIONS_WITH_FILES. So I would leave that out and keep the > list of destinations listed instead. And we are talking about two > places here, only within syslogger.c. >

Re: Minimal logical decoding on standbys

On Wed, Sep 15, 2021 at 8:36 AM Drouvot, Bertrand wrote: > > Another rebase attached. > > The patch proposal to address Andre's walsender corner cases is still a dedicated commit (as i think it may be easier to discuss). > Did one more battery of tests and everything went well... But doing some

Re: prevent immature WAL streaming

On 2021-Sep-17, Bossart, Nathan wrote: > I gave the patch a read-through. I'm wondering if the > XLOG_OVERWRITE_CONTRECORD records are actually necessary. IIUC we > will set XLP_FIRST_IS_ABORTED_PARTIAL on the next page, and > xlp_pageaddr on that page will already be validated in >

Re: initdb --pwfile /dev/zero

Andres Freund writes: > On 2021-09-17 14:48:42 -0400, Tom Lane wrote: >> I don't think '\0' is the problem. The only fix for this would be to >> re-introduce some fixed limit on how long a line we'll read, which >> I'm not too thrilled about. > Well, '\0' can be classified as the end of a line

Re: initdb --pwfile /dev/zero

Hi, On 2021-09-17 14:48:42 -0400, Tom Lane wrote: > Andres Freund writes: > > A colleague tried PG 14 internally and it failed during cluster creation, > > when > > using the PGDG rpm packages. A bit of debugging shows that the problem is > > that the packaging script specifies the password

Re: right join with partitioned table crash

Justin Pryzby writes: > On Wed, Sep 15, 2021 at 07:53:49PM -0400, Tom Lane wrote: >> Jaime Casanova writes: >>> Here's another crash caught by sqlsmith. >> Fun. Looks like it fails back to v12, but not in v11, >> so it's some optimization we added in v12 that's at fault. > It seems to be a

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

Greg Stark writes: > However I have a different question. Are the system certificates > intended or general purpose certificates? Do they have their intended > uses annotated on the certificates? Does SSL Verification have any > logic deciding which certificates are appropriate for signing

Re: prevent immature WAL streaming

On 9/17/21, 10:35 AM, "Alvaro Herrera" wrote: > On 2021-Sep-17, Bossart, Nathan wrote: > >> On 9/17/21, 9:37 AM, "Alvaro Herrera" wrote: > >> > If you have some time to try your reproducers with this new proposed >> > fix, I would appreciate it. >> >> I haven't had a chance to look at the patch

Re: initdb --pwfile /dev/zero

Andres Freund writes: > A colleague tried PG 14 internally and it failed during cluster creation, when > using the PGDG rpm packages. A bit of debugging shows that the problem is > that the packaging script specifies the password using --pwfile /dev/zero. > In 14+ this turns out to lead to an

initdb --pwfile /dev/zero

Hi, A colleague tried PG 14 internally and it failed during cluster creation, when using the PGDG rpm packages. A bit of debugging shows that the problem is that the packaging script specifies the password using --pwfile /dev/zero. In 14+ this turns out to lead to an endless loop in

Re: prevent immature WAL streaming

On 2021-Sep-17, Bossart, Nathan wrote: > On 9/17/21, 9:37 AM, "Alvaro Herrera" wrote: > > If you have some time to try your reproducers with this new proposed > > fix, I would appreciate it. > > I haven't had a chance to look at the patch yet, but it appears to fix > things with my original

Re: prevent immature WAL streaming

On 2021-Sep-17, Alvaro Herrera wrote: > Added Matsumura-san to CC, because he was interested in this topic too > per the earlier thread. I failed to do this, so hopefully this serves as a ping. -- Álvaro Herrera 39°49'30"S 73°17'W — https://www.EnterpriseDB.com/

Re: prevent immature WAL streaming

On 9/17/21, 9:37 AM, "Alvaro Herrera" wrote: > OK, this version is much more palatable, because here we verify that the > OVERWRITE_CONTRECORD we replay matches the record that was lost. Also, > I wrote a test script that creates such a broken record (by the simple > expedient of deleting the

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

On Tue, 7 Sept 2021 at 12:59, Tom Lane wrote: > > I guess what it > comes down to is whether you think that public or private certs are > likely to be the majority use-case in the long run. The shortage of > previous requests for this feature says that right now, just about > everyone is using

Re: prevent immature WAL streaming

OK, this version is much more palatable, because here we verify that the OVERWRITE_CONTRECORD we replay matches the record that was lost. Also, I wrote a test script that creates such a broken record (by the simple expedient of deleting the WAL file containing the second half while the server is

Re: POC: Cleaning up orphaned files using undo logs

> On Tue, Sep 14, 2021 at 10:51:42AM +0200, Antonin Houska wrote: > Antonin Houska wrote: > > > Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > > * By throwing at the patchset `make installcheck` I'm getting from time > > > to time > > > and error on the restart: > > > > > > TRAP:

Re: Logical replication timeout problem

the publisher and the subscriber run on the same postgres instance. Regards, Fabrice On Fri, Sep 17, 2021 at 12:26 PM Amit Kapila wrote: > On Fri, Sep 17, 2021 at 3:29 PM Fabrice Chapuis > wrote: > > > > Hi, > > > > Logical replication is configured on one instance in version 10.18. > Timeout

Re: proposal: possibility to read dumped table's name from file

> > > > Initially not, but now, when I am thinking about it, I don't think so > Bison helps. The syntax of the filter file is nicely linear. Now, the code > of the parser is a little bit larger than minimalistic, but it is due to > nicer error's messages. The raw implementation in Bison raised

Re: proposal: possibility to read dumped table's name from file

pá 17. 9. 2021 v 14:07 odesílatel Daniel Gustafsson napsal: > > On 17 Sep 2021, at 13:59, Pavel Stehule wrote: > > pá 17. 9. 2021 v 13:56 odesílatel Daniel Gustafsson > napsal: > > > On 17 Sep 2021, at 13:51, Pavel Stehule > wrote: > > >

Re: Added schema level support for publication.

On Thu, Sep 16, 2021 at 11:24 AM Amit Kapila wrote: > > On Wed, Sep 15, 2021 at 4:45 PM Greg Nancarrow wrote: > > > > On Tue, Sep 14, 2021 at 6:38 PM vignesh C wrote: > > > > > > I have handled this in the patch attached. > > > > > > > Regarding the following function in the v28-0002 patch: > >

Re: Added schema level support for publication.

On Thu, Sep 16, 2021 at 9:54 AM Amit Kapila wrote: > > On Wed, Sep 15, 2021 at 12:30 PM Amit Kapila wrote: > > > > On Tue, Sep 14, 2021 at 2:08 PM vignesh C wrote: > > > > > > I have handled this in the patch attached. > > > > > > > 4. > > AlterPublicationSchemas() > > { > > .. > > + /* > > + *

Re: proposal: possibility to read dumped table's name from file

Greetings, On Fri, Sep 17, 2021 at 14:07 Daniel Gustafsson wrote: > > On 17 Sep 2021, at 13:59, Pavel Stehule wrote: > > pá 17. 9. 2021 v 13:56 odesílatel Daniel Gustafsson > napsal: > > > On 17 Sep 2021, at 13:51, Pavel Stehule >

Re: proposal: possibility to read dumped table's name from file

> On 17 Sep 2021, at 13:59, Pavel Stehule wrote: > pá 17. 9. 2021 v 13:56 odesílatel Daniel Gustafsson > napsal: > > On 17 Sep 2021, at 13:51, Pavel Stehule > > wrote: > > pá 17. 9. 2021 v 13:42 odesílatel Daniel Gustafsson >

Re: proposal: possibility to read dumped table's name from file

Greetings, On Fri, Sep 17, 2021 at 13:59 Pavel Stehule wrote: > > > pá 17. 9. 2021 v 13:56 odesílatel Daniel Gustafsson > napsal: > >> > On 17 Sep 2021, at 13:51, Pavel Stehule >> wrote: >> > pá 17. 9. 2021 v 13:42 odesílatel Daniel Gustafsson > > napsal: >> >> > I am

Re: proposal: possibility to read dumped table's name from file

pá 17. 9. 2021 v 13:56 odesílatel Daniel Gustafsson napsal: > > On 17 Sep 2021, at 13:51, Pavel Stehule wrote: > > pá 17. 9. 2021 v 13:42 odesílatel Daniel Gustafsson > napsal: > > > I am unable to write a filter statement which can > > handle this relname: > > > >

Re: Added schema level support for publication.

On Thu, Sep 16, 2021 at 8:59 AM houzj.f...@fujitsu.com wrote: > > On Tuesday, September 14, 2021 4:39 PM vignesh C wrote: > > > > I have handled this in the patch attached. > > Thanks for updating the patch. > Here are some comments. > > 1) > +static void >

Re: proposal: possibility to read dumped table's name from file

> On 17 Sep 2021, at 13:51, Pavel Stehule wrote: > pá 17. 9. 2021 v 13:42 odesílatel Daniel Gustafsson > napsal: > I am unable to write a filter statement which can > handle this relname: > > CREATE TABLE "a"" > ""b" (a integer); > > Are you able to craft one for that?

Re: proposal: possibility to read dumped table's name from file

pá 17. 9. 2021 v 13:42 odesílatel Daniel Gustafsson napsal: > > On 15 Sep 2021, at 19:31, Pavel Stehule wrote: > > po 13. 9. 2021 v 15:01 odesílatel Daniel Gustafsson > napsal: > > > One issue with this syntax is that the include keyword can be quite > misleading > > as

Re: proposal: possibility to read dumped table's name from file

Hi > A main concern among most (all?) participants of the thread, regardless of > format supported, is that quoting is hard and must be done right for all > object > names postgres support (including any not currently in scope by this > patch). > > Just a small note - when quoting is calculated

Re: proposal: possibility to read dumped table's name from file

> On 15 Sep 2021, at 19:31, Pavel Stehule wrote: > po 13. 9. 2021 v 15:01 odesílatel Daniel Gustafsson > napsal: > One issue with this syntax is that the include keyword can be quite misleading > as it's semantic interpretion of "include table t" can be different from >

Re: proposal: possibility to read dumped table's name from file

As there have been a lot of differing opinions raised in this thread, I re-read it and tried to summarize the discussion so far to try and figure out where we agree and on what (and disagree) before we get deep into the technicalities wrt the current patch. If anyone feel I've misrepresented them

Re: [BUG] Unexpected action when publishing partition tables

On Fri, Sep 17, 2021 at 11:36 AM houzj.f...@fujitsu.com wrote: > > On Thursday, September 16, 2021 6:05 PM Amit Kapila > > > On Tuesday, September 14, 2021 10:41 PM vignesh C > > > wrote: > > > > On Tue, Sep 7, 2021 at 11:38 AM houzj.f...@fujitsu.com > > > > wrote: > > > > > > Thanks for the

Re: Logical replication timeout problem

On Fri, Sep 17, 2021 at 3:29 PM Fabrice Chapuis wrote: > > Hi, > > Logical replication is configured on one instance in version 10.18. Timeout > errors occur regularly and the worker process exit with an exit code 1 > > 2021-09-16 12:06:50 CEST [24881]: [1-1] user=postgres,db=foo,client=[local]

Re: walsender timeout on logical replication set

On Fri, Sep 17, 2021 at 12:48 PM Kyotaro Horiguchi wrote: > > Thank you vary much for coming in! > > At Fri, 17 Sep 2021 10:18:11 +0530, Amit Kapila > wrote in > > On Mon, Sep 13, 2021 at 7:01 AM Kyotaro Horiguchi > > wrote: > > > > > > Hello. > > > > > > As reported in [1] it seems that

Re: [PATCH] psql: \dn+ to show size of each schema (and \dA+ for AMs)

pá 17. 9. 2021 v 11:10 odesílatel Justin Pryzby napsal: > On Wed, Jul 14, 2021 at 07:42:33AM +0200, Laurenz Albe wrote: > > Besides, schemas are not physical, but logical containers. So I see a > point in > > measuring the storage used in a certain tablespace, but not so much by > all objects >

Logical replication timeout problem

Hi, Logical replication is configured on one instance in version 10.18. Timeout errors occur regularly and the worker process exit with an exit code 1 2021-09-16 12:06:50 CEST [24881]: [1-1] user=postgres,db=foo,client=[local] LOG: duration: 1281408.171 ms statement: COPY schem.tab (col1,

Re: Logical replication keepalive flood

On Thu, Sep 16, 2021 at 10:36 PM Amit Kapila wrote: > > I think here the reason is that the first_lsn of a transaction is > always equal to end_lsn of the previous transaction (See comments > above first_lsn and end_lsn fields of ReorderBufferTXN). That may be the case, but those comments

Re: Logical replication keepalive flood

On Fri, Sep 17, 2021 at 12:42 PM Peter Smith wrote: > > On Thu, Sep 16, 2021 at 10:59 AM houzj.f...@fujitsu.com > wrote: > > Hello Hous-san, thanks for including the steps. Unfortunately, no > matter what I tried, I could never get the patch to display the > problem "BEGIN 709" for the 2nd time

Re: Correct handling of blank/commented lines in PSQL interactive-mode history

On 07.09.21 21:16, Tom Lane wrote: [ this is a digression from the main point of the thread, but ... ] Alvaro Herrera writes: I am particularly bothered by the uselessness that M-# results in -- namely, inserting a # at the start of the buffer. Fixing that might be as simple as the

Re: Teach pg_receivewal to use lz4 compression

‐‐‐ Original Message ‐‐‐ On Friday, September 17th, 2021 at 09:39, Michael Paquier wrote: > On Thu, Sep 16, 2021 at 03:17:15PM +, gkokola...@pm.me wrote: > > > Hopefully fixed. > > Thanks for the new version. I have put my hands on the patch, and > began reviewing its internals

Re: Patch to avoid orphaned dependencies

Hello Bertrand, Le mardi 4 mai 2021, 11:55:43 CEST Drouvot, Bertrand a écrit : > > Implementation overview: > > * A new catalog snapshot is added: DirtyCatalogSnapshot. > * This catalog snapshot is a dirty one to be able to look for > in-flight dependencies. > * Its usage is

Re: Teach pg_receivewal to use lz4 compression

On Thu, Sep 16, 2021 at 03:17:15PM +, gkokola...@pm.me wrote: > Hopefully fixed. Thanks for the new version. I have put my hands on the patch, and began reviewing its internals with LZ4. I am not done with it yet, and I have noticed some places that could be improved (error handling, some

Re: walsender timeout on logical replication set

Thank you vary much for coming in! At Fri, 17 Sep 2021 10:18:11 +0530, Amit Kapila wrote in > On Mon, Sep 13, 2021 at 7:01 AM Kyotaro Horiguchi > wrote: > > > > Hello. > > > > As reported in [1] it seems that walsender can suffer timeout in > > certain cases. It is not clearly confirmed, but

Re: Logical replication keepalive flood

On Thu, Sep 16, 2021 at 10:59 AM houzj.f...@fujitsu.com wrote: > > From Tuesday, September 14, 2021 1:39 PM Greg Nancarrow > wrote: > > However, the problem I found is that, with the patch applied, there is > > a test failure when running “make check-world”: > > > > t/006_logical_decoding.pl

Re: Refactoring postgres_fdw code to rollback remote transaction

On Fri, Sep 17, 2021 at 8:28 AM Fujii Masao wrote: > > On 2021/09/17 11:40, Zhihong Yu wrote: > > + goto fail; /* Trouble clearing prepared statements */ > > > > The label fail can be removed. Under the above condition, > > entry->changing_xact_state is still true. You can

RE: Logical replication keepalive flood

On Thursday, September 16, 2021 8:36 PM Amit Kapila : > On Thu, Sep 16, 2021 at 6:29 AM houzj.f...@fujitsu.com > wrote: > > > > After applying the patch, > > I saw the same problem and can reproduce it by the following steps: > > > > 1) execute the SQLs. > > ---SQL--- > > CREATE

Re: POC: Cleaning up orphaned files using undo logs

On Thu, Sep 9, 2021 at 8:33 PM Antonin Houska wrote: > > The cfbot complained that the patch series no longer applies, so I've rebased > it and also tried to make sure that the other flags become green. > > One particular problem was that pg_upgrade complained that "live undo data" > remains in

Re: improve pg_receivewal code

On Thu, Sep 16, 2021 at 9:31 AM Michael Paquier wrote: > > > Here's the v2 with above modifications. > > I was looking at this patch, and I agree that checking for the system > ID and the timeline by setting sysidentifier beforehand looks like an > improvement. > > The extra IDENTIFY_SYSTEM done

RE: [BUG] Unexpected action when publishing partition tables

On Thursday, September 16, 2021 6:05 PM Amit Kapila > > On Tuesday, September 14, 2021 10:41 PM vignesh C > > wrote: > > > On Tue, Sep 7, 2021 at 11:38 AM houzj.f...@fujitsu.com > > > wrote: > > > > Thanks for the comment. > > Attach new version patches which clean the table at the end. > > >