Re: pg13: xlogreader API adjust

2020-05-15 Thread Michael Paquier
On Thu, May 14, 2020 at 02:12:25PM +0900, Kyotaro Horiguchi wrote: > Good catch! That's not only for CreateDecodingContet. That happens > everywhere in the query loop in PostgresMain() until logreader is > initialized. So that also happens, for example, by starting logical > replication using

Re: ldap tls test fails in some environments

2020-05-15 Thread Christoph Berg
> I'll see if I can catch a shell in the environment where it fails. It failed right away when I tried on the buildd machine: The slapd debug log is mostly garbage to me, the error seems to be this: ldap_read: want=8 error=Resource temporarily unavailable src/test/ldap/t/001_auth.pl:

Re: pg_bsd_indent and -Wimplicit-fallthrough

2020-05-15 Thread Julien Rouhaud
On Fri, May 15, 2020 at 9:17 AM Daniel Gustafsson wrote: > > > On 15 May 2020, at 08:28, Julien Rouhaud wrote: > > On Fri, May 15, 2020 at 8:03 AM Michael Paquier wrote: > > >> Something like the attached is fine to take care of those warnings, > >> but what's our current patching policy for

Re: PG 13 release notes, first draft

2020-05-15 Thread Bruce Momjian
On Fri, May 15, 2020 at 03:55:19PM +0900, Fujii Masao wrote: > > > On 2020/05/05 12:16, Bruce Momjian wrote: > > I have committed the first draft of the PG 13 release notes. You can > > see them here: > > > > https://momjian.us/pgsql_docs/release-13.html > > > > It still needs markup,

Re: documenting the backup manifest file format

2020-05-15 Thread David Steele
On 5/15/20 9:34 AM, Tom Lane wrote: Robert Haas writes: It's a good question. My inclination was to think that GMT would be the clearest thing, but I also didn't realize that the result would thus be inconsistent with backup_label. Not sure what's best here. I vote for following the

Re: documenting the backup manifest file format

2020-05-15 Thread Tom Lane
David Steele writes: > On 5/15/20 9:34 AM, Tom Lane wrote: >> I vote for following the backup_label precedent; that's stood for quite >> some years now. > Of course, my actual preference is to use epoch time which is easy to > work with and eliminates the possibility of conversion errors. It is

Re: PG 13 release notes, first draft

2020-05-15 Thread Fujii Masao
On 2020/05/15 21:29, Bruce Momjian wrote: On Fri, May 15, 2020 at 03:55:19PM +0900, Fujii Masao wrote: On 2020/05/05 12:16, Bruce Momjian wrote: I have committed the first draft of the PG 13 release notes. You can see them here: https://momjian.us/pgsql_docs/release-13.html It

Re: PG 13 release notes, first draft

2020-05-15 Thread Bruce Momjian
On Fri, May 15, 2020 at 09:54:55PM +0900, Fujii Masao wrote: > > Actually, it should be: > > > > > > > > because we are using the text from the link. > > Yes, this works. > > > See > > doc/src/sgml/README.links for details on xref links. Release notes > > updated. > > Thanks! > > >

Re: Parallel copy

2020-05-15 Thread Robert Haas
On Fri, May 15, 2020 at 12:19 AM Amit Kapila wrote: > > My sense is that it would be a lot more sensible to do it at the > > *beginning* of the parallel operation. Once we do it once, we > > shouldn't ever do it again; that's how it works now. Deferring it > > until later seems much more likely

Re: documenting the backup manifest file format

2020-05-15 Thread Tom Lane
Robert Haas writes: > It's a good question. My inclination was to think that GMT would be > the clearest thing, but I also didn't realize that the result would > thus be inconsistent with backup_label. Not sure what's best here. I vote for following the backup_label precedent; that's stood for

Re: ldap tls test fails in some environments

2020-05-15 Thread Tom Lane
Christoph Berg writes: > The slapd debug log is mostly garbage to me, the error seems to be > this: > ldap_read: want=8 error=Resource temporarily unavailable Hm, so EAGAIN (although that's a BSD-ish spelling of the strerror string, which seems pretty odd in a Debian context). I don't think

Re: Transactions involving multiple postgres foreign servers, take 2

2020-05-15 Thread Masahiko Sawada
On Fri, 15 May 2020 at 19:06, Muhammad Usama wrote: > > > > On Fri, May 15, 2020 at 9:59 AM Masahiko Sawada > wrote: >> >> On Fri, 15 May 2020 at 13:26, Muhammad Usama wrote: >> > >> > >> > >> > On Fri, May 15, 2020 at 7:20 AM Masahiko Sawada >> > wrote: >> >> >> >> On Fri, 15 May 2020 at

Re: ldap tls test fails in some environments

2020-05-15 Thread Christoph Berg
Re: Tom Lane > Somebody should get out the LDAP RFCs and decode the packet contents > that this log helpfully provides, but I suspect that we're just looking > at an authentication failure; there's still not much clue as to why. The non-TLS tests work, so it's not a plain auth failure... I'm

Re: Problem with logical replication

2020-05-15 Thread Euler Taveira
On Fri, 15 May 2020 at 02:47, Michael Paquier wrote: > > Agreed. I don't think either that we need to update this comment. I > was playing with this patch and what you have here looks fine by me. > Two nits: the extra parenthesis in the assert are not necessary, and > the indentation had some

Re: Index Skip Scan

2020-05-15 Thread Dmitry Dolgov
> On Wed, May 13, 2020 at 02:37:21PM +0530, Dilip Kumar wrote: > > + if (_bt_scankey_within_page(scan, so->skipScanKey, so->currPos.buf, dir)) > + { > > Here we expect whether the "next" unique key can be found on this page > or not, but we are using the function which suggested whether the >

Re: documenting the backup manifest file format

2020-05-15 Thread Robert Haas
On Fri, May 15, 2020 at 2:10 AM Fujii Masao wrote: > I have one question related to this; Why don't we use log_timezone, > like backup_label? log_timezone is used for "START TIME" field in > backup_label. Sorry if this was already discussed. > > /* Use the log timezone here, not

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-05-15 Thread Dilip Kumar
On Fri, May 15, 2020 at 4:35 PM Amit Kapila wrote: > > On Fri, May 15, 2020 at 4:20 PM Dilip Kumar wrote: > > > > On Fri, May 15, 2020 at 4:04 PM Amit Kapila wrote: > > > > > > > > > > > > - In the example we can not show a real example, because of the > > > > in-progress transaction to show

Re: documenting the backup manifest file format

2020-05-15 Thread David Steele
On 5/15/20 10:17 AM, Tom Lane wrote: David Steele writes: On 5/15/20 9:34 AM, Tom Lane wrote: I vote for following the backup_label precedent; that's stood for quite some years now. Of course, my actual preference is to use epoch time which is easy to work with and eliminates the

Re: Add A Glossary

2020-05-15 Thread Alvaro Herrera
Applied all these suggestions, and made a few additional very small edits, and pushed -- better to ship what we have now in beta1, but further edits are still possible. Other possible terms to define, including those from the tweet I linked to and a couple more: archive availability backup

Re: pg_stat_wal_receiver and flushedUpto/writtenUpto

2020-05-15 Thread Alvaro Herrera
On 2020-May-15, Michael Paquier wrote: > As discussed in the thread that introduced d140f2f3 to rename > receivedUpto to flushedUpto and add writtenUpto to the WAL receiver's > shared memory information, the equivalent columns in > pg_stat_wal_receiver have not been renamed: > When I have

Spawned Background Process Knows the Exit of Client Process?

2020-05-15 Thread Shichao Jin
Hi Postgres Hackers, I am wondering is there any elegant way for self-spawned background process (forked by us) to get notified when the regular client-connected process exit from the current database (switch db or even terminate)? The background is that we are integrating a thread-model based

Re: Transactions involving multiple postgres foreign servers, take 2

2020-05-15 Thread Muhammad Usama
On Fri, May 15, 2020 at 7:52 PM Masahiko Sawada < masahiko.saw...@2ndquadrant.com> wrote: > On Fri, 15 May 2020 at 19:06, Muhammad Usama wrote: > > > > > > > > On Fri, May 15, 2020 at 9:59 AM Masahiko Sawada < > masahiko.saw...@2ndquadrant.com> wrote: > >> > >> On Fri, 15 May 2020 at 13:26,

Re: Potentially misleading name of libpq pass phrase hook

2020-05-15 Thread Magnus Hagander
On Thu, May 14, 2020 at 3:03 PM Daniel Gustafsson wrote: > Reviewing TLS changes for v13 I came across one change which I think might > be > better off with a library qualified name. The libpq frontend sslpassword > hook > added in 4dc63552109f65 is OpenSSL specific, but it has a very generic >

Re: pgindent && weirdness

2020-05-15 Thread Alvaro Herrera
On 2020-May-16, Thomas Munro wrote: > Here's the patch I propose to commit to pg_bsd_indent, if the repo > lets me, and here's the result of running it on the PG tree today. Looks good. Of all these changes in PG, only two are of the STACK_OK() nature, and there are 38 places that get better.

Re: pgindent && weirdness

2020-05-15 Thread Tom Lane
Thomas Munro writes: > Here's the patch I propose to commit to pg_bsd_indent, if the repo > lets me, and here's the result of running it on the PG tree today. +1. I think the repo will let you in, but if not, I can do it. regards, tom lane

Re: calling procedures is slow and consumes extra much memory against calling function

2020-05-15 Thread Pavel Stehule
Hi > The problem is in plpgsql implementation of CALL statement >> >> In non atomic case - case of using procedures from DO block, the >> expression plan is not cached, and plan is generating any time. This is >> reason why it is slow. >> >> Unfortunately, generated plans are not released until

Re: Multiple FPI_FOR_HINT for the same block during killing btree index items

2020-05-15 Thread Alvaro Herrera
On 2020-Apr-10, Masahiko Sawada wrote: > Okay. I think only adding the check would also help with reducing the > likelihood. How about the changes for the current HEAD I've attached? Pushed this to all branches. (Branches 12 and older obviously needed an adjustment.) Thanks! > Related to this

[PATCH] Fix pg_dump --no-tablespaces for the custom format

2020-05-15 Thread Christopher Baines
Hey, So I'm new to poking around in the PostgreSQL code, so this is a bit of a shot in the dark. I'm having some problems with pg_dump, and a database with tablespaces. A couple of the tables are not in the default tablespace, and I want to ignore this for the dump. Looking at the pg_dump

Re: pgindent && weirdness

2020-05-15 Thread Thomas Munro
On Tue, Feb 18, 2020 at 12:42 PM Tom Lane wrote: > Thomas Munro writes: > > Another problem is that there is one thing in our tree that looks like > > a non-cast under the new rule, but it actually expands to a type name, > > so now we get that wrong! (I mean, unpatched indent doesn't really >

Re: Potentially misleading name of libpq pass phrase hook

2020-05-15 Thread Alvaro Herrera
On 2020-May-15, Magnus Hagander wrote: > On Thu, May 14, 2020 at 3:03 PM Daniel Gustafsson wrote: > > Since we haven't shipped this there is still time to rename, which > > IMO is the right way forward. PQsslKeyPassHook__type would > > be one option, but perhaps there are better alternatives?

Re: pgindent && weirdness

2020-05-15 Thread Thomas Munro
On Sat, May 16, 2020 at 10:15 AM Tom Lane wrote: > Thomas Munro writes: > > Here's the patch I propose to commit to pg_bsd_indent, if the repo > > lets me, and here's the result of running it on the PG tree today. > > +1. I think the repo will let you in, but if not, I can do it. It seems I

Re: calling procedures is slow and consumes extra much memory against calling function

2020-05-15 Thread Ranier Vilela
Em sáb., 16 de mai. de 2020 às 00:07, Pavel Stehule escreveu: > > > so 16. 5. 2020 v 0:34 odesílatel Ranier Vilela > napsal: > >> Em dom., 10 de mai. de 2020 às 17:21, Pavel Stehule < >> pavel.steh...@gmail.com> escreveu: >> >>> Hi >>> >>> I try to use procedures in Orafce package, and I did

Re: pgindent && weirdness

2020-05-15 Thread Tom Lane
Thomas Munro writes: > On Sat, May 16, 2020 at 10:15 AM Tom Lane wrote: >> +1. I think the repo will let you in, but if not, I can do it. > It seems I cannot. Please go ahead. [ yawn... ] It's about bedtime here, but I'll take care of it in the morning. Off the critical path, we oughta

Re: Potentially misleading name of libpq pass phrase hook

2020-05-15 Thread Jonathan S. Katz
On 5/15/20 8:21 PM, Tom Lane wrote: > Magnus Hagander writes: >> On Thu, May 14, 2020 at 3:03 PM Daniel Gustafsson wrote: >>> Since we haven't shipped this there is still time to rename, which IMO >>> is the right way forward. PQsslKeyPassHook__type would be one >>> option, but perhaps there

Re: PostgreSQL 13 Beta 1 Release: 2020-05-21

2020-05-15 Thread Jonathan S. Katz
On 4/24/20 12:27 PM, Jonathan S. Katz wrote: > Hi, > > The PostgreSQL 13 Release Management Team is pleased to announce that > the release date for PostgreSQL 13 Beta 1 is set to be 2020-05-21. The > Open Items page is updated to reflect this. > > We’re excited to make the Beta available for

Re: PostgreSQL 13 Beta 1 Release: 2020-05-21

2020-05-15 Thread Tom Lane
"Jonathan S. Katz" writes: > Just a reminder that the Beta 1 release is this upcoming Thursday. The > Open Items list is pretty small at this point, but if you are planning > to commit anything before the release, please be sure to do so over the > weekend so we can wrap on Monday. Pursuant to

Re: Potentially misleading name of libpq pass phrase hook

2020-05-15 Thread Michael Paquier
On Fri, May 15, 2020 at 09:21:52PM -0400, Jonathan S. Katz wrote: > +1 on all of the above. > > I noticed this has been added to Open Items; I added a note that the > plan is to fix before the Beta 1 wrap. +1. Thanks. Agreed. PQsslKeyPassHook__type sounds fine to me as convention. Wouldn't

Re: pg_stat_wal_receiver and flushedUpto/writtenUpto

2020-05-15 Thread Michael Paquier
On Fri, May 15, 2020 at 01:43:11PM -0400, Alvaro Herrera wrote: > Why do you put the column at the end? I would put written_lsn before > flushed_lsn. Fine by me. I was thinking yesterday about putting the written position after the flushed one, and finished with something that maps with the

Re: pg_stat_wal_receiver and flushedUpto/writtenUpto

2020-05-15 Thread Alvaro Herrera
On 2020-May-16, Michael Paquier wrote: > On Fri, May 15, 2020 at 01:43:11PM -0400, Alvaro Herrera wrote: > > Why do you put the column at the end? I would put written_lsn before > > flushed_lsn. > > Fine by me. I was thinking yesterday about putting the written > position after the flushed

Re: [PATCH] Fix pg_dump --no-tablespaces for the custom format

2020-05-15 Thread Tom Lane
Christopher Baines writes: > So I'm new to poking around in the PostgreSQL code, so this is a bit of > a shot in the dark. I'm having some problems with pg_dump, and a > database with tablespaces. A couple of the tables are not in the default > tablespace, and I want to ignore this for the dump.

Re: calling procedures is slow and consumes extra much memory against calling function

2020-05-15 Thread Pavel Stehule
so 16. 5. 2020 v 0:34 odesílatel Ranier Vilela napsal: > Em dom., 10 de mai. de 2020 às 17:21, Pavel Stehule < > pavel.steh...@gmail.com> escreveu: > >> Hi >> >> I try to use procedures in Orafce package, and I did some easy >> performance tests. I found some hard problems: >> >> 1. test case >>

Re: calling procedures is slow and consumes extra much memory against calling function

2020-05-15 Thread Pavel Stehule
so 16. 5. 2020 v 5:55 odesílatel Ranier Vilela napsal: > Em sáb., 16 de mai. de 2020 às 00:07, Pavel Stehule < > pavel.steh...@gmail.com> escreveu: > >> >> >> so 16. 5. 2020 v 0:34 odesílatel Ranier Vilela >> napsal: >> >>> Em dom., 10 de mai. de 2020 às 17:21, Pavel Stehule < >>>

Re: Index Skip Scan

2020-05-15 Thread Dilip Kumar
On Fri, 15 May 2020 at 6:06 PM, Dmitry Dolgov <9erthali...@gmail.com> wrote: > > On Wed, May 13, 2020 at 02:37:21PM +0530, Dilip Kumar wrote: > > > > + if (_bt_scankey_within_page(scan, so->skipScanKey, so->currPos.buf, > dir)) > > + { > > > > Here we expect whether the "next" unique key can be

Re: calling procedures is slow and consumes extra much memory against calling function

2020-05-15 Thread Ranier Vilela
Em dom., 10 de mai. de 2020 às 17:21, Pavel Stehule escreveu: > Hi > > I try to use procedures in Orafce package, and I did some easy performance > tests. I found some hard problems: > > 1. test case > > create or replace procedure p1(inout r int, inout v int) as $$ > begin v := random() * r;

Re: pg13: xlogreader API adjust

2020-05-15 Thread Alvaro Herrera
On 2020-May-15, Michael Paquier wrote: > On Thu, May 14, 2020 at 02:12:25PM +0900, Kyotaro Horiguchi wrote: > > Good catch! That's not only for CreateDecodingContet. That happens > > everywhere in the query loop in PostgresMain() until logreader is > > initialized. So that also happens, for

Re: pgindent && weirdness

2020-05-15 Thread Tom Lane
Alvaro Herrera writes: > On 2020-May-16, Thomas Munro wrote: >> Here's the patch I propose to commit to pg_bsd_indent, if the repo >> lets me, and here's the result of running it on the PG tree today. > Looks good. Of all these changes in PG, only two are of the STACK_OK() > nature, and there

Re: Potentially misleading name of libpq pass phrase hook

2020-05-15 Thread Tom Lane
Magnus Hagander writes: > On Thu, May 14, 2020 at 3:03 PM Daniel Gustafsson wrote: >> Since we haven't shipped this there is still time to rename, which IMO >> is the right way forward. PQsslKeyPassHook__type would be one >> option, but perhaps there are better alternatives? > ISTM this should

Re: pg_stat_wal_receiver and flushedUpto/writtenUpto

2020-05-15 Thread Michael Paquier
On Fri, May 15, 2020 at 07:34:46PM -0400, Alvaro Herrera wrote: > IIRC the only reason to put the written LSN where it is is so that it's > below the mutex, to indicate it's not protected by it. Conceptually, > the written LSN is "before" the flushed LSN, which is why I propose to > put it ahead

Re: documenting the backup manifest file format

2020-05-15 Thread Fujii Masao
On 2020/04/15 5:33, Andrew Dunstan wrote: On 4/14/20 4:09 PM, Alvaro Herrera wrote: On 2020-Apr-14, Andrew Dunstan wrote: OK, but I think if we're putting a timestamp string in ISO-8601 format in the manifest it should be in UTC / Zulu time, precisely to avoid these issues. If that's too

Re: pg_bsd_indent and -Wimplicit-fallthrough

2020-05-15 Thread Julien Rouhaud
On Fri, May 15, 2020 at 8:03 AM Michael Paquier wrote: > > Hi, > > I have just noticed that pg_bsd_indent complains since > -Wimplicit-fallthrough=3 has been added to the default set of switches > if available. Oh Indeed. > Something like the attached is fine to take care of those warnings, >

pg_bsd_indent and -Wimplicit-fallthrough

2020-05-15 Thread Michael Paquier
Hi, I have just noticed that pg_bsd_indent complains since -Wimplicit-fallthrough=3 has been added to the default set of switches if available. Something like the attached is fine to take care of those warnings, but what's our current patching policy for this tool? Thanks, -- Michael diff --git

Re: pg_bsd_indent and -Wimplicit-fallthrough

2020-05-15 Thread Daniel Gustafsson
> On 15 May 2020, at 08:28, Julien Rouhaud wrote: > On Fri, May 15, 2020 at 8:03 AM Michael Paquier wrote: >> Something like the attached is fine to take care of those warnings, >> but what's our current patching policy for this tool? > > The patch looks good to me. It looks like we already

Re: PG 13 release notes, first draft

2020-05-15 Thread Fujii Masao
On 2020/05/05 12:16, Bruce Momjian wrote: I have committed the first draft of the PG 13 release notes. You can see them here: https://momjian.us/pgsql_docs/release-13.html It still needs markup, word wrap, and indenting. The community doc build should happen in a few hours. Many

Re: pg_regress cleans up tablespace twice.

2020-05-15 Thread Kyotaro Horiguchi
Thank you for looking this! At Fri, 15 May 2020 11:58:55 +0900, Michael Paquier wrote in > On Mon, May 11, 2020 at 05:13:54PM +0900, Kyotaro Horiguchi wrote: > > Tablespace directory cleanup is not done for all testing > > targets. Actually it is not done for the tools under bin/ except > >

Re: Is it useful to record whether plans are generic or custom?

2020-05-15 Thread Atsushi Torikoshi
On Thu, May 14, 2020 at 2:28 AM legrand legrand wrote: > Hello, > > yes this can be usefull, under the condition of differentiating all the > counters > for a queryid using a generic plan and the one using a custom one. > > For me one way to do that is adding a generic_plan column to >

Re: MultiXact\SLRU buffers configuration

2020-05-15 Thread Andrey M. Borodin
> 15 мая 2020 г., в 05:03, Kyotaro Horiguchi > написал(а): > > At Thu, 14 May 2020 11:44:01 +0500, "Andrey M. Borodin" > wrote in >>> GetMultiXactIdMembers believes that 4 is successfully done if 2 >>> returned valid offset, but actually that is not obvious. >>> >>> If we add a single

pg_stat_wal_receiver and flushedUpto/writtenUpto

2020-05-15 Thread Michael Paquier
Hi, As discussed in the thread that introduced d140f2f3 to rename receivedUpto to flushedUpto and add writtenUpto to the WAL receiver's shared memory information, the equivalent columns in pg_stat_wal_receiver have not been renamed:

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-05-15 Thread Dilip Kumar
On Wed, May 13, 2020 at 4:50 PM Amit Kapila wrote: > > On Wed, May 13, 2020 at 11:35 AM Dilip Kumar wrote: > > > > On Tue, May 12, 2020 at 4:39 PM Amit Kapila wrote: > > > > > > > > > v20-0003-Extend-the-output-plugin-API-with-stream-methods > > >

Re: Transactions involving multiple postgres foreign servers, take 2

2020-05-15 Thread Muhammad Usama
On Fri, May 15, 2020 at 9:59 AM Masahiko Sawada < masahiko.saw...@2ndquadrant.com> wrote: > On Fri, 15 May 2020 at 13:26, Muhammad Usama wrote: > > > > > > > > On Fri, May 15, 2020 at 7:20 AM Masahiko Sawada < > masahiko.saw...@2ndquadrant.com> wrote: > >> > >> On Fri, 15 May 2020 at 03:08,

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-05-15 Thread Amit Kapila
On Fri, May 15, 2020 at 2:47 PM Dilip Kumar wrote: > > > 6. I think it will be good if we can provide an example of streaming > > changes via test_decoding at > > https://www.postgresql.org/docs/devel/test-decoding.html. I think we > > can also explain there why the user is not expected to see

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-05-15 Thread Dilip Kumar
On Fri, May 15, 2020 at 4:04 PM Amit Kapila wrote: > > On Fri, May 15, 2020 at 2:47 PM Dilip Kumar wrote: > > > > > 6. I think it will be good if we can provide an example of streaming > > > changes via test_decoding at > > > https://www.postgresql.org/docs/devel/test-decoding.html. I think we >

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-05-15 Thread Amit Kapila
On Fri, May 15, 2020 at 4:20 PM Dilip Kumar wrote: > > On Fri, May 15, 2020 at 4:04 PM Amit Kapila wrote: > > > > > > > > - In the example we can not show a real example, because of the > > > in-progress transaction to show the changes, we might have to > > > implement a lot of tuple. I think