Re: make tuplestore helper function

2022-02-27 Thread Michael Paquier
On Thu, Feb 24, 2022 at 08:25:06PM +0900, Michael Paquier wrote: > This is the remaining piece, as attached, that I have not been able to > poke much at yet. So, I have finally poked at this last part of the patch set, and I found that we can be more aggressive with the refactoring, by moving

RE: Design of pg_stat_subscription_workers vs pgstats

2022-02-27 Thread tanghy.f...@fujitsu.com
On Mon, Feb 28, 2022 12:32 PM Amit Kapila wrote: > > On Mon, Feb 28, 2022 at 8:17 AM Masahiko Sawada > wrote: > > > > On Mon, Feb 28, 2022 at 11:33 AM Amit Kapila > wrote: > > > > > > > > > > > (2) doc/src/sgml/monitoring.sgml > > > > > > > > +Resets statistics for a single

RE: Logical replication timeout problem

2022-02-27 Thread wangw.f...@fujitsu.com
On Thur, Feb 24, 2022 at 4:06 PM Kuroda, Hayato/黒田 隼人 wrote: > Dear Wang, Thanks for your review. > > According to our discussion, we need to send keepalive messages to > > subscriber when skipping changes. > > One approach is that **for each skipped change**, we try to send > > keepalive

RE: Logical replication timeout problem

2022-02-27 Thread wangw.f...@fujitsu.com
On Wed, Feb 22, 2022 at 4:56 PM Amit Kapila wrote: > Thanks for your review. > On Tue, Feb 22, 2022 at 9:17 AM wangw.f...@fujitsu.com > wrote: > > > > On Fri, Feb 18, 2022 at 10:51 AM Ajin Cherian wrote: > > > Some comments: > > Thanks for your review. > > > > > I see you only track skipped

Re: support for MERGE

2022-02-27 Thread Julien Rouhaud
On Sun, Feb 27, 2022 at 09:17:13PM +0100, Daniel Gustafsson wrote: > > On 27 Feb 2022, at 18:42, Tom Lane wrote: > > > I'd rather keep all the ModifyTable code in one .c file, even if that one is > > bigger than our usual practice. > > Agreed, I also prefer a (too) large file over a set of .c

Re: parse/analyze API refactoring

2022-02-27 Thread Peter Eisentraut
You set this commit fest entry to Waiting on Author, but there were no reviews posted and the patch still applies and builds AFAICT, so I don't know what you meant by that. On 13.01.22 00:49, Bossart, Nathan wrote: On 12/28/21, 8:25 AM, "Peter Eisentraut" wrote: (The "withcb" naming maybe

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-27 Thread Julien Rouhaud
Hi, On Mon, Feb 28, 2022 at 10:21:23AM +0530, Bharath Rupireddy wrote: > > Another thought for my review comment: > > 1) Can't we use pg_is_in_recovery to determine if it's a restartpoint > > or checkpoint instead of having a new function > > pg_stat_get_progress_checkpoint_type? > > I don't

Re: generic plans and "initial" pruning

2022-02-27 Thread Amit Langote
Hi Andres, On Fri, Feb 11, 2022 at 10:29 AM Andres Freund wrote: > On 2022-02-10 17:13:52 +0900, Amit Langote wrote: > > The attached patch implements this idea. Sorry for the delay in > > getting this out and thanks to Robert for the off-list discussions on > > this. > > I did not follow this

Re: BufferAlloc: don't take two simultaneous locks

2022-02-27 Thread Yura Sokolov
В Пт, 25/02/2022 в 09:01 -0800, Andres Freund пишет: > Hi, > > On 2022-02-25 12:51:22 +0300, Yura Sokolov wrote: > > > > +* The usage_count starts out at 1 so that the buffer can > > > > survive one > > > > +* clock-sweep pass. > > > > +* > > > > +* We use direct

Re: why do hash index builds use smgrextend() for new splitpoint pages

2022-02-27 Thread Amit Kapila
On Sat, Feb 26, 2022 at 9:17 PM Melanie Plageman wrote: > > On Fri, Feb 25, 2022 at 11:17 PM Amit Kapila wrote: > > > > On Sat, Feb 26, 2022 at 3:01 AM Melanie Plageman > > wrote: > > > > > > Since _hash_alloc_buckets() WAL-logs the last page of the > > > splitpoint, is it safe to skip the

Re: Synchronizing slots from primary to standby

2022-02-27 Thread Bharath Rupireddy
On Thu, Feb 24, 2022 at 12:46 AM James Coleman wrote: > I've been working on adding test coverage to prove this out, but I've > encountered the problem reported in [1]. > > My assumption, but Andres please correct me if I'm wrong, that we > should see issues with the following steps (given the

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-27 Thread Bharath Rupireddy
On Sun, Feb 27, 2022 at 8:44 PM Bharath Rupireddy wrote: > > On Fri, Feb 25, 2022 at 8:38 PM Nitin Jadhav > wrote: > > Had a quick look over the v3 patch. I'm not sure if it's the best way > to have pg_stat_get_progress_checkpoint_type, > pg_stat_get_progress_checkpoint_kind and >

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-27 Thread Amit Kapila
On Mon, Feb 28, 2022 at 8:17 AM Masahiko Sawada wrote: > > On Mon, Feb 28, 2022 at 11:33 AM Amit Kapila wrote: > > > > > > > > (2) doc/src/sgml/monitoring.sgml > > > > > > +Resets statistics for a single subscription shown in the > > > +pg_stat_subscription_stats view to > > >

RE: Design of pg_stat_subscription_workers vs pgstats

2022-02-27 Thread osumi.takami...@fujitsu.com
On Monday, February 28, 2022 12:57 PM Amit Kapila > On Mon, Feb 28, 2022 at 8:49 AM osumi.takami...@fujitsu.com > wrote: > > > > On Monday, February 28, 2022 11:34 AM Amit Kapila > wrote: > > > On Sat, Feb 26, 2022 at 1:35 PM osumi.takami...@fujitsu.com > > > wrote: > > > > > > > > On

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-27 Thread Amit Kapila
On Mon, Feb 28, 2022 at 8:49 AM osumi.takami...@fujitsu.com wrote: > > On Monday, February 28, 2022 11:34 AM Amit Kapila > wrote: > > On Sat, Feb 26, 2022 at 1:35 PM osumi.takami...@fujitsu.com > > wrote: > > > > > > On Saturday, February 26, 2022 11:51 AM Amit Kapila > > wrote: > > > > I

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-27 Thread Amit Kapila
On Mon, Feb 28, 2022 at 8:59 AM Peter Smith wrote: > > > 2. doc/src/sgml/monitoring.sgml > > +Resets statistics for a single subscription shown in the > +pg_stat_subscription_stats view to zero. If > +the argument is NULL, reset statistics for all > +subscriptions.

Re: Commitfest manager for 2022-03

2022-02-27 Thread Amit Kapila
On Sun, Feb 27, 2022 at 11:37 PM Euler Taveira wrote: > > On Sat, Feb 26, 2022, at 9:37 PM, Justin Pryzby wrote: > > |https://commitfest.postgresql.org/37/2906/ > |Row filtering for logical replication > If I'm not wrong, this is merged and should be closed? > > I think Amit forgot to mark it as

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-27 Thread Peter Smith
Below are my comments for the v4 patch. These are only nitpicking comments now; Otherwise, it LGTM. (Sorry, now I see there are some overlaps with comments posted in the last 20 mins so take or leave these as you wish) == 1. doc/src/sgml/monitoring.sgml - - OID of the relation

RE: Design of pg_stat_subscription_workers vs pgstats

2022-02-27 Thread osumi.takami...@fujitsu.com
On Monday, February 28, 2022 11:34 AM Amit Kapila wrote: > On Sat, Feb 26, 2022 at 1:35 PM osumi.takami...@fujitsu.com > wrote: > > > > On Saturday, February 26, 2022 11:51 AM Amit Kapila > wrote: > > > I have reviewed the latest version and made a few changes along with > > > fixing some of

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-27 Thread Masahiko Sawada
On Mon, Feb 28, 2022 at 11:52 AM Amit Kapila wrote: > > On Mon, Feb 28, 2022 at 8:17 AM Masahiko Sawada wrote: > > > > On Mon, Feb 28, 2022 at 11:33 AM Amit Kapila > > wrote: > > > > > > > > > > > (2) doc/src/sgml/monitoring.sgml > > > > > > > > +Resets statistics for a single

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-27 Thread Amit Kapila
On Mon, Feb 28, 2022 at 8:17 AM Masahiko Sawada wrote: > > On Mon, Feb 28, 2022 at 11:33 AM Amit Kapila wrote: > > > > > > > > (2) doc/src/sgml/monitoring.sgml > > > > > > +Resets statistics for a single subscription shown in the > > > +pg_stat_subscription_stats view to > > >

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-27 Thread Masahiko Sawada
On Mon, Feb 28, 2022 at 11:33 AM Amit Kapila wrote: > > On Sat, Feb 26, 2022 at 1:35 PM osumi.takami...@fujitsu.com > wrote: > > > > On Saturday, February 26, 2022 11:51 AM Amit Kapila > > wrote: > > > I have reviewed the latest version and made a few changes along with > > > fixing > > >

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-27 Thread Amit Kapila
On Sat, Feb 26, 2022 at 1:35 PM osumi.takami...@fujitsu.com wrote: > > On Saturday, February 26, 2022 11:51 AM Amit Kapila > wrote: > > I have reviewed the latest version and made a few changes along with fixing > > some of the pending comments by Peter Smith. The changes are as > > follows:

Re: pg_stat_statements: remove redundant function call in pg_stat_statements_internal

2022-02-27 Thread Michael Paquier
On Sun, Feb 27, 2022 at 09:08:56PM +0900, Michael Paquier wrote: > Yes, let's clean up that on HEAD. No objections from here. I'll do > that tomorrow or so. And done. -- Michael signature.asc Description: PGP signature

Re: Is it correct to update db state in control file as "shutting down" during end-of-recovery checkpoint?

2022-02-27 Thread Michael Paquier
On Mon, Feb 28, 2022 at 10:51:06AM +0900, Kyotaro Horiguchi wrote: > That sounds like we should reject the patch as we don't agree to its > objective. If someday end-of-recovery checkpoints functionally > diverge from shutdown checkpoints but leave (somehow) the transition > alone, we may visit

Re: Is it correct to update db state in control file as "shutting down" during end-of-recovery checkpoint?

2022-02-27 Thread Kyotaro Horiguchi
At Sat, 26 Feb 2022 12:11:15 +0900, Michael Paquier wrote in > On Fri, Feb 25, 2022 at 01:09:53PM -0800, Nathan Bossart wrote: > > This one has been quiet for a while. Should we mark it as > > returned-with-feedback? > > Yes, that's my feeling and I got cold feet about this change. This >

Re: Checkpointer sync queue fills up / loops around pg_usleep() are bad

2022-02-27 Thread Andres Freund
Hi, On February 27, 2022 4:19:21 PM PST, Thomas Munro wrote: >With the attached, 027_stream_regress.pl drops from ~29.5s to ~19.6s >on my FreeBSD workstation! That's impressive - wouldn't have guessed it to make that much of a difference. I assume that running the tests on freebsd for an

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-27 Thread Masahiko Sawada
On Sat, Feb 26, 2022 at 11:51 AM Amit Kapila wrote: > > On Thu, Feb 24, 2022 at 9:20 PM Masahiko Sawada wrote: > > > > I have reviewed the latest version and made a few changes along with > fixing some of the pending comments by Peter Smith. Thank you for updating the patch! > The changes are

Re: CREATE DATABASE IF NOT EXISTS in PostgreSQL

2022-02-27 Thread Japin Li
On Mon, 28 Feb 2022 at 01:53, Tom Lane wrote: > Japin Li writes: >> Why don't support CREATE DATABASE IF NOT EXISTS syntax in PostgreSQL? > > FWIW, I'm generally hostile to CREATE IF NOT EXISTS semantics across > the board, because of its exceedingly squishy semantics: it ensures > that an

Re: Checkpointer sync queue fills up / loops around pg_usleep() are bad

2022-02-27 Thread Thomas Munro
On Sun, Feb 27, 2022 at 10:29 PM Julien Rouhaud wrote: > On Sun, Feb 27, 2022 at 06:10:45PM +0900, Michael Paquier wrote: > > On Sat, Feb 26, 2022 at 01:39:42PM -0800, Andres Freund wrote: > > > I suspect the easiest is to just convert that usleep to a WaitLatch(). > > > That'd > > > require

Re: Missed condition-variable wakeups on FreeBSD

2022-02-27 Thread Thomas Munro
On Sun, Feb 27, 2022 at 8:07 AM Tom Lane wrote: > I don't know much about how gdb interacts with kernel calls on > FreeBSD, but I speculate that the poll(2) call returns with EINTR > after gdb releases the process, and then things resume fine, Yeah, at least FreeBSD and macOS interrupt system

Re: Proposal: Support custom authentication methods using hooks

2022-02-27 Thread Jeff Davis
On Fri, 2022-02-25 at 14:10 -0500, Tom Lane wrote: > I'm happy to add support for custom auth methods if they can use > a protocol that's safer than cleartext-password. But if that's the > only feasible option, then we're just encouraging people to use > insecure methods. FWIW, I'd like to be

Re: Missed condition-variable wakeups on FreeBSD

2022-02-27 Thread Tom Lane
Andres Freund writes: > On 2022-02-26 14:07:05 -0500, Tom Lane wrote: >> I have observed this three times in the REL_11 branch, once >> in REL_12, and a couple of times last summer before it occurred >> to me to start keeping notes. Over that time the machine has >> been running various

Re: Separate the result of \watch for each query execution (psql)

2022-02-27 Thread Noboru Saito
Hi, 2022年2月25日(金) 13:42 Pavel Stehule : > > > > pá 25. 2. 2022 v 5:23 odesílatel Noboru Saito napsal: >> >> Hi, >> >> Pavel Stehule : >> > > I strongly agree. It was a lot of work to find a workable solution for >> > > pspg. Special chars that starting result and maybe other, that ending >> >

Re: support for MERGE

2022-02-27 Thread Daniel Gustafsson
> On 27 Feb 2022, at 18:42, Tom Lane wrote: > I'd rather keep all the ModifyTable code in one .c file, even if that one is > bigger than our usual practice. Agreed, I also prefer a (too) large file over a set of .c #include's. -- Daniel Gustafsson https://vmware.com/

Re: support for MERGE

2022-02-27 Thread Zhihong Yu
On Sun, Feb 27, 2022 at 9:25 AM Alvaro Herrera wrote: > I attach v12 of MERGE. Significant effort went into splitting > ExecUpdate and ExecDelete into parts that can be reused from the MERGE > routines in a way that doesn't involve jumping out in the middle of > TM_Result processing to

Re: Commitfest manager for 2022-03

2022-02-27 Thread Euler Taveira
On Sat, Feb 26, 2022, at 9:37 PM, Justin Pryzby wrote: > |https://commitfest.postgresql.org/37/2906/ > |Row filtering for logical replication > If I'm not wrong, this is merged and should be closed? I think Amit forgot to mark it as committed. Done. -- Euler Taveira EDB

Re: CREATE DATABASE IF NOT EXISTS in PostgreSQL

2022-02-27 Thread Tom Lane
Japin Li writes: > Why don't support CREATE DATABASE IF NOT EXISTS syntax in PostgreSQL? FWIW, I'm generally hostile to CREATE IF NOT EXISTS semantics across the board, because of its exceedingly squishy semantics: it ensures that an object by that name exists, but you have exactly no guarantees

Re: support for MERGE

2022-02-27 Thread Tom Lane
Alvaro Herrera writes: > I think we should make a decision on code arrangement here. From my > perspective, MERGE isn't its own executor node; rather it's just another > "method" in ModifyTable. Which makes sense, given that all it does is > call parts of INSERT, UPDATE, DELETE which are the

Re: support for MERGE

2022-02-27 Thread Alvaro Herrera
On 2022-Jan-28, Andres Freund wrote: > Any chance you could split this into something more reviewable? The overall > diff stats are: 102 files changed, 8589 insertions(+), 234 deletions(-) thats > pretty hard to really review. Incremental commits don't realy help with that. I'll work on

Re: Add index scan progress to pg_stat_progress_vacuum

2022-02-27 Thread Imseih (AWS), Sami
> On Wed, Feb 23, 2022 at 10:02 AM Imseih (AWS), Sami wrote: >> If the failsafe kicks in midway through a vacuum, the number indexes_total will not be reset to 0. If INDEX_CLEANUP is turned off, then the value will be 0 at the start of the vacuum. > > The way that this works

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-27 Thread Bharath Rupireddy
On Fri, Feb 25, 2022 at 8:38 PM Nitin Jadhav wrote: Had a quick look over the v3 patch. I'm not sure if it's the best way to have pg_stat_get_progress_checkpoint_type, pg_stat_get_progress_checkpoint_kind and pg_stat_get_progress_checkpoint_start_time just for printing info in readable format in

Re: Support for grabbing multiple consecutive values with nextval()

2022-02-27 Thread Jille Timmermans
On 2022-02-27 14:22, Julien Rouhaud wrote: Hi, On Sun, Feb 27, 2022 at 10:42:25AM +0100, Jille Timmermans wrote: First time PostgreSQL contributor here :) Welcome! Thanks! I wanted to be able to allocate a bunch of numbers from a sequence at once. Multiple people seem to be struggling

CREATE DATABASE IF NOT EXISTS in PostgreSQL

2022-02-27 Thread Japin Li
Hi, hackers When I try to use CREATE DATABASE IF NOT EXISTS in PostgreSQL, it complains this syntax is not supported. We can use the following command to achieve this, however, it's not straightforward. SELECT 'CREATE DATABASE mydb' WHERE NOT EXISTS (SELECT FROM pg_database WHERE

Re: PATCH: add "--config-file=" option to pg_rewind

2022-02-27 Thread Gunnar "Nick" Bluth
Am 27.02.22 um 13:06 schrieb Michael Paquier: On Sat, Feb 26, 2022 at 09:55:20AM +0100, Gunnar "Nick" Bluth wrote: Am 26.02.22 um 06:51 schrieb Michael Paquier: Shouldn't this one use appendShellString() on config_file? It probably should, yes. I don't fancy this repetitive code myself. But

Re: Support for grabbing multiple consecutive values with nextval()

2022-02-27 Thread Julien Rouhaud
Hi, On Sun, Feb 27, 2022 at 10:42:25AM +0100, Jille Timmermans wrote: > > First time PostgreSQL contributor here :) Welcome! > I wanted to be able to allocate a bunch of numbers from a sequence at once. > Multiple people seem to be struggling with this >

Support for grabbing multiple consecutive values with nextval()

2022-02-27 Thread Jille Timmermans
Hi, First time PostgreSQL contributor here :) I wanted to be able to allocate a bunch of numbers from a sequence at once. Multiple people seem to be struggling with this (https://stackoverflow.com/questions/896274/select-multiple-ids-from-a-postgresql-sequence,

Re: pg_stat_statements: remove redundant function call in pg_stat_statements_internal

2022-02-27 Thread Michael Paquier
On Sun, Feb 27, 2022 at 07:55:26PM +0800, Julien Rouhaud wrote: > Indeed. I doubt it will make any real difference but it doesn't hurt to fix > it. > > Patch looks good to me. Yes, let's clean up that on HEAD. No objections from here. I'll do that tomorrow or so. -- Michael signature.asc

Re: PATCH: add "--config-file=" option to pg_rewind

2022-02-27 Thread Michael Paquier
On Sat, Feb 26, 2022 at 09:55:20AM +0100, Gunnar "Nick" Bluth wrote: > Am 26.02.22 um 06:51 schrieb Michael Paquier: >> Shouldn't this one use appendShellString() on config_file? > > It probably should, yes. I don't fancy this repetitive code myself. > But then, pg_rewind as a whole could use an

Re: pg_stat_statements: remove redundant function call in pg_stat_statements_internal

2022-02-27 Thread Julien Rouhaud
Hi, On Sun, Feb 27, 2022 at 08:45:13PM +0900, Dong Wook Lee wrote: > > I found some redundant function calls in > pg_stat_statements.c/pg_stat_statements_internal(), > There is no need to call GetUserId() again because the value was > previously obtained. Indeed. I doubt it will make any real

pg_stat_statements: remove redundant function call in pg_stat_statements_internal

2022-02-27 Thread Dong Wook Lee
Hi, I found some redundant function calls in pg_stat_statements.c/pg_stat_statements_internal(), There is no need to call GetUserId() again because the value was previously obtained. so I propose a patch to fix it. --- a/contrib/pg_stat_statements/pg_stat_statements.c +++

Re: Some optimisations for numeric division

2022-02-27 Thread Dean Rasheed
On Fri, 25 Feb 2022 at 18:30, Tom Lane wrote: > > Dean Rasheed writes: > > And another update following feedback from the cfbot. > > This patchset LGTM. On my machine there doesn't seem to be any > measurable performance change for the numeric regression test, > but numeric_big gets about 15%

Re: Is it worth adding ReplicationSlot active_pid to ReplicationSlotPersistentData?

2022-02-27 Thread Bharath Rupireddy
On Mon, Jan 10, 2022 at 6:50 AM Bharath Rupireddy wrote: > > On Wed, Jan 5, 2022 at 12:13 PM Kyotaro Horiguchi > wrote: > > > Here's the v2 patch. Please provide your thoughts. > > > > Thanks! I have three comments on this version. > > Thanks for your review. > > > - I still think

Re: Checkpointer sync queue fills up / loops around pg_usleep() are bad

2022-02-27 Thread Julien Rouhaud
On Sun, Feb 27, 2022 at 06:10:45PM +0900, Michael Paquier wrote: > On Sat, Feb 26, 2022 at 01:39:42PM -0800, Andres Freund wrote: > > I suspect the easiest is to just convert that usleep to a WaitLatch(). > > That'd > > require adding a new enum value to WaitEventTimeout in 14. Which probably is

Re: Checkpointer sync queue fills up / loops around pg_usleep() are bad

2022-02-27 Thread Michael Paquier
On Sat, Feb 26, 2022 at 01:39:42PM -0800, Andres Freund wrote: > I suspect the easiest is to just convert that usleep to a WaitLatch(). That'd > require adding a new enum value to WaitEventTimeout in 14. Which probably is > fine? We've added wait events in back-branches in the past, so this does

Re: Commitfest manager for 2022-03

2022-02-27 Thread Michael Paquier
On Sun, Feb 27, 2022 at 04:51:16PM +0800, Julien Rouhaud wrote: > I don't really understand what that field is supposed to mean. But now that > we're in the final pg15 commit fest, wouldn't it be simpler to actually move > the patches for which there's a agreement that they can't make it to pg15?

Re: Commitfest manager for 2022-03

2022-02-27 Thread Julien Rouhaud
On Sat, Feb 26, 2022 at 06:37:21PM -0600, Justin Pryzby wrote: > Can I suggest to update the CF APP to allow: > | Target version: 16 > > I also suggest to update patches to indicate which are (not) being considered > for v15. I don't really understand what that field is supposed to mean. But