Re: Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2024-01-11 Thread Bertrand Drouvot
Hi, On Fri, Jan 12, 2024 at 07:01:55AM +0900, Michael Paquier wrote: > On Thu, Jan 11, 2024 at 11:00:01PM +0300, Alexander Lakhin wrote: > > Bertrand, I've relaunched tests in the same slowed down VM with both > > patches applied (but with no other modifications) and got a failure > > with

Re: Synchronizing slots from primary to standby

2024-01-11 Thread Bertrand Drouvot
Hi, On Fri, Jan 12, 2024 at 03:46:00AM +, Zhijie Hou (Fujitsu) wrote: > On Thursday, January 11, 2024 11:42 PM Bertrand Drouvot > wrote: > > Hi, > > > On Thu, Jan 11, 2024 at 04:22:56PM +0530, Amit Kapila wrote: > > IIUC, this would be a sync slot (so not usable until promotion) that

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-11 Thread Junwang Zhao
Hi, On Wed, Jan 10, 2024 at 2:20 PM Sutou Kouhei wrote: > > Hi, > > In > "Re: Make COPY format extendable: Extract COPY TO format implementations" > on Fri, 22 Dec 2023 10:58:05 +0800, > Junwang Zhao wrote: > > >> 1. Add an opaque space for custom COPY TO handler > >>* Add

Re: Synchronizing slots from primary to standby

2024-01-11 Thread Bertrand Drouvot
Hi, On Fri, Jan 12, 2024 at 08:42:39AM +0530, Amit Kapila wrote: > On Thu, Jan 11, 2024 at 9:11 PM Bertrand Drouvot > wrote: > > > > On Thu, Jan 11, 2024 at 04:22:56PM +0530, Amit Kapila wrote: > > > > > > > > To close the above race, I could think of the following ways: > > > > 1. Drop and

Re: Make NUM_XLOGINSERT_LOCKS configurable

2024-01-11 Thread Bharath Rupireddy
On Wed, Jan 10, 2024 at 11:43 AM Tom Lane wrote: > > Bharath Rupireddy writes: > > On Wed, Jan 10, 2024 at 10:00 AM Tom Lane wrote: > >> Maybe. I bet just bumping up the constant by 2X or 4X or so would get > >> most of the win for far less work; it's not like adding a few more > >> LWLocks is

RE: speed up a logical replica setup

2024-01-11 Thread Hayato Kuroda (Fujitsu)
Dear Euler, Sorry for disturbing your work and thanks for updates. I will review your patch again. > * 0001 patch It is almost the same as v3-0001, which was posted by Euler. An unnecessary change for Mkvcbuild.pm (this file was removed) was ignored. v5 removes the MSVC support. > Confirmed

Re: doc: add LITERAL tag to RETURNING

2024-01-11 Thread Ashutosh Bapat
On Fri, Jan 12, 2024 at 11:27 AM torikoshia wrote: > > Hi, > > RETURNING is usually tagged with appropriate tags, such as , > but not in the 'query' section of COPY. I have the same observation. > > https://www.postgresql.org/docs/devel/sql-copy.html > > Would it be better to put here as well?

doc: add LITERAL tag to RETURNING

2024-01-11 Thread torikoshia
Hi, RETURNING is usually tagged with appropriate tags, such as , but not in the 'query' section of COPY. https://www.postgresql.org/docs/devel/sql-copy.html Would it be better to put here as well? -- Regards, -- Atsushi Torikoshi NTT DATA Group CorporationFrom

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-11 Thread Sutou Kouhei
Hi, Here is the current summary for a this discussion to make COPY format extendable. It's for reaching consensus and starting implementing the feature. (I'll start implementing the feature once we reach consensus.) If you have any opinion, please share it. Confirmed: 1.1 Making COPY format

Re: Synchronizing slots from primary to standby

2024-01-11 Thread Amit Kapila
On Thu, Jan 11, 2024 at 9:11 PM Bertrand Drouvot wrote: > > On Thu, Jan 11, 2024 at 04:22:56PM +0530, Amit Kapila wrote: > > On Tue, Jan 9, 2024 at 6:39 PM Amit Kapila wrote: > > > > > > +static bool > > > +synchronize_one_slot(WalReceiverConn *wrconn, RemoteSlot *remote_slot) > > > { > > > ...

Re: Compile warnings in dbcommands.c building with meson

2024-01-11 Thread jian he
On Fri, Jan 12, 2024 at 1:05 AM Magnus Hagander wrote: > > On Wed, Jan 10, 2024 at 1:16 PM Aleksander Alekseev > wrote: > > > > Hi, > > > > > When building current head on debian bullseye I get this compile warning: > > > > > > In file included from ../src/backend/commands/dbcommands.c:20: > > >

RE: speed up a logical replica setup

2024-01-11 Thread Hayato Kuroda (Fujitsu)
Dear Amit, Euler, > > > > > > 3. > > Options -o/-O were added to specify options for publications/subscriptions. > > > > > > Flexibility is cool. However, I think the cost benefit of it is not good. > > You > > have to parse the options to catch preliminary errors. Things like publish > > only

Re: pub/sub - specifying optional parameters without values.

2024-01-11 Thread Peter Smith
On Mon, Jan 30, 2023 at 8:36 AM Tom Lane wrote: > > Zheng Li writes: > > The behavior is due to the following code > > https://github.com/postgres/postgres/blob/master/src/backend/commands/define.c#L113 > > Yeah, so you can grep for places that have this behavior by looking > for defGetBoolean

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-11 Thread Michael Paquier
On Thu, Jan 11, 2024 at 01:43:27PM -0500, Robert Haas wrote: > On Tue, Jan 9, 2024 at 2:35 PM Andres Freund wrote: >> I don't have that strong feelings about it. If both of you think it >> looks good, go ahead... > > Done. Thanks for e2d5b3b9b643. -- Michael signature.asc Description: PGP

Re: Adding facility for injection points (or probe points?) for more advanced tests

2024-01-11 Thread Michael Paquier
On Fri, Jan 12, 2024 at 08:35:42AM +0900, Michael Paquier wrote: > It also seems like you've missed this message, where this has been > mentioned (spoiler: first version of the patch used an alternate > output): > https://www.postgresql.org/message-id/zunuzpimkzcov...@paquier.xyz The refactoring

reorganize "Shared Memory and LWLocks" section of docs

2024-01-11 Thread Nathan Bossart
I recently began trying to write documentation for the dynamic shared memory registry feature [0], and I noticed that the "Shared Memory and LWLocks" section of the documentation might need some improvement. At least, I felt that it would be hard to add any new content to this section without

Re: Adding facility for injection points (or probe points?) for more advanced tests

2024-01-11 Thread Ashutosh Bapat
On Fri, Jan 12, 2024 at 5:05 AM Michael Paquier wrote: > > On Thu, Jan 11, 2024 at 02:47:27PM +0530, Ashutosh Bapat wrote: > > On Thu, Jan 11, 2024 at 9:42 AM Michael Paquier wrote: > >> I don't really aim for complicated here, just useful. > > > > It isn't complicated. Such simple error check

Re: A failure in t/038_save_logical_slots_shutdown.pl

2024-01-11 Thread Amit Kapila
On Thu, Jan 11, 2024 at 10:03 PM Bharath Rupireddy wrote: > > On Thu, Jan 11, 2024 at 4:35 PM vignesh C wrote: > > > > On further analysis, it was found that in the failing test, > > CHECKPOINT_SHUTDOWN was started in a new page, so there was the WAL > > page header present just before the

Re: Recovering from detoast-related catcache invalidations

2024-01-11 Thread Xiaoran Wang
> Also, I'm pretty dubious that GetNonHistoricCatalogSnapshot rather > than GetCatalogSnapshot is the right thing, because the catcaches > use the latter. Yes, you are right, should use GetCatalogSnapshot here. > Maybe, but that undocumented hack in SetHintBits seems completely > unacceptable.

RE: Synchronizing slots from primary to standby

2024-01-11 Thread Zhijie Hou (Fujitsu)
On Thursday, January 11, 2024 11:42 PM Bertrand Drouvot wrote: Hi, > On Thu, Jan 11, 2024 at 04:22:56PM +0530, Amit Kapila wrote: > > On Tue, Jan 9, 2024 at 6:39 PM Amit Kapila > wrote: > > > > > > +static bool > > > +synchronize_one_slot(WalReceiverConn *wrconn, RemoteSlot > > >

Re: speed up a logical replica setup

2024-01-11 Thread Amit Kapila
On Fri, Jan 12, 2024 at 4:30 AM Euler Taveira wrote: > > > 3. > Options -o/-O were added to specify options for publications/subscriptions. > > > Flexibility is cool. However, I think the cost benefit of it is not good. You > have to parse the options to catch preliminary errors. Things like

Re: Synchronizing slots from primary to standby

2024-01-11 Thread Amit Kapila
On Thu, Jan 11, 2024 at 9:11 PM Bertrand Drouvot wrote: > > On Thu, Jan 11, 2024 at 04:22:56PM +0530, Amit Kapila wrote: > > On Tue, Jan 9, 2024 at 6:39 PM Amit Kapila wrote: > > > > > > +static bool > > > +synchronize_one_slot(WalReceiverConn *wrconn, RemoteSlot *remote_slot) > > > { > > > ...

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-01-11 Thread torikoshia
On Wed, Jan 10, 2024 at 4:42 PM Masahiko Sawada wrote: Yeah, I'm still thinking it's better to implement this feature incrementally. Given we're closing to feature freeze, I think it's unlikely to get the whole feature into PG17 since there are still many design discussions we need in

Re: initdb --no-locale=C doesn't work as specified when the environment is not C

2024-01-11 Thread Kyotaro Horiguchi
At Wed, 10 Jan 2024 18:16:03 -0500, Tom Lane wrote in > Kyotaro Horiguchi writes: > > It seems somewhat intentional that only lc_messages references the > > environment at boot time. On the other hand, previously, in the > > absence of a specified locale, initdb would embed the environmental >

Re: Improve the connection failure error messages

2024-01-11 Thread Peter Smith
Thanks for the patch! Here are a couple of review comments for it. == src/backend/commands/subscriptioncmds.c 1. @@ -742,7 +742,7 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt, if (!wrconn) ereport(ERROR, (errcode(ERRCODE_CONNECTION_FAILURE), - errmsg("could

Re: Error "initial slot snapshot too large" in create replication slot

2024-01-11 Thread Kyotaro Horiguchi
Thank you for the comments. This will help move the discussion forward. At Fri, 5 Jan 2024 15:17:11 -0500, Robert Haas wrote in > On Thu, Mar 23, 2023 at 11:02 PM Kyotaro Horiguchi > wrote: > > [ new patch ] > > Well, I guess nobody is too excited about fixing this, because it's > been

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-11 Thread Melanie Plageman
On Thu, Jan 11, 2024 at 4:49 PM Robert Haas wrote: > > On Thu, Jan 11, 2024 at 2:30 PM Melanie Plageman > wrote: > > I'm still kind of hung up on the changes that 0001 makes to vacuumlazy.c. > > Say we didn't add the recordfreespace parameter to lazy_scan_prune(). > Couldn't the caller just

RE: cleanup patches for incremental backup

2024-01-11 Thread Shinoda, Noriyoshi (HPE Services Japan - FSIP)
Hi, Thank you for developing the new tool. I have attached a patch that corrects the spelling of the --individual option in the SGML file. Regards, Noriyoshi Shinoda -Original Message- From: Robert Haas Sent: Friday, January 12, 2024 3:13 AM To: PostgreSQL Hackers ; Jakub Wartak

Re: Fix log_line_prefix to display the transaction id (%x) for statements not in a transaction block

2024-01-11 Thread jian he
Hi another big difference compare to HEAD: select name,setting frompg_settings where name in ('auto_explain.log_timing','auto_explain.log_analyze', 'auto_explain.log_min_duration','log_statement','log_line_prefix') ; name |

Re: Confine vacuum skip logic to lazy_scan_skip

2024-01-11 Thread Melanie Plageman
On Fri, Jan 5, 2024 at 5:51 AM Nazir Bilal Yavuz wrote: > > On Fri, 5 Jan 2024 at 02:25, Jim Nasby wrote: > > > > On 1/4/24 2:23 PM, Andres Freund wrote: > > > > On 2024-01-02 12:36:18 -0500, Melanie Plageman wrote: > > > > Subject: [PATCH v2 1/6] lazy_scan_skip remove unnecessary local var > >

Re: Confine vacuum skip logic to lazy_scan_skip

2024-01-11 Thread Melanie Plageman
v3 attached On Thu, Jan 4, 2024 at 3:23 PM Andres Freund wrote: > > Hi, > > On 2024-01-02 12:36:18 -0500, Melanie Plageman wrote: > > Subject: [PATCH v2 1/6] lazy_scan_skip remove unnecessary local var > > rel_pages > > Subject: [PATCH v2 2/6] lazy_scan_skip remove unneeded local var > >

Re: A recent message added to pg_upgade

2024-01-11 Thread Michael Paquier
On Thu, Jan 11, 2024 at 10:01:16AM -0500, Tom Lane wrote: > Alvaro Herrera writes: >> "The logical replication launcher is disabled during binary upgrades, to >> avoid logical replication workers running on the source cluster. That >> would cause replication origins to move forward after having

Re: Built-in CTYPE provider

2024-01-11 Thread Jeff Davis
On Tue, 2024-01-09 at 14:17 -0800, Jeremy Schneider wrote: > I think we missed something in psql, pretty sure I applied all the > patches but I see this error: > > =# \l > ERROR:  42703: column d.datlocale does not exist > LINE 8:   d.datlocale as "Locale", >   ^ > HINT:  Perhaps you

Re: Adding facility for injection points (or probe points?) for more advanced tests

2024-01-11 Thread Michael Paquier
On Thu, Jan 11, 2024 at 02:47:27PM +0530, Ashutosh Bapat wrote: > On Thu, Jan 11, 2024 at 9:42 AM Michael Paquier wrote: >> I don't really aim for complicated here, just useful. > > It isn't complicated. Such simple error check improve user's > confidence on the feature and better be part of the

Re: Streaming I/O, vectored I/O (WIP)

2024-01-11 Thread Thomas Munro
On Fri, Jan 12, 2024 at 3:31 AM Heikki Linnakangas wrote: > Ok. It feels surprising to have three steps. I understand that you need > two steps, one to start the I/O and another to wait for them to finish, > but why do you need separate Prepare and Start steps? What can you do in > between them?

Re: speed up a logical replica setup

2024-01-11 Thread Euler Taveira
On Thu, Jan 11, 2024, at 9:18 AM, Hayato Kuroda (Fujitsu) wrote: > > I have been concerned that the patch has not been tested by cfbot due to the > application error. Also, some comments were raised. Therefore, I created a > patch > to move forward. > I also tried to address some comments which

Re: Recovering from detoast-related catcache invalidations

2024-01-11 Thread Tom Lane
Xiaoran Wang writes: >>> The detection of "get an invalidation" could be refined: what I did >>> here is to check for any advance of SharedInvalidMessageCounter, >>> which clearly will have a significant number of false positives. > I have reviewed your patch, and it looks good. But instead of

Re: speed up a logical replica setup

2024-01-11 Thread Euler Taveira
On Thu, Jan 11, 2024, at 9:18 AM, Hayato Kuroda (Fujitsu) wrote: > I have been concerned that the patch has not been tested by cfbot due to the > application error. Also, some comments were raised. Therefore, I created a > patch > to move forward. Let me send an updated patch to hopefully keep

Re: Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2024-01-11 Thread Michael Paquier
On Thu, Jan 11, 2024 at 11:00:01PM +0300, Alexander Lakhin wrote: > Bertrand, I've relaunched tests in the same slowed down VM with both > patches applied (but with no other modifications) and got a failure > with pg_class, similar to what we had seen before: > 9   #   Failed test 'activeslot

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-11 Thread Robert Haas
On Thu, Jan 11, 2024 at 2:30 PM Melanie Plageman wrote: > Attached v7 is rebased over the commit you just made to remove > LVPagePruneState->hastup. Apologies for making you rebase but I was tired of thinking about that patch. I'm still kind of hung up on the changes that 0001 makes to

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-11 Thread Melanie Plageman
Attached v7 is rebased over the commit you just made to remove LVPagePruneState->hastup. On Thu, Jan 11, 2024 at 11:54 AM Robert Haas wrote: > > On Wed, Jan 10, 2024 at 5:28 PM Melanie Plageman > wrote: > > Yes, the options I can think of are: > > > > 1) rename the parameter to "immed_reap" or

buildfarm failures in pg_walsummary checks

2024-01-11 Thread Robert Haas
I tried to post this elsewhere but it got moderated, so retrying: On Thu, Jan 11, 2024 at 1:49 PM Robert Haas wrote: > On Thu, Jan 11, 2024 at 12:56 PM Robert Haas wrote: > > Add new pg_walsummary tool. > > culicidae is unhappy with this, but I don't yet understand why. The output is: > > #

Re: Built-in CTYPE provider

2024-01-11 Thread Jeff Davis
On Wed, 2024-01-10 at 23:56 +0100, Daniel Verite wrote: > $ bin/initdb --locale=C.UTF-8 --locale-provider=builtin -D/tmp/pgdata >   >   The database cluster will be initialized with this locale > configuration: >     default collation provider:  builtin >     default collation locale:    C.UTF-8 >

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-11 Thread Robert Haas
On Tue, Jan 9, 2024 at 2:35 PM Andres Freund wrote: > I don't have that strong feelings about it. If both of you think it looks > good, go ahead... Done. -- Robert Haas EDB: http://www.enterprisedb.com

Re: cleanup patches for incremental backup

2024-01-11 Thread Robert Haas
On Tue, Jan 9, 2024 at 1:18 PM Robert Haas wrote: > Here's v2. I plan to commit the rest of this fairly soon if there are > no comments. Done, with a minor adjustment to 0003. -- Robert Haas EDB: http://www.enterprisedb.com

Re: Recovering from detoast-related catcache invalidations

2024-01-11 Thread Xiaoran Wang
Hi, >> BTW, while nosing around I found what seems like a very nasty related >> bug. Suppose that a catalog tuple being loaded into syscache contains >> some toasted fields. CatalogCacheCreateEntry will flatten the tuple, >> involving fetches from toast tables that will certainly cause >>

Re: Stack overflow issue

2024-01-11 Thread Robert Haas
On Wed, Jan 10, 2024 at 4:25 PM Heikki Linnakangas wrote: > The problem with CommitTransactionCommand (or rather > AbortCurrentTransaction() which has the same problem) > and ShowTransactionStateRec is that they get called in a state where > aborting can lead to a panic. If you add a

Re: Compile warnings in dbcommands.c building with meson

2024-01-11 Thread Magnus Hagander
On Wed, Jan 10, 2024 at 1:16 PM Aleksander Alekseev wrote: > > Hi, > > > When building current head on debian bullseye I get this compile warning: > > > > In file included from ../src/backend/commands/dbcommands.c:20: > > ../src/backend/commands/dbcommands.c: In function ‘createdb’: > >

Suggest two small improvements for PITR.

2024-01-11 Thread Yura Sokolov
Good day, hackers. Here I am to suggest two small improvements to Point In Time Recovery. First is ability to recover recovery-target-time with timestamp stored in XLOG_RESTORE_POINT. Looks like historically this ability did exist and were removed unintentionally during refactoring at commit

Re: recovery modules

2024-01-11 Thread Nathan Bossart
rebased -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 6cee7d220b886e9be309929da5274c5770e59845 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Wed, 15 Feb 2023 14:28:53 -0800 Subject: [PATCH v17 1/5] introduce routine for checking mutually exclusive string GUCs ---

Re: System username in pg_stat_activity

2024-01-11 Thread Bertrand Drouvot
Hi, On Thu, Jan 11, 2024 at 02:24:58PM +0100, Magnus Hagander wrote: > On Wed, Jan 10, 2024 at 3:12 PM Bertrand Drouvot > wrote: > > > > If we go the 2 fields way, then what about auth_identity and auth_method > > then? > > > Here is an updated patch based on this idea. Thanks! + +

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-11 Thread Robert Haas
On Wed, Jan 10, 2024 at 5:28 PM Melanie Plageman wrote: > Yes, the options I can think of are: > > 1) rename the parameter to "immed_reap" or similar and make very clear > in heap_page_prune_opt() that you are not to pass true. > 2) make immediate reaping work for on-access pruning. I would need

Re: A failure in t/038_save_logical_slots_shutdown.pl

2024-01-11 Thread Bharath Rupireddy
On Thu, Jan 11, 2024 at 4:35 PM vignesh C wrote: > > On further analysis, it was found that in the failing test, > CHECKPOINT_SHUTDOWN was started in a new page, so there was the WAL > page header present just before the CHECKPOINT_SHUTDOWN which was > causing the failure. We could alternatively

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-11 Thread Jelte Fennema-Nio
On Wed, 10 Jan 2024 at 23:53, Jelte Fennema-Nio wrote: > Honestly, I care more about patch 0010 than patch 0008. Patch 0008 > simply seemed like the easiest way to demonstrate the ParameterSet > message. So to be clear, if you consider 0008 the most controversial/risky part of this patchset

Re: Synchronizing slots from primary to standby

2024-01-11 Thread Bertrand Drouvot
Hi, On Thu, Jan 11, 2024 at 04:22:56PM +0530, Amit Kapila wrote: > On Tue, Jan 9, 2024 at 6:39 PM Amit Kapila wrote: > > > > +static bool > > +synchronize_one_slot(WalReceiverConn *wrconn, RemoteSlot *remote_slot) > > { > > ... > > + /* Slot ready for sync, so sync it. */ > > + else > > + { > >

Re: [BUG] autovacuum may skip tables when session_authorization/role is set on database

2024-01-11 Thread vignesh C
On Thu, 14 Dec 2023 at 02:13, Imseih (AWS), Sami wrote: > > Hi, > > > > A recent case in the field in which a database session_authorization is > > altered to a non-superuser, non-owner of tables via alter database .. set > session_authorization .. > > caused autovacuum to skip tables. > > > >

Re: [PATCH] LockAcquireExtended improvement

2024-01-11 Thread vignesh C
On Tue, 28 Nov 2023 at 18:23, Jingxian Li wrote: > > Hi hackers, > > I found a problem when doing the test shown below: > > Time > > Session A > > Session B > > T1 > > postgres=# create table test(a int); > > CREATE TABLE > > postgres=# insert into test values (1); > > INSERT 0 1 > > > > T2 > >

Re: Wrong results with grouping sets

2024-01-11 Thread vignesh C
On Thu, 7 Dec 2023 at 13:52, Richard Guo wrote: > > > On Mon, Sep 25, 2023 at 3:11 PM Richard Guo wrote: >> >> If the grouping expression is a Var or PHV, we can just set its >> nullingrels, very straightforward. For an expression that is neither a >> Var nor a PHV, I'm not quite sure how to

Re: Should the archiver process always make sure that the timeline history files exist in the archive?

2024-01-11 Thread vignesh C
On Tue, 29 Aug 2023 at 06:29, Jimmy Yih wrote: > > Thanks for the insightful response! I have attached an updated patch > that moves the proposed logic to the end of StartupXLOG where it seems > more correct to do this. It also helps with backporting (if it's > needed) since the archiver process

Re: Postgres Partitions Limitations (5.11.2.3)

2024-01-11 Thread Laurenz Albe
On Thu, 2024-01-11 at 14:44 +0100, Magnus Hagander wrote: > Thanks, applied and backpatched all the way. Thanks for taking care of that! Yours, Laurenz Albe

Re: A recent message added to pg_upgade

2024-01-11 Thread Tom Lane
Alvaro Herrera writes: > "The logical replication launcher is disabled during binary upgrades, to > avoid logical replication workers running on the source cluster. That > would cause replication origins to move forward after having been copied > to the target cluster, potentially creating

Re: Test slots invalidations in 035_standby_logical_decoding.pl only if dead rows are removed

2024-01-11 Thread Bertrand Drouvot
Hi, On Thu, Jan 11, 2024 at 01:00:00PM +0300, Alexander Lakhin wrote: > Hi Bertrand, > > 10.01.2024 19:32, Bertrand Drouvot wrote: > > > > > is an absent message 'obsolete replication slot "row_removal_activeslot"' > > Looking at > >

Re: heavily contended lwlocks with long wait queues scale badly

2024-01-11 Thread Jonathan S. Katz
On 1/10/24 10:45 PM, Michael Paquier wrote: On Wed, Jan 10, 2024 at 09:17:47PM -0600, Nathan Bossart wrote: Now that commit a4adc31 has had some time to bake and concerns about unintended consequences may have abated, I wanted to revive this back-patching discussion. I see a few

Re: SQL:2011 application time

2024-01-11 Thread Peter Eisentraut
On 31.12.23 09:51, Paul Jungwirth wrote: On Wed, Dec 6, 2023 at 12:59 AM Peter Eisentraut wrote: > > On 02.12.23 19:41, Paul Jungwirth wrote: > > So what do you think of this idea instead?: > > > > We could add a new (optional) support function to GiST that translates > > "well-known"

Re: Streaming I/O, vectored I/O (WIP)

2024-01-11 Thread Heikki Linnakangas
On 11/01/2024 05:19, Thomas Munro wrote: On Thu, Jan 11, 2024 at 8:58 AM Heikki Linnakangas wrote: On 10/01/2024 06:13, Thomas Munro wrote: Bikeshedding call: I am open to better suggestions for the names PrepareReadBuffer() and CompleteReadBuffers(), they seem a little grammatically clumsy.

Re: Issue in postgres_fdw causing unnecessary wait for cancel request reply

2024-01-11 Thread vignesh C
On Thu, 13 Apr 2023 at 23:34, Fujii Masao wrote: > > > > On 2023/04/13 11:00, Kyotaro Horiguchi wrote: > > Agreed, it seems to be a leftover when we moved to parse_int_param() > > in that area. > > It looks like there was an oversight in commit e7a2217978. I've attached a > patch (0002) that

Re: Show WAL write and fsync stats in pg_stat_io

2024-01-11 Thread Melanie Plageman
On Thu, Jan 11, 2024 at 6:19 AM Nazir Bilal Yavuz wrote: > > On Thu, 11 Jan 2024 at 08:01, Michael Paquier wrote: > > > > On Wed, Jan 10, 2024 at 07:24:50PM -0500, Melanie Plageman wrote: > > > On Wed, Jan 3, 2024 at 8:11 AM Nazir Bilal Yavuz > > > wrote: > > >> On Sun, 31 Dec 2023 at 03:58,

Re: Assertion failure in SnapBuildInitialSnapshot()

2024-01-11 Thread vignesh C
On Thu, 9 Feb 2023 at 12:02, Masahiko Sawada wrote: > > On Wed, Feb 8, 2023 at 1:13 PM Amit Kapila wrote: > > > > On Wed, Feb 8, 2023 at 1:19 AM Andres Freund wrote: > > > > > > On 2023-02-01 11:23:57 +0530, Amit Kapila wrote: > > > > On Tue, Jan 31, 2023 at 6:08 PM Masahiko Sawada > > > >

Re: Set log_lock_waits=on by default

2024-01-11 Thread Michael Banck
Hi, On Thu, Dec 21, 2023 at 02:29:05PM +0100, Laurenz Albe wrote: > Here is a patch to implement this. > Being stuck behind a lock for more than a second is almost > always a problem, so it is reasonable to turn this on by default. I also think that this should be set to on. I had a look at the

Re: odd buildfarm failure - "pg_ctl: control file appears to be corrupt"

2024-01-11 Thread vignesh C
On Tue, 17 Oct 2023 at 04:18, Thomas Munro wrote: > > I pushed the retry-loop-in-frontend-executables patch and the > missing-locking-in-SQL-functions patch yesterday. That leaves the > backup ones, which I've rebased and attached, no change. It sounds > like we need some more healthy debate

Re: Error "initial slot snapshot too large" in create replication slot

2024-01-11 Thread vignesh C
On Sat, 6 Jan 2024 at 01:47, Robert Haas wrote: > > On Thu, Mar 23, 2023 at 11:02 PM Kyotaro Horiguchi > wrote: > > [ new patch ] > > Well, I guess nobody is too excited about fixing this, because it's > been another 10 months with no discussion. Andres doesn't even seem to > think this is as

Re: Slightly improved meson error for docs tools

2024-01-11 Thread Magnus Hagander
On Wed, Jan 10, 2024 at 1:05 PM Aleksander Alekseev wrote: > > Hi, > > > least lists what's expected -- I'm not sure if this is the way to go, > > or if we should somehow list the individual tools that are failing > > here? > > Personally I think the patch is OK. Thanks. I've pushed this one for

Re: Postgres Partitions Limitations (5.11.2.3)

2024-01-11 Thread Magnus Hagander
On Thu, Jan 11, 2024 at 11:24 AM Ashutosh Bapat wrote: > > On Wed, Jan 10, 2024 at 10:38 PM Laurenz Albe > wrote: > > > > On Wed, 2024-01-10 at 13:41 +0100, Magnus Hagander wrote: > > > It still reads a bit weird to me. How about the attached wording instead? > > > > Thanks! I am fine with

Re: System username in pg_stat_activity

2024-01-11 Thread Magnus Hagander
On Wed, Jan 10, 2024 at 3:12 PM Bertrand Drouvot wrote: > > Hi, > > On Wed, Jan 10, 2024 at 02:59:42PM +0100, Magnus Hagander wrote: > > On Wed, Jan 10, 2024 at 2:56 PM Bertrand Drouvot > > I definitely think it should be the same. If it's not exactly the > > same, then it should be *two*

Re: Fix log_line_prefix to display the transaction id (%x) for statements not in a transaction block

2024-01-11 Thread jian he
Hi. + /* Log immediately if dictated by log_statement and XID assigned. */ + if (GetTopTransactionIdIfAny() != InvalidTransactionId && + check_log_statement(parsetree_list)) change to + /* Log immediately if dictated by log_statement and XID assigned. */ + if (

Re: tablecmds.c/MergeAttributes() cleanup

2024-01-11 Thread Alvaro Herrera
On 2023-Dec-06, Peter Eisentraut wrote: > One of your (Álvaro's) comments about this earlier was > > > Hmm, crazy. I'm not sure I like this, because it seems much too clever. > > The number of lines that are deleted is alluring, though. > > > > Maybe it'd be better to create a separate routine

Re: make pg_ctl more friendly

2024-01-11 Thread Nazir Bilal Yavuz
Hi, On Wed, 10 Jan 2024 at 06:33, Junwang Zhao wrote: > > Hi Nazir, > > On Tue, Jan 9, 2024 at 9:23 PM Nazir Bilal Yavuz wrote: > > > > v2-0001-PITR-shutdown-should-not-report-error-by-pg_ctl.patch: > > > > - "Examine the log output.\n"), > > +

RE: speed up a logical replica setup

2024-01-11 Thread Hayato Kuroda (Fujitsu)
Dear hackers, I have been concerned that the patch has not been tested by cfbot due to the application error. Also, some comments were raised. Therefore, I created a patch to move forward. I also tried to address some comments which is not so claimed by others. They were included in 0003 patch.

Re: A recent message added to pg_upgade

2024-01-11 Thread Alvaro Herrera
On 2024-Jan-11, Michael Paquier wrote: > Hence, how about something like that: > "The logical replication launcher is disabled during binary upgrades, > as a logical replication workers running on the cluster upgrading from > could cause replication origins to move forward after they are copied >

Re: Synchronizing slots from primary to standby

2024-01-11 Thread Amit Kapila
On Thu, Jan 11, 2024 at 3:42 PM Dilip Kumar wrote: > > On Wed, Jan 10, 2024 at 5:53 PM Zhijie Hou (Fujitsu) > wrote: > > > > > > IIUC on the standby we just want to overwrite what we get from primary > > > no? If > > > so why we are using those APIs that are meant for the actual decoding > > >

Re: POC: GROUP BY optimization

2024-01-11 Thread Alexander Korotkov
new version of the whole patchset > should be sent even if most patches are unchanged. Please, find the revised patchset with some refactoring and comments improvement from me. I'll continue looking into this. -- Regards, Alexander Korotkov 0001-Generalize-common-code-of-addin

Re: Make all Perl warnings fatal

2024-01-11 Thread Bharath Rupireddy
On Sat, Dec 30, 2023 at 12:57 AM Peter Eisentraut wrote: > > committed With the commit c5385929 converting perl warnings to FATAL, use of psql/safe_psql with timeout parameters [1] fail with the following error: Use of uninitialized value $ret in bitwise and (&) at

Re: Show WAL write and fsync stats in pg_stat_io

2024-01-11 Thread Nazir Bilal Yavuz
Hi, On Thu, 11 Jan 2024 at 08:01, Michael Paquier wrote: > > On Wed, Jan 10, 2024 at 07:24:50PM -0500, Melanie Plageman wrote: > > I have code review feedback as well, but I've saved that for my next email. > > Ah, cool. > > > On Wed, Jan 3, 2024 at 8:11 AM Nazir Bilal Yavuz wrote: > >> On Sun,

Re: proposal: psql: show current user in prompt

2024-01-11 Thread Jelte Fennema-Nio
On Tue, 12 Sept 2023 at 09:46, Peter Eisentraut wrote: > ISTM that for a purpose like pgbouncer, it would be simpler to add a new > GUC "report these variables" and send that in the startup message? That > might not help with the psql use case, but it would be much simpler. FYI I implemented it

Re: Clean up some signal usage mainly related to Windows

2024-01-11 Thread vignesh C
On Thu, 7 Dec 2023 at 04:50, Nathan Bossart wrote: > > On Wed, Dec 06, 2023 at 11:30:02AM -0600, Nathan Bossart wrote: > > On Wed, Dec 06, 2023 at 06:27:04PM +0100, Peter Eisentraut wrote: > >> Makes sense. Can you commit that? > > > > Yes, I will do so shortly. > > Committed. Apologies for the

Re: A failure in t/038_save_logical_slots_shutdown.pl

2024-01-11 Thread vignesh C
On Wed, 10 Jan 2024 at 18:37, vignesh C wrote: > > On Wed, 10 Jan 2024 at 14:08, Bharath Rupireddy > wrote: > > > > Hi, > > > > I've been observing a failure in t/038_save_logical_slots_shutdown.pl > > of late on my developer system: > > > > t/038_save_logical_slots_shutdown.pl .. 1/? > > #

Re: Synchronizing slots from primary to standby

2024-01-11 Thread Amit Kapila
On Tue, Jan 9, 2024 at 6:39 PM Amit Kapila wrote: > > +static bool > +synchronize_one_slot(WalReceiverConn *wrconn, RemoteSlot *remote_slot) > { > ... > + /* Slot ready for sync, so sync it. */ > + else > + { > + /* > + * Sanity check: With hot_standby_feedback enabled and > + * invalidations

Re: Postgres Partitions Limitations (5.11.2.3)

2024-01-11 Thread Ashutosh Bapat
On Wed, Jan 10, 2024 at 10:38 PM Laurenz Albe wrote: > > On Wed, 2024-01-10 at 13:41 +0100, Magnus Hagander wrote: > > It still reads a bit weird to me. How about the attached wording instead? > > Thanks! I am fine with your wording. Works for me too. -- Best Wishes, Ashutosh Bapat

Re: Make attstattarget nullable

2024-01-11 Thread Peter Eisentraut
On 10.01.24 14:16, Alvaro Herrera wrote: Here is an updated patch rebased over 3e2e0d5ad7. The 0001 patch stands on its own, but I also tacked on two additional WIP patches that simplify some pg_attribute handling and make these kinds of refactorings simpler in the future. See description in

Re: Synchronizing slots from primary to standby

2024-01-11 Thread Dilip Kumar
On Wed, Jan 10, 2024 at 5:53 PM Zhijie Hou (Fujitsu) wrote: > > > IIUC on the standby we just want to overwrite what we get from primary no? > > If > > so why we are using those APIs that are meant for the actual decoding slots > > where it needs to take certain logical decisions instead of

Re: Exposing the lock manager's WaitForLockers() to SQL

2024-01-11 Thread Will Mortensen
Here is a new series adding a single pg_wait_for_lockers() function that takes a boolean argument to control the interpretation of the lock mode. It omits LOCK's handling of descendant tables so it requires permissions directly on descendants in order to wait for locks on them. Not sure if that

Re: Synchronizing slots from primary to standby

2024-01-11 Thread shveta malik
On Thu, Jan 11, 2024 at 7:28 AM Peter Smith wrote: > > Here are some review comments for patch v58-0002 Thank You for the feedback. These are addressed in v60. Please find my response inline for a few. > (FYI - I quickly checked with the latest v59-0002 and AFAIK all these > review comments

Re: Random pg_upgrade test failure on drongo

2024-01-11 Thread Amit Kapila
On Thu, Jan 11, 2024 at 8:15 AM Hayato Kuroda (Fujitsu) wrote: > > > > But tomorrow it could be for other tables and if we change this > > > TRUNCATE logic for pg_largeobject (of which chances are less) then > > > there is always a chance that one misses changing this comment. I feel > > >

Fix bugs not to discard statistics when changing stats_fetch_consistency

2024-01-11 Thread Shinya Kato
Hi, hackers There is below description in docs for stats_fetch_consistency. "Changing this parameter in a transaction discards the statistics snapshot." However, I wonder if changes stats_fetch_consistency in a transaction, statistics is not discarded in some cases. Example: -- * session 1

Re: Adding facility for injection points (or probe points?) for more advanced tests

2024-01-11 Thread Ashutosh Bapat
On Thu, Jan 11, 2024 at 9:42 AM Michael Paquier wrote: > > >> Yeah, that's an intended design choice to keep the code simpler and > >> faster as there is no need to track the library and function names in > >> the local caches or implement something similar to invalidation > >> messages for this

Re: Update docs for default value of fdw_tuple_cost

2024-01-11 Thread Umair Shahid
On Thu, Jan 11, 2024 at 7:07 AM John Naylor wrote: > On Tue, Dec 26, 2023 at 11:27 PM Umair Shahid > wrote: > > > > Commit cac169d686eddb277880a0d8a760ac3007b4846a updated the default > value of fdw_tuple_cost from 0.01 to 0.2. The attached patch updates the > docs to reflect this change. > >

Re: Is the subtype_diff function in CREATE TYPE only can be C function?

2024-01-11 Thread ddme
> 2024年1月10日 18:04,Ashutosh Bapat 写道: > > On Wed, Jan 10, 2024 at 1:49 PM ddme > wrote: >> >> Hi all, >> >> I notice that the CREATE TYPE syntax can specify subtype_diff function >> >> CREATE TYPE name AS RANGE ( >>SUBTYPE = subtype >>[ , SUBTYPE_OPCLASS =

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-01-11 Thread Kartyshov Ivan
Rebased and ready for review. I left only versions (due to irreparable problems) 1) Classic (wait_classic_v4.patch) https://www.postgresql.org/message-id/3cc883048264c2e9af022033925ff8db%40postgrespro.ru == advantages: multiple wait events, separate WAIT FOR statement disadvantages: new

Improve the connection failure error messages

2024-01-11 Thread Nisha Moond
Hi Hackers, Various sections of the code utilize the walrcv_connect() function, employed by various processes such as walreceiver, logical replication apply worker, etc., to establish connections with other hosts. Presently, in case of connection failures, the error message lacks information

Re: pg_ctl start may return 0 even if the postmaster has been already started on Windows

2024-01-11 Thread Kyotaro Horiguchi
Thanks for restarting this thread. At Tue, 9 Jan 2024 09:40:23 +0900, Michael Paquier wrote in > On Fri, Jan 05, 2024 at 02:58:55PM -0500, Robert Haas wrote: > > I'm not a Windows expert, but my guess is that 0001 is a very good > > idea. I hope someone who is a Windows expert will comment on

  1   2   >