Re: Synchronizing slots from primary to standby

2024-02-22 Thread Bertrand Drouvot
Hi, On Fri, Feb 23, 2024 at 09:43:48AM +0530, shveta malik wrote: > On Fri, Feb 23, 2024 at 8:35 AM shveta malik wrote: > > > > On Thu, Feb 22, 2024 at 4:35 PM Bertrand Drouvot > > wrote: > > > > > > Suppose that in synchronize_slots() the query would be: > > > > > > const char *query =

Re: Synchronizing slots from primary to standby

2024-02-22 Thread Bertrand Drouvot
Hi, On Fri, Feb 23, 2024 at 08:35:44AM +0530, shveta malik wrote: > On Thu, Feb 22, 2024 at 4:35 PM Bertrand Drouvot > wrote: > > > > Suppose that in synchronize_slots() the query would be: > > > > const char *query = "SELECT slot_name, plugin, confirmed_flush_lsn," > > "

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2024-02-22 Thread Dilip Kumar
On Fri, Feb 23, 2024 at 1:48 AM Alvaro Herrera wrote: > > On 2024-Feb-07, Dilip Kumar wrote: > > > On Wed, Feb 7, 2024 at 3:49 PM Alvaro Herrera > > wrote: > > > > Sure, but is that really what we want? > > > > So your question is do we want these buffers to be in multiple of > >

Re: Have pg_basebackup write "dbname" in "primary_conninfo"?

2024-02-22 Thread Amit Kapila
On Wed, Feb 21, 2024 at 8:34 AM Kyotaro Horiguchi wrote: > > At Tue, 20 Feb 2024 19:56:10 +0530, Robert Haas wrote > in > > It seems like maybe somebody should look into why this is happening, > > and perhaps fix it. > > GetConnection()@streamutil.c wants to ensure conninfo has a fallback >

Re: Have pg_basebackup write "dbname" in "primary_conninfo"?

2024-02-22 Thread Amit Kapila
On Wed, Feb 21, 2024 at 7:46 AM Hayato Kuroda (Fujitsu) wrote: > > > > Just FYI - here is an extreme case. And note that I have applied proposed > > > patch. > > > > > > When `pg_basebackup -D data_N2 -R` is used: > > > ``` > > > primary_conninfo = 'user=hayato ... dbname=hayato ... > > > ``` >

Re: Synchronizing slots from primary to standby

2024-02-22 Thread shveta malik
On Fri, Feb 23, 2024 at 10:06 AM Zhijie Hou (Fujitsu) wrote: > > > I noticed one CFbot failure[1] which is because the tap-test doesn't wait for > the > standby to catch up before promoting, thus the data inserted after promotion > could not be replicated to the subscriber. Add a

Re: Test to dump and restore objects left behind by regression

2024-02-22 Thread Ashutosh Bapat
On Thu, Feb 22, 2024 at 8:35 PM Tom Lane wrote: > > Peter Eisentraut writes: > > The problem is, we don't really have any end-to-end coverage of > > > dump > > restore > > dump again > > compare the two dumps > > > with a database with lots of interesting objects in it. > > I'm very much against

Re: Test to dump and restore objects left behind by regression

2024-02-22 Thread Ashutosh Bapat
On Thu, Feb 22, 2024 at 3:50 PM Peter Eisentraut wrote: > > On 22.02.24 11:00, Daniel Gustafsson wrote: > >> On 22 Feb 2024, at 10:55, Ashutosh Bapat > >> wrote: > >> On Thu, Feb 22, 2024 at 3:03 PM Daniel Gustafsson wrote: > > > >> Somebody looking for dump/restore tests wouldn't search > >>

Re: RFC: Logging plan of the running query

2024-02-22 Thread Robert Haas
On Thu, Feb 22, 2024 at 6:25 AM James Coleman wrote: > This is potentially a bit of a wild idea, but I wonder if having some > kind of argument to CHECK_FOR_INTERRUPTS() signifying we're in > "normal" as opposed to "critical" (using that word differently than > the existing critical sections)

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-02-22 Thread Ajin Cherian
On Fri, Feb 23, 2024 at 12:29 AM Давыдов Виталий wrote: > Dear All, > > I'd like to present and talk about a problem when 2PC transactions are > applied quite slowly on a replica during logical replication. There is a > master and a replica with established logical replication from the master >

Re: About a recently-added message

2024-02-22 Thread Peter Smith
On Thu, Feb 22, 2024 at 11:36 AM Kyotaro Horiguchi wrote: > > At Wed, 21 Feb 2024 14:57:42 +0530, Amit Kapila > wrote in > > On Tue, Feb 20, 2024 at 3:21 PM shveta malik wrote: > > > > > > okay, attached v2 patch with changed error msgs and double quotes > > > around logical. > > > > > > >

Re: speed up a logical replica setup

2024-02-22 Thread Amit Kapila
On Fri, Feb 23, 2024 at 8:16 AM Euler Taveira wrote: > > On Wed, Feb 21, 2024, at 5:00 AM, Shlok Kyal wrote: > > I found some issues and fixed those issues with top up patches > v23-0012 and v23-0013 > 1. > Suppose there is a cascade physical replication node1->node2->node3. > Now if we run

Re: LogwrtResult contended spinlock

2024-02-22 Thread Robert Haas
On Fri, Feb 23, 2024 at 1:18 AM Jeff Davis wrote: > I don't see the global non-shared variable as a huge problem, so if it > serves a purpose then I'm fine keeping it. Perhaps we could make it a > bit safer by using some wrapper functions. I actually really hate these kinds of variables. I think

RE: Synchronizing slots from primary to standby

2024-02-22 Thread Zhijie Hou (Fujitsu)
On Friday, February 23, 2024 10:18 AM Zhijie Hou (Fujitsu) wrote: > > > > Hi, > > > > Since the slotsync worker patch has been committed, I rebased the > > remaining patches. > > And here is the V95 patch set. > > > > Also, I fixed a bug in the current 0001 patch where the member of the > >

Re: Synchronizing slots from primary to standby

2024-02-22 Thread shveta malik
On Fri, Feb 23, 2024 at 8:35 AM shveta malik wrote: > > On Thu, Feb 22, 2024 at 4:35 PM Bertrand Drouvot > wrote: > > > > Suppose that in synchronize_slots() the query would be: > > > > const char *query = "SELECT slot_name, plugin, confirmed_flush_lsn," > > " restart_lsn,

Re: Why is subscription/t/031_column_list.pl failing so much?

2024-02-22 Thread Amit Kapila
On Mon, Feb 19, 2024 at 2:19 PM Amit Kapila wrote: > > On Thu, Feb 15, 2024 at 10:46 AM vignesh C wrote: > > > > On Thu, 15 Feb 2024 at 08:42, Amit Kapila wrote: > > > > > > > > To resolve the BF > > > failure, I still feel, we should just recreate the subscription. This > > > is a pre-existing

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-02-22 Thread Amit Kapila
On Thu, Feb 22, 2024 at 6:59 PM Давыдов Виталий wrote: > > I'd like to present and talk about a problem when 2PC transactions are > applied quite slowly on a replica during logical replication. There is a > master and a replica with established logical replication from the master to > the

Re: Synchronizing slots from primary to standby

2024-02-22 Thread shveta malik
On Thu, Feb 22, 2024 at 4:35 PM Bertrand Drouvot wrote: > > Suppose that in synchronize_slots() the query would be: > > const char *query = "SELECT slot_name, plugin, confirmed_flush_lsn," > " restart_lsn, catalog_xmin, two_phase, failover," > " database, conflict_reason" >

Re: speed up a logical replica setup

2024-02-22 Thread Euler Taveira
On Wed, Feb 21, 2024, at 5:00 AM, Shlok Kyal wrote: > I found some issues and fixed those issues with top up patches > v23-0012 and v23-0013 > 1. > Suppose there is a cascade physical replication node1->node2->node3. > Now if we run pg_createsubscriber with node1 as primary and node2 as > standby,

Re: Add publisher and subscriber to glossary documentation.

2024-02-22 Thread Peter Smith
Here are some comments for patch v2. == 1. There are whitespace problems [postgres@CentOS7-x64 oss_postgres_misc]$ git apply ../patches_misc/v2-0001-Add-publisher-and-subscriber-to-glossary-document.patch ../patches_misc/v2-0001-Add-publisher-and-subscriber-to-glossary-document.patch:43:

RE: Synchronizing slots from primary to standby

2024-02-22 Thread Zhijie Hou (Fujitsu)
On Friday, February 23, 2024 10:02 AM Zhijie Hou (Fujitsu) wrote: > > Hi, > > Since the slotsync worker patch has been committed, I rebased the remaining > patches. > And here is the V95 patch set. > > Also, I fixed a bug in the current 0001 patch where the member of the standby > slot names

RE: Synchronizing slots from primary to standby

2024-02-22 Thread Zhijie Hou (Fujitsu)
Hi, Since the slotsync worker patch has been committed, I rebased the remaining patches. And here is the V95 patch set. Also, I fixed a bug in the current 0001 patch where the member of the standby slot names list pointed to the freed memory after calling ProcessConfigFile(). Now, we will

Re: Improve readability by using designated initializers when possible

2024-02-22 Thread Jeff Davis
On Fri, 2024-02-23 at 01:35 +0100, Jelte Fennema-Nio wrote: > On Thu, Feb 22, 2024, 23:46 Jeff Davis wrote: > > > > Am I missing something? > > The main benefits it has are: Sorry, I was unclear. I was asking a question about the reason the ObjectClass and the object_classes[] array exist in

Re: Improve readability by using designated initializers when possible

2024-02-22 Thread Jelte Fennema-Nio
On Thu, Feb 22, 2024, 23:46 Jeff Davis wrote: > > Am I missing something? The main benefits it has are: 1. The order of the array doesn't have to exactly match the order of the enum for the arrays to contain the correct mapping. 2. Typos in the enum variant names are caught by the compiler

Re: bug report: some issues about pg_15_stable(8fa4a1ac61189efffb8b851ee77e1bc87360c445)

2024-02-22 Thread Tom Lane
I wrote: > I think that this is a band-aid that's just masking an error in the > rowmarking logic: it's not doing the right thing for appendrels > made from UNION ALL subqueries. I've not wrapped my head around > exactly where it's going off the rails, but I feel like this ought > to get fixed

Re: Running the fdw test from the terminal crashes into the core-dump

2024-02-22 Thread Tom Lane
Alvaro Herrera writes: > After having pushed that, I wonder if we should document this. It seems > quite the minor thing, but I'm sure somebody will complain if we don't. Yup, no doubt. > I propose the attached. (Extra context so that the full paragraph can > be read from the comfort of your

Re: WIP Incremental JSON Parser

2024-02-22 Thread Andrew Dunstan
On 2024-02-22 Th 15:29, Jacob Champion wrote: On Thu, Feb 22, 2024 at 1:38 AM Andrew Dunstan wrote: Patch 5 in this series fixes those issues and adjusts most of the tests to add some trailing junk to the pieces of json, so we can be sure that this is done right. This fixes the test failure

Re: Documentation to upgrade logical replication cluster

2024-02-22 Thread Peter Smith
Hi Vignesh, I have no further comments. Patch v9 LGTM. == Kind Regards, Peter Smith. Fujitsu Australia

Re: Avoid stack frame setup in performance critical routines using tail calls

2024-02-22 Thread Andres Freund
Hi, On 2024-02-23 00:46:26 +1300, David Rowley wrote: > I've rebased the 0001 patch and gone over it again and made a few > additional changes besides what I mentioned in my review. > > On Wed, 9 Aug 2023 at 20:44, David Rowley wrote: > > Here's a review of v2-0001: > > 2. Why do you need to add

Re: Improve readability by using designated initializers when possible

2024-02-22 Thread Jeff Davis
On Wed, 2024-02-21 at 16:03 +0100, Jelte Fennema-Nio wrote: > Usage of designated initializers came up in: > https://www.postgresql.org/message-id/flat/ZdWXhAt9Tz4d-lut%40paquier.xyz#9dc17e604e58569ad35643672bf74acc > > This converts all arrays that I could find that could clearly benefit > from

Re: Add lookup table for replication slot invalidation causes

2024-02-22 Thread Peter Smith
On Thu, Feb 22, 2024 at 5:56 PM Michael Paquier wrote: > > On Thu, Feb 22, 2024 at 05:30:08PM +1100, Peter Smith wrote: > > Oops. Perhaps I meant more like below -- in any case, the point was > > the same -- to ensure RS_INVAL_NONE is what returns if something > > unexpected happens. > > You are

Re: Oom on temp (un-analyzed table caused by JIT) V16.1 [ NOT Fixed ]

2024-02-22 Thread Michael Banck
Hi, On Wed, Jan 24, 2024 at 02:50:52PM -0500, Kirk Wolak wrote: > On Mon, Jan 22, 2024 at 1:30 AM Kirk Wolak wrote: > > On Fri, Jan 19, 2024 at 7:03 PM Daniel Gustafsson wrote: > >> > On 19 Jan 2024, at 23:09, Kirk Wolak wrote: > > Thank you, that made it possible to build and run... > >

Re: speed up a logical replica setup

2024-02-22 Thread Euler Taveira
On Thu, Feb 22, 2024, at 9:43 AM, Hayato Kuroda (Fujitsu) wrote: > > The possible solution would be > > 1) allow to run pg_createsubscriber if standby is initially stopped . > > I observed that pg_logical_createsubscriber also uses this approach. > > 2) read GUCs via SHOW command and restore them

Re: WIP Incremental JSON Parser

2024-02-22 Thread Jacob Champion
On Thu, Feb 22, 2024 at 1:38 AM Andrew Dunstan wrote: > Patch 5 in this series fixes those issues and > adjusts most of the tests to add some trailing junk to the pieces of > json, so we can be sure that this is done right. This fixes the test failure for me, thanks! I've attached my current

Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock

2024-02-22 Thread Alvaro Herrera
On 2024-Feb-07, Dilip Kumar wrote: > On Wed, Feb 7, 2024 at 3:49 PM Alvaro Herrera wrote: > > Sure, but is that really what we want? > > So your question is do we want these buffers to be in multiple of > SLRU_BANK_SIZE? Maybe we can have the last bank to be partial, I > don't think it should

Re: Running the fdw test from the terminal crashes into the core-dump

2024-02-22 Thread Alvaro Herrera
On 2024-Feb-20, Tom Lane wrote: > > So, this means we can fix this by simply requiring ACL_SELECT privileges > > on a DO NOTHING action. We don't need to request specific privileges on > > any particular column (perminfo->selectedCols continues to be the empty > > set) -- which means that any

Re: speed up a logical replica setup

2024-02-22 Thread 'Alvaro Herrera'
Hello, On 2024-Feb-22, Hayato Kuroda (Fujitsu) wrote: > Dear Alvaro, > > Hmm, but doesn't this mean that the server will log an ugly message > > that "client closed connection unexpectedly"? I think it's nicer to > > close the connection before terminating the process (especially > > since the

Re: locked reads for atomics

2024-02-22 Thread Jeff Davis
On Thu, 2024-02-22 at 12:58 +0530, Bharath Rupireddy wrote: > There's some immediate use for reads/writes with barrier semantics - Is this mainly a convenience for safety/readability? Or is it faster in some cases than doing an atomic access with separate memory barriers? Regards, Jeff

Re: LogwrtResult contended spinlock

2024-02-22 Thread Jeff Davis
On Thu, 2024-02-22 at 10:17 +0530, Robert Haas wrote: > I think the problems tend to be worst when you have some bit of data > that's being frequently modified by multiple backends. Every backend > that wants to modify the value needs to steal the cache line, and > eventually you spend most of

Re: Unlinking Parallel Hash Join inner batch files sooner

2024-02-22 Thread Thomas Munro
On Thu, Feb 22, 2024 at 5:37 PM Andrei Lepikhov wrote: > On 22/2/2024 06:42, Thomas Munro wrote: > > extreme skew for one version of the problem, but even with zero/normal > > skewness and perfect estimation of the number of partitions, if you Sorry, I meant to write "but even with no

re: Speeding up COPY TO for uuids and arrays

2024-02-22 Thread Ranier Vilela
Hi, On 2024-02-17 17:48:23 +0100, Laurenz Albe wrote: > As a test case, I created a table with 1 rows, each of which > had an array of 1 uuids. The table resided in shared buffers. Can you share exactly script used to create a table? best regards, Ranier Vilela

Re: Optimize planner memory consumption for huge arrays

2024-02-22 Thread Alena Rybakina
Hi! On 20.02.2024 07:41, Andrei Lepikhov wrote: On 20/2/2024 04:51, Tom Lane wrote: Tomas Vondra writes: On 2/19/24 16:45, Tom Lane wrote: Tomas Vondra writes: For example, I don't think we expect selectivity functions to allocate long-lived objects, right? So maybe we could run them in

how to read table options during smgropen()

2024-02-22 Thread Dima Rybakov (Tlt)
Dear pgsql hackers, I am developing custom storage for pgsql tables. I am using md* functions and smgrsw[] structure to switch between different magnetic disk access methods. I want to add some custom options while table created psql# create table t(...) with (my_option='value'); And thus I

Re: Transaction timeout

2024-02-22 Thread Andrey M. Borodin
> On 19 Feb 2024, at 15:17, Japin Li wrote: > > > +1 PFA patch set of 4 patches: 1. remove all potential flaky tests. BTW recently we had a bingo when 3 of them failed together [0] 2-3. waiting injection points patchset by Michael Paquier, intact v2 from nearby thread. 4. prototype of

Re: Experiments with Postgres and SSL

2024-02-22 Thread Heikki Linnakangas
On 22/02/2024 01:43, Matthias van de Meent wrote: On Wed, 10 Jan 2024 at 09:31, Heikki Linnakangas wrote: 4. The number of combinations of sslmode, gssencmode and sslnegotiation settings is scary. And we have very few tests for them. Yeah, it's not great. We could easily automate this better

Potential issue in ecpg-informix decimal converting functions

2024-02-22 Thread a . imamov
Hi, everyone! I found a potential bug in dectoint() and dectolong() functions from informix.c. "Informix Compatibility Mode" doc chapter says that ECPG_INFORMIX_NUM_OVERFLOW is returned if an overflow occurred. But check this line in dectoint() or dectolong() (it is present in both): if (ret ==

Re: Sequence Access Methods, round two

2024-02-22 Thread Tomas Vondra
Hi Michael, I took a quick look at this patch series, mostly to understand how it works and how it might interact with the logical decoding patches discussed in a nearby thread. First, some general review comments: 0001 -- I think this bit in pg_proc.dat is not quite right:

Re: speed up a logical replica setup

2024-02-22 Thread vignesh C
On Thu, 22 Feb 2024 at 21:17, Hayato Kuroda (Fujitsu) wrote: > > Dear Vignesh, > > > Few comments on the tests: > > 1) If the dry run was successful because of some issue then the server > > will be stopped so we can check for "pg_ctl status" if the server is > > running otherwise the connection

Re: bug report: some issues about pg_15_stable(8fa4a1ac61189efffb8b851ee77e1bc87360c445)

2024-02-22 Thread Tom Lane
Dean Rasheed writes: > Attached is a patch that prevents UNION ALL subquery pullup in MERGE only. I think that this is a band-aid that's just masking an error in the rowmarking logic: it's not doing the right thing for appendrels made from UNION ALL subqueries. I've not wrapped my head around

RE: speed up a logical replica setup

2024-02-22 Thread Hayato Kuroda (Fujitsu)
Dear Vignesh, > Few comments on the tests: > 1) If the dry run was successful because of some issue then the server > will be stopped so we can check for "pg_ctl status" if the server is > running otherwise the connection will fail in this case. Another way > would be to check if it does not have

RE: speed up a logical replica setup

2024-02-22 Thread Hayato Kuroda (Fujitsu)
Dear Vignesh, > Few comments: > 1) The below code can lead to assertion failure if the publisher is > stopped while dropping the replication slot: > + if (primary_slot_name != NULL) > + { > + conn = connect_database(dbinfo[0].pubconninfo); > + if (conn !=

RE: speed up a logical replica setup

2024-02-22 Thread Hayato Kuroda (Fujitsu)
Dear Vignesh, > Few comments regarding the documentation: > 1) max_replication_slots information seems to be present couple of times: > > + > + The target instance must have > + linkend="guc-max-replication-slots">max_replication_slots me> > + and

RE: speed up a logical replica setup

2024-02-22 Thread Hayato Kuroda (Fujitsu)
Dear Alvaro, > > 15. > > > > You said in case of failure, cleanups is not needed if the process exits > > soon [1]. > > But some functions call PQfinish() then exit(1) or pg_fatal(). Should we > > follow? > > Hmm, but doesn't this mean that the server will log an ugly message that > "client

Re: btree: downlink right separator/HIKEY optimization

2024-02-22 Thread Matthias van de Meent
On Sat, 6 Jan 2024 at 16:40, vignesh C wrote: > > CFBot shows the following compilation error at [1]: > [16:56:22.153] FAILED: > src/backend/postgres_lib.a.p/access_nbtree_nbtsearch.c.obj > [...] > ../src/backend/access/nbtree/nbtsearch.c > [16:56:22.153]

Re: Reducing output size of nodeToString

2024-02-22 Thread Matthias van de Meent
On Thu, 22 Feb 2024 at 13:37, Matthias van de Meent wrote: > > On Mon, 19 Feb 2024 at 14:19, Matthias van de Meent > wrote: > > Attached the updated version of the patch on top of 5497daf3, which > > incorporates this last round of feedback. > > Now attached rebased on top of 93db6cbd to fix

Re: Test to dump and restore objects left behind by regression

2024-02-22 Thread Tom Lane
Peter Eisentraut writes: > The problem is, we don't really have any end-to-end coverage of > dump > restore > dump again > compare the two dumps > with a database with lots of interesting objects in it. I'm very much against adding another full run of the core regression tests to support this.

Re: Add missing error codes to PANIC/FATAL error reports in xlog.c and relcache.c

2024-02-22 Thread Daniel Gustafsson
> On 6 Dec 2023, at 14:03, Nazir Bilal Yavuz wrote: > There is an ongoing thread [1] for adding missing SQL error codes to > PANIC and FATAL error reports in xlogrecovery.c file. I did the same > but for xlog.c and relcache.c files. - elog(PANIC, "space reserved for WAL record does not

Re: Documentation to upgrade logical replication cluster

2024-02-22 Thread vignesh C
On Thu, 22 Feb 2024 at 09:35, Peter Smith wrote: > > Here are some minor comments for patch v8-0001. > > == > doc/src/sgml/glossary.sgml > > 1. > + > + > + A set of publisher and subscriber instances with publisher instance > + replicating changes to the subscriber instance. >

Re: btree: downlink right separator/HIKEY optimization

2024-02-22 Thread Matthias van de Meent
On Tue, 5 Dec 2023 at 08:43, Heikki Linnakangas wrote: > > On 01/11/2023 00:08, Matthias van de Meent wrote: > > By caching the right separator index tuple in _bt_search, we can > > compare the downlink's right separator and the HIKEY, and when they > > are equal (memcmp() == 0) we don't have to

Slow catchup of 2PC (twophase) transactions on replica in LR

2024-02-22 Thread Давыдов Виталий
Dear All, I'd like to present and talk about a problem when 2PC transactions are applied quite slowly on a replica during logical replication. There is a master and a replica with established logical replication from the master to the replica with twophase = true. With some load level on the

Re: When extended query protocol ends?

2024-02-22 Thread Jelte Fennema-Nio
On Thu, 22 Feb 2024 at 13:01, Michael Zhilin wrote: > I would like to say this document states that "at completion... frontend > should issue a Sync message... causes the backend to close the current > transaction" > It looks like the sense of wording is "to complete transaction" at the >

RE: Synchronizing slots from primary to standby

2024-02-22 Thread Zhijie Hou (Fujitsu)
On Thursday, February 22, 2024 8:41 PM Amit Kapila wrote: > > On Thu, Feb 22, 2024 at 5:23 PM Amit Kapila > wrote: > > > > On Thu, Feb 22, 2024 at 4:35 PM Bertrand Drouvot > > wrote: > > > > > > On Thu, Feb 22, 2024 at 04:01:34PM +0530, shveta malik wrote: > > > > On Thu, Feb 22, 2024 at 3:44 

RE: speed up a logical replica setup

2024-02-22 Thread Hayato Kuroda (Fujitsu)
Dear Shlok, > Hi, > > I have reviewed the v21 patch. And found an issue. > > Initially I started the standby server with a new postgresql.conf file > (not the default postgresql.conf that is present in the instance). > pg_ctl -D ../standby start -o "-c config_file=/new_path/postgresql.conf" >

Re: Synchronizing slots from primary to standby

2024-02-22 Thread Amit Kapila
On Thu, Feb 22, 2024 at 5:23 PM Amit Kapila wrote: > > On Thu, Feb 22, 2024 at 4:35 PM Bertrand Drouvot > wrote: > > > > On Thu, Feb 22, 2024 at 04:01:34PM +0530, shveta malik wrote: > > > On Thu, Feb 22, 2024 at 3:44 PM Bertrand Drouvot > > > wrote: > > > > > > > > Hi, > > > > > > > > Thanks!

Re: Reducing output size of nodeToString

2024-02-22 Thread Matthias van de Meent
On Mon, 19 Feb 2024 at 14:19, Matthias van de Meent wrote: > Attached the updated version of the patch on top of 5497daf3, which > incorporates this last round of feedback. Now attached rebased on top of 93db6cbd to fix conflicts with fbc93b8b and an issue in the previous patchset: I attached

Re: Porting PostgresSQL libraries for QNX710

2024-02-22 Thread Daniel Gustafsson
> On 22 Feb 2024, at 11:35, Rajith Rao .B(App Software) > wrote: > I have been using the Qt IDE with C++ for database connection and query > execution, and unfortunately, I cannot share the code with you. No worries, I have no intention to work on this. > You mentioned that PostgreSQL

Re: When extended query protocol ends?

2024-02-22 Thread Michael Zhilin
Hi, On 2/22/24 14:09, Jelte Fennema-Nio wrote: Apparently, sending an extra message would increase the overhead of the protocol, thus reducing the efficiency of the application. What is the benefit of sending extra Sync?

Re: Synchronizing slots from primary to standby

2024-02-22 Thread Amit Kapila
On Thu, Feb 22, 2024 at 4:35 PM Bertrand Drouvot wrote: > > On Thu, Feb 22, 2024 at 04:01:34PM +0530, shveta malik wrote: > > On Thu, Feb 22, 2024 at 3:44 PM Bertrand Drouvot > > wrote: > > > > > > Hi, > > > > > > Thanks! > > > > > > Some random comments about v92_001 (Sorry if it has already

Re: Avoid stack frame setup in performance critical routines using tail calls

2024-02-22 Thread David Rowley
I've rebased the 0001 patch and gone over it again and made a few additional changes besides what I mentioned in my review. On Wed, 9 Aug 2023 at 20:44, David Rowley wrote: > Here's a review of v2-0001: > 2. Why do you need to add the NULL check here? > > #ifdef USE_VALGRIND > - if (method !=

Re: When extended query protocol ends?

2024-02-22 Thread Jelte Fennema-Nio
On Thu, 22 Feb 2024 at 10:28, Vladimir Sitnikov wrote: > > >When splitting a multi insert statement you're going to duplicate some work > > I do not know how this could be made more efficient as I execute parse only > once, and then I send bind+exec+bind+exec > without intermediate sync

Re: Synchronizing slots from primary to standby

2024-02-22 Thread Bertrand Drouvot
Hi, On Thu, Feb 22, 2024 at 04:01:34PM +0530, shveta malik wrote: > On Thu, Feb 22, 2024 at 3:44 PM Bertrand Drouvot > wrote: > > > > Hi, > > > > Thanks! > > > > Some random comments about v92_001 (Sorry if it has already been discussed > > up-thread): > > Thanks for the feedback. The patch is

Re: bug report: some issues about pg_15_stable(8fa4a1ac61189efffb8b851ee77e1bc87360c445)

2024-02-22 Thread Dean Rasheed
On Thu, 22 Feb 2024 at 03:46, zwj wrote: > > If I want to get the same results as Oracle, do I need to adjust the lock > behavior of the update and merge statements? > If I want to achieve the same results as Oracle, can I achieve exclusive > locking by adjusting update and merge? Do you

Re: About a recently-added message

2024-02-22 Thread Amit Kapila
On Thu, Feb 22, 2024 at 11:10 AM Kyotaro Horiguchi wrote: > > At Thu, 22 Feb 2024 10:51:07 +0530, Amit Kapila > wrote in > > > Do you think some additional tests for the rest of the messages are > > > worth the trouble? > > > > > > > We have discussed this during development and didn't find it

Re: speed up a logical replica setup

2024-02-22 Thread vignesh C
On Mon, 19 Feb 2024 at 11:15, Hayato Kuroda (Fujitsu) wrote: > > Dear hackers, > > > Since it may be useful, I will post top-up patch on Monday, if there are no > > updating. > > And here are top-up patches. Feel free to check and include. > > v22-0001: Same as v21-0001. > === rebased patches ===

Re: Synchronizing slots from primary to standby

2024-02-22 Thread shveta malik
On Thu, Feb 22, 2024 at 3:44 PM Bertrand Drouvot wrote: > > Hi, > > Thanks! > > Some random comments about v92_001 (Sorry if it has already been discussed > up-thread): Thanks for the feedback. The patch is pushed 15 minutes back. I will prepare a top-up patch for your comments. > 1 === > > +

Re: Test to dump and restore objects left behind by regression

2024-02-22 Thread Peter Eisentraut
On 22.02.24 11:00, Daniel Gustafsson wrote: On 22 Feb 2024, at 10:55, Ashutosh Bapat wrote: On Thu, Feb 22, 2024 at 3:03 PM Daniel Gustafsson wrote: Somebody looking for dump/restore tests wouldn't search src/bin/pg_upgrade, I think. Quite possibly not, but pg_upgrade is already today an

Re: Synchronizing slots from primary to standby

2024-02-22 Thread Bertrand Drouvot
Hi, On Thu, Feb 22, 2024 at 12:16:34PM +0530, shveta malik wrote: > On Thu, Feb 22, 2024 at 10:31 AM shveta malik wrote: > There was a recent commit 801792e to improve error messaging in > slotsync.c which resulted in conflict. Thus rebased the patch. There > is no new change in the patch

Re: Porting PostgresSQL libraries for QNX710

2024-02-22 Thread Daniel Gustafsson
> On 22 Feb 2024, at 06:12, Rajith Rao .B(App Software) > wrote: > Hope my query is clear for you and expecting a resolution for this. There is no official port of libpq to QNX, so the short answer is that you're on your own. QNX support was removed in 8.2, so maybe looking at the code

Porting PostgresSQL libraries for QNX710

2024-02-22 Thread Rajith Rao .B(App Software)
Hello, I have been working on ubuntu 22.04 LTS with postgres in my applications and need to deploy that application on QNX710. I have a requirement to port postgresSQL 12.18 to QNX 7.1 ,is it possible to build/port postgreSQL libraries for QNX7.1 Intel and Aarch64 architectures. Hope my

?????? bug report: some issues about pg_15_stable(8fa4a1ac61189efffb8b851ee77e1bc87360c445)

2024-02-22 Thread zwj
Thanks your reply. I understand what you mean and have tried to correct this patch. According to the previous use case, the result obtained is as follows: id | name | year | xmax | xmin | ctid +--+--+--+--+--- 1 | liuwei | 20 | 0 | 859 | (0,1) 2 | zhangbin |

Re: [PATCH] Avoid mixing custom and OpenSSL BIO functions

2024-02-22 Thread David Benjamin
Thanks for the very thorough comments! I've attached a new version of the patch. On Thu, Feb 15, 2024 at 4:17 PM Andres Freund wrote: > Hi, > > On 2024-02-11 13:19:00 -0500, David Benjamin wrote: > > I've attached a patch for the master branch to fix up the custom BIOs > used > > by PostgreSQL,

Re: Test to dump and restore objects left behind by regression

2024-02-22 Thread Daniel Gustafsson
> On 22 Feb 2024, at 10:55, Ashutosh Bapat wrote: > On Thu, Feb 22, 2024 at 3:03 PM Daniel Gustafsson wrote: > Somebody looking for dump/restore tests wouldn't search > src/bin/pg_upgrade, I think. Quite possibly not, but pg_upgrade is already today an important testsuite for testing pg_dump

Re: Test to dump and restore objects left behind by regression

2024-02-22 Thread Ashutosh Bapat
On Thu, Feb 22, 2024 at 3:03 PM Daniel Gustafsson wrote: > > > On 22 Feb 2024, at 10:16, Peter Eisentraut wrote: > > > We have somewhat relied on the pg_upgrade test to provide this testing, but > > we have recently discovered that the dumps in binary-upgrade mode are > > different enough to

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-02-22 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Thu, 22 Feb 2024 15:44:16 +0900, Michael Paquier wrote: > I was comparing what you have here, and what's been attached by Andres > at [1] and the top of the changes on my development branch at [2] >

Re: Test to dump and restore objects left behind by regression

2024-02-22 Thread Daniel Gustafsson
> On 22 Feb 2024, at 10:16, Peter Eisentraut wrote: > We have somewhat relied on the pg_upgrade test to provide this testing, but > we have recently discovered that the dumps in binary-upgrade mode are > different enough to not test the normal dumps well. > > Yes, this test is a bit

Re: When extended query protocol ends?

2024-02-22 Thread Vladimir Sitnikov
>When splitting a multi insert statement you're going to duplicate some work I do not understand why I am going to duplicate some work. I assume the database does its best to perform all the needed preparation when processing "parse" message, and it should perform only the minimum required work

Re: A problem about partitionwise join

2024-02-22 Thread Ashutosh Bapat
On Wed, Feb 21, 2024 at 4:55 PM Richard Guo wrote: > > > On Tue, Aug 2, 2022 at 4:24 AM Jacob Champion wrote: >> >> Once you think you've built up some community support and the patchset >> is ready for review, you (or any interested party) can resurrect the >> patch entry by visiting >> >>

Re: Test to dump and restore objects left behind by regression

2024-02-22 Thread Peter Eisentraut
On 22.02.24 02:01, Michael Paquier wrote: On Wed, Feb 21, 2024 at 12:18:45PM +0530, Ashutosh Bapat wrote: Even with 1 and 2 the test is useful to detect dump/restore anomalies. I think we should improve 3, but I don't have a good and simpler solution. I didn't find any way to compare two given

Re: When extended query protocol ends?

2024-02-22 Thread Jelte Fennema-Nio
On Wed, 21 Feb 2024 at 17:07, Vladimir Sitnikov wrote: > From many measurements we know that insert into table(id, name) > values(?,?),(?,?),(?,?) is much more efficient than > sending individual bind-exec-bind-exec-bind-exec-sync messages like "insert > into table(id, name) values(?,?)" > For

Re: Test to dump and restore objects left behind by regression

2024-02-22 Thread Ashutosh Bapat
On Thu, Feb 22, 2024 at 6:32 AM Michael Paquier wrote: > > On Wed, Feb 21, 2024 at 12:18:45PM +0530, Ashutosh Bapat wrote: > > Even with 1 and 2 the test is useful to detect dump/restore anomalies. > > I think we should improve 3, but I don't have a good and simpler > > solution. I didn't find

Re: Removing unneeded self joins

2024-02-22 Thread Andrei Lepikhov
On 21/2/2024 14:26, Richard Guo wrote: I think the right fix for these issues is to introduce a new element 'sublevels_up' in ReplaceVarnoContext, and enhance replace_varno_walker to 1) recurse into subselects with sublevels_up increased, and 2) perform the replacement only when varlevelsup is

Re: Reduce useless changes before reassembly during logical replication

2024-02-22 Thread Andy Fan
Hi, > > This is all true but note that in successful cases (where the table is > published) all the work done by FilterByTable(accessing caches, > transaction-related stuff) can add noticeable overhead as anyway we do > that later in pgoutput_change(). I think I gave the same comment > earlier

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-02-22 Thread Bertrand Drouvot
Hi, On Wed, Feb 21, 2024 at 08:10:00PM +0530, Bharath Rupireddy wrote: > On Wed, Feb 21, 2024 at 5:55 PM Bertrand Drouvot > wrote: > > My initial thought was to put "conflict" value in this new field in case of > > conflict (not to mention the conflict reason in it). With the current > >

Re: Injection points: some tools to wait and wake

2024-02-22 Thread Bertrand Drouvot
Hi, On Thu, Feb 22, 2024 at 12:02:01PM +0900, Michael Paquier wrote: > On Wed, Feb 21, 2024 at 11:50:21AM +, Bertrand Drouvot wrote: > > A few comments: > > > > 1 === > > I think "up" is missing at several places in the patch where "wake" is used. > > I could be wrong as non native english