Re: speed up a logical replica setup

On Wed, 22 May 2024 at 16:50, Amit Kapila wrote: > > On Mon, May 20, 2024 at 4:30 PM Shlok Kyal wrote: > > > > > > I was trying to test this utility when 'sync_replication_slots' is on > > > and it gets in an ERROR loop [1] and never finishes. Please fi

Re: State of pg_createsubscriber

will be replicated to N2 instead of N1. And once N1 is up again, subscription on N1 will not be able to connect to publication on N3 as it is already connected to N2. This can lead to data inconsistency. This error did not happen before running pg_createsubscriber on standby node N2, because there is no 'logical replication launcher' process on standby node. Thanks and Regards, Shlok Kyal

Re: speed up a logical replica setup

J22UEfrqx222h5j9DQ7nxGrTbAa_BC%2B%3DmQXdXs-RCsew%40mail.gmail.com Thanks and Regards, Shlok Kyal # cleanup rm -rf ../primary ../standby primary.log standby.log # setup primary node ./initdb -D ../primary cat << EOF >> ../primary/postgresql.conf wal_level = logical EOF ./pg_ctl -D ../prim

Re: Pgoutput not capturing the generated columns

at generated columns are decoded even if they are not > in > the column list? If so, why? I think such columns should not be sent. Fixed Thanks and Regards, Shlok Kyal v2-0001-Support-generated-column-capturing-generated-colu.patch Description: Binary data

Re: speed up a logical replica setup

e code, so I have added it to the documentation in the warning > section. Thoughts? > I am not changing the version as I have not made any changes in > v30-0001 and v30-0002. > > [1]: > https://www.postgresql.org/message-id/cahv8rj+5mzk9jt+7ecogjzf

Re: speed up a logical replica setup

and v30-0002. [1]: https://www.postgresql.org/message-id/cahv8rj+5mzk9jt+7ecogjzfm5czvdccd5jo1_rcx0bteypb...@mail.gmail.com Thanks and regards, Shlok Kyal v30-0001-pg_createsubscriber-creates-a-new-logical-replic.patch Description: Binary data v30-0002-Stop-the-target-server-earlier.pa

Re: Add publisher and subscriber to glossary documentation.

should be on the preceding sentence too. > > > Anyway, hopefully these examples show “node” and “database” are mixed and > perhaps others agree using one consistently might help the goals of the docs. For me the existing content looks good, I felt let's keep it as it is unless others feel differently. Thanks and regards, Shlok Kyal

Re: speed up a logical replica setup

:nP:rS:t:v", + long_options, _index)) != -1) + { Here 'p', 's' and 'U' options are missing so we are getting the error. Also, I think the 'S' option should be removed from here. I also see that specifying long options like --subscriber-port, --subscriber-username, --socket-directory works fine. Thoughts? Thanks and regards, Shlok Kyal

Re: Add publisher and subscriber to glossary documentation.

ed > + for logical > replication. > + > + > + > + > > (same comment as above) > > I felt the word "node" here should link to the glossary term "Node", > instead of directly to the term "Instance". I have addressed the comments and have attached the updated version. Thanks and Regards, Shlok Kyal v4-0001-Add-publisher-and-subscriber-to-glossary-document.patch Description: Binary data

Re: Add publisher and subscriber to glossary documentation.

more information, see Section 30.1 > > Publication node > A node where a publication is defined for logical replication. > > Subscriber > See "Subscription node" > > Subscription > A subscription receives the changes of one or more tables from the > publications it subscribes to. For more information, see Section 30.2 > > Subscription node > A node where a subscription is defined for logical replication. I have addressed the comments and added an updated patch. Thanks and Regards, Shlok Kyal v3-0001-Add-publisher-and-subscriber-to-glossary-document.patch Description: Binary data

Re: speed up a logical replica setup

Hi, On Tue, 20 Feb 2024 at 06:59, Euler Taveira wrote: > > On Mon, Feb 19, 2024, at 7:22 AM, Shlok Kyal wrote: > > I have reviewed the v21 patch. And found an issue. > > Initially I started the standby server with a new postgresql.conf file > (not the default postgresql

Re: speed up a logical replica setup

atesubscriber also uses this approach. 2) read GUCs via SHOW command and restore them when server restarts I would prefer the 1st solution. Thanks and Regards, Shlok Kyal sudo pkill -9 postgres rm -rf ../primary ../standby ../new_path rm -rf primary.log standby.log ./initdb -D ../primary cat

Re: Add publisher and subscriber to glossary documentation.

e we > provide an overview of "logical replication". Maybe that's enough, but > we should consider whether we want a separate definition of logical > replication (I'm leaning towards not having one, but it's worth asking.) Modified. Added the term "logical replication" in the definitions. Used reference to "replication". Thanks and Regards, Shlok Kyal v2-0001-Add-publisher-and-subscriber-to-glossary-document.patch Description: Binary data

Add publisher and subscriber to glossary documentation.

Hi, There are several places where publisher and subscriber terms are used across the documentation. But the publisher and subscriber were missing in the documentation. I felt this should be added in the glossary. I have created a patch for the same. Thanks and Regards Shlok Kyal v1-0001-Add

Re: speed up a logical replica setup

A1C29F56F2%40TY3PR01MB9889.jpnprd01.prod.outlook.com Thanks and Regards Shlok Kyal On Fri, 12 Jan 2024 at 03:46, Euler Taveira wrote: > > On Thu, Jan 11, 2024, at 9:18 AM, Hayato Kuroda (Fujitsu) wrote: > > I have been concerned that the patch has not been tested by cfbot due to the > appl

Re: Extend pgbench partitioning to pgbench_history

Sorry, I did not intend to send this message for this email. I by mistake sent this mail. Please ignore this mail On Wed, 10 Jan 2024 at 15:27, Shlok Kyal wrote: > > Hi, > > There are some test failures reported by Cfbot at [1]: > > [09:15:01.794] 192/276 postgresql:p

Re: Extend pgbench partitioning to pgbench_history

r.pl line 1257. [09:15:01.794] # Looks like you failed 3 tests of 439. [09:15:01.794] [09:15:01.794] (test program exited with status code 3) [1] - https://cirrus-ci.com/task/5139049757802496 Thanks and regards Shlok Kyal

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

Hi, This patch is not applying on the HEAD. Please rebase and share the updated patch. Thanks and Regards Shlok Kyal On Wed, 10 Jan 2024 at 14:55, Peter Smith wrote: > > Oops - now with attachments > > On Mon, Aug 21, 2023 at 5:56 PM Peter Smith wrote: >> >> Hi M

Re: speed up a logical replica setup

On Fri, 5 Jan 2024 at 12:19, Shlok Kyal wrote: > > On Thu, 4 Jan 2024 at 16:46, Amit Kapila wrote: > > > > On Thu, Jan 4, 2024 at 12:22 PM Shlok Kyal wrote: > > > > > > Hi, > > > I was testing the patch with following test cases: > &g

Re: speed up a logical replica setup

On Thu, 4 Jan 2024 at 16:46, Amit Kapila wrote: > > On Thu, Jan 4, 2024 at 12:22 PM Shlok Kyal wrote: > > > > Hi, > > I was testing the patch with following test cases: > > > > Test 1 : > > - Create a 'primary' node > > - Setup physical

Re: speed up a logical replica setup

LOG: invalid record length at 0/3000150: expected at least 24, got 0 I was not clear about how to use pg_basebackup in this case, can you let me know if any changes need to be made for test2 and test3. Thanks and regards Shlok Kyal

Re: Intermittent failure with t/003_logical_slots.pl test on windows

for the same as well. Thanks and Regards Shlok Kyal On Tue, 26 Dec 2023 at 17:39, Shlok Kyal wrote: > > Hi, > The same intermittent failure is reproducible on my system. > For the intermittent issues I found that many issues are due to errors > where commands like 'psql -V' are no

Re: Intermittent failure with t/003_logical_slots.pl test on windows

facing some more intermittent issues. Will analyse and share it as well. Thanks and regards Shlok Kyal On Tue, 7 Nov 2023 at 11:05, Kyotaro Horiguchi wrote: > > At Mon, 6 Nov 2023 19:42:21 +0530, Nisha Moond > wrote in > > > Appending '2>&1 test: > > > The comm

Re: speed up a logical replica setup

built the code using Meson, on installing postgres, pg_subscriber binary was created. Is this behaviour intentional? [1] https://github.com/postgres/postgres/commit/1301c80b2167feb658a738fa4ceb1c23d0991e23 Thanks and Regards, Shlok Kyal

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

and all the tests passed on each branch. I also reviewed the patch and it looks good to me. Thanks and Regards, Shlok Kyal

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

fixed the comment in previous approach and attached here as 'v2-0001-Deadlock-when-apply-worker-tablesync-worker-and-c.patch' Thanks and Regards Shlok Kyal From 11072d138d900227b963b54d1a3626cf256db721 Mon Sep 17 00:00:00 2001 From: Shlok Kyal Date: Fri, 24 Nov 2023 16:37:14 +0530 Subject: [PATCH v

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

On Tue, 5 Dec 2023 at 17:18, Tomas Vondra wrote: > > On 12/5/23 08:14, Shlok Kyal wrote: > > Hi, > > > >> As for the test results, I very much doubt the differences are not > >> caused simply by random timing variations, or something like that. And I >

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

The machine has Total Memory of 755.536 GB, 120 CPUs and RHEL 7 Operating System Also find the detailed info of the performance machine attached. Thanks and Regards, Shlok Kyal MemTotal: 755.536 GB MemFree: 748.281 GB MemAvailable: 748.581 GB Buffers: 0.002 GB Cached: 1.366 GB SwapCached: 0.0

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

and Sub (sec)| 1000 tables in pub and Sub (sec) Without patch Release | 115.871 | 6.656 | 81.157 With patch Release| 115.922 | 6.7305 | 81.1525 thoughts? Thanks and Regards

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

eated the v1 patch to resolve the issue. Have tested the patch on HEAD to PG12. The same patch applies to all the versions. The changes are similar to the one posted in the thread https://www.postgresql.org/message-id/1412708.1674417574%40sss.pgh.pa.us Thanks and Regards, Shlok Kyal v1-0001-Deadlock

Re: undetected deadlock in ALTER SUBSCRIPTION ... REFRESH PUBLICATION

Hi, I tried to reproduce the issue and was able to reproduce it with scripts shared by Tomas. I tried testing it from PG17 to PG 11. This issue is reproducible for each version. Next I would try to test with the patch in the thread shared by Amit. Thanks, Shlok Kumar Kyal

Re: pg_upgrade and logical replication

On Wed, 22 Nov 2023 at 06:48, Peter Smith wrote: > == > doc/src/sgml/ref/pgupgrade.sgml > > 1. > + > + Create all the new tables that were created in the publication during > + upgrade and refresh the publication by executing > + ALTER > SUBSCRIPTION ... REFRESH

Failure during Building Postgres in Windows with Meson

Hi, I am trying to build postgres with meson on Windows. And I am stuck in the process. Steps I followed: 1. I clone postgres repo 2.Installed meson and ninja pip install meson ninja 3. Then running following command: meson setup build --buildtype debug 4. Then I ran cd build ninja Got

Re: [PoC] Implementation of distinct in Window Aggregates: take two

Hi, I went through the Cfbot, and some of the test cases are failing for this patch. It seems like some tests are crashing: https://api.cirrus-ci.com/v1/artifact/task/6291153444667392/crashlog/crashlog-postgres.exe_03b0_2023-11-07_10-41-39-624.txt [10:46:56.546] Summary of Failures:

Re: [ psql - review request ] review request for \d+ tablename, \d+ indexname indenting

Hi, On Mon, 6 Nov 2023 at 13:47, 쿼리트릭스 wrote: > > The error was corrected and a new diff file was created. > The diff file was created based on 16 RC1. > We confirmed that 5 places where errors occurred when performing make check > were changed to ok. > I went through Cfbot and still see that

Re: POC: Extension for adding distributed tracing - pg_tracing

Hi, On Thu, 12 Oct 2023 at 14:32, Anthonin Bonnefoy wrote: > > Hi! > > I've made a new batch of changes and improvements. > New features: > - Triggers are now correctly supported. They were not correctly > attached to the ExecutorFinish Span before. > - Additional configuration: exporting

Re: [PoC] Federated Authn/z with OAUTHBEARER

Hi, On Fri, 3 Nov 2023 at 17:14, Jacob Champion wrote: > > v12 implements a first draft of a client hook, so applications can > replace either the device prompt or the entire OAuth flow. (Andrey and > Mahendrakar: hopefully this is close to what you need.) It also cleans > up some of the JSON

Re: Force the old transactions logs cleanup even if checkpoint is skipped

Hi, I went through the Cfbot and saw that some test are failing for it (link: https://cirrus-ci.com/task/4631357628874752): test: postgresql:recovery / recovery/019_replslot_limit # test failed --- stderr --- #

Re: [HACKERS] Allow INSTEAD OF DELETE triggers to modify the tuple for RETURNING

Hi, On Thu, 19 Oct 2023 at 16:35, Marko Tiikkaja wrote: > > Hi, > > Thank you for the feedback. > > Apparently it took me six years, but I've attached the latest version > of the patch which I believe addresses all issues. I'll add it to the > open commitfest. > > > .m I went through the CFbot

Re: psql not responding to SIGINT upon db reconnection

Hi, > That sounds like a much better solution. Attached you will find a v4 > that implements your suggestion. Please let me know if there is > something that I missed. I can confirm that the patch works. > > $ ./build/src/bin/psql/psql -h pg.neon.tech > NOTICE: Welcome to Neon! >

Re: [patch] pg_basebackup: mention that spread checkpoints are the default in --help

Hi, On Thu, 26 Oct 2023 at 13:58, Michael Banck wrote: > > Hi, > > On Wed, Oct 25, 2023 at 04:36:41PM +0200, Peter Eisentraut wrote: > > On 19.10.23 11:39, Michael Banck wrote: > > > Hi, > > > > > > I believed that spread (not fast) checkpoints are the default in > > > pg_basebackup, but noticed

Re: [PoC] pg_upgrade: allow to upgrade publisher node

I tested a test scenario: I started a new publisher with 'max_replication_slots' parameter set to '1' and created a streaming replication with the new publisher as primary node. Then I did a pg_upgrade from old publisher to new publisher. The upgrade failed with following error: Restoring logical

Re: [PoC] pg_upgrade: allow to upgrade publisher node

> Few comments: > 1) We will be able to override the value of max_slot_wal_keep_size by > using --new-options like '--new-options "-c > max_slot_wal_keep_size=val"': > + /* > +* Use max_slot_wal_keep_size as -1 to prevent the WAL removal by the > +* checkpointer process. If

Re: pg_ctl start may return 0 even if the postmaster has been already started on Windows

On Fri, 6 Oct 2023 at 11:38, Hayato Kuroda (Fujitsu) wrote: > > Dear Horiguchi-san, > > Thank you for making a patch! They can pass ci. > I'm still not sure what should be, but I can respond a part. > > > Another issue is.. that I haven't been able to cause the false > > positive of pg_ctl

Re: Invalidate the subscription worker in cases where a user loses their superuser status

On Wed, 4 Oct 2023 at 16:56, Peter Smith wrote: > > On Tue, Oct 3, 2023 at 5:42 PM vignesh C wrote: > > > > Thanks for the comments, the attached v6 version patch has the changes > > for the same. > > > > v6 LGTM. > I have verified the patch and it is working fine for me.