Re: Conflict detection for update_deleted in logical replication

2025-08-12 Thread shveta malik
025 at 2:21 PM Amit Kapila > > > wrote: > > > > > > > > On Tue, Aug 12, 2025 at 2:06 PM shveta malik > > > wrote: > > > > > > > > > > 2) > > > > > postgres=# create subscription sub2 connection 'dbname

Re: Conflict detection for update_deleted in logical replication

2025-08-12 Thread shveta malik
On Mon, Aug 11, 2025 at 2:40 PM Zhijie Hou (Fujitsu) wrote: > > I agree. So, following the above points and some off-list discussions, I have > revised the option to be a subscription option in the V60 version. > Thank You for the patches. Tried to test the new sub-level parameter, have few comme

Re: Excessive LOG messages from replication slot sync worker

2025-08-11 Thread shveta malik
On Tue, Aug 12, 2025 at 8:52 AM Fujii Masao wrote: > > Hi, > > While testing the replication slot synchronization feature, I set up > three instances on the same machine: > > - Physical replication primary (also Logical replication publisher) > - Physical replication standby > - Logical replicatio

Re: Proposal: Conflict log history table for Logical Replication

2025-08-08 Thread shveta malik
On Fri, Aug 8, 2025 at 10:01 AM Dilip Kumar wrote: > > On Fri, Aug 8, 2025 at 8:58 AM shveta malik wrote: > > > > On Thu, Aug 7, 2025 at 3:08 PM Dilip Kumar wrote: > > > > > > So logically for PostgreSQL its an > > > user table but yeah

Re: Conflict detection for update_deleted in logical replication

2025-08-07 Thread shveta malik
On Thu, Aug 7, 2025 at 10:10 AM Zhijie Hou (Fujitsu) wrote: > > On Tuesday, August 5, 2025 10:09 AM Zhijie Hou (Fujitsu) > wrote: > > Here is V57 patch set which addressed most of comments. > > > > In this version, I also fixed a bug that the apply worker continued to find > > dead > > tuples e

Re: Issue with logical replication slot during switchover

2025-08-07 Thread shveta malik
On Thu, Aug 7, 2025 at 6:50 PM Fabrice Chapuis wrote: > > Hi, > > An issue occurred during the initial switchover using PostgreSQL version > 17.5. The setup consists of a cluster with two nodes, managed by Patroni > version 4.0.5. > Logical replication is configured on the same instance, and the

Re: Proposal: Conflict log history table for Logical Replication

2025-08-07 Thread shveta malik
On Thu, Aug 7, 2025 at 3:08 PM Dilip Kumar wrote: > > On Thu, Aug 7, 2025 at 1:43 PM shveta malik wrote: > > > > On Thu, Aug 7, 2025 at 12:25 PM shveta malik wrote: > > Thanks Shveta for your opinion on the design. > > > > On Tue, Aug 5

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-08-07 Thread shveta malik
On Fri, Aug 8, 2025 at 3:30 AM Masahiko Sawada wrote: > > On Tue, Aug 5, 2025 at 11:23 PM shveta malik wrote: > > > > Please find a few comments on v6: > > > > 1) > > +/* > > + * Initialize logical decoding status on shmem at server startup. This > &

Re: Conflict detection for update_deleted in logical replication

2025-08-07 Thread shveta malik
On Thu, Aug 7, 2025 at 6:05 PM Dilip Kumar wrote: > > On Mon, Aug 4, 2025 at 3:11 PM Amit Kapila wrote: > > > > On Mon, Aug 4, 2025 at 11:46 AM shveta malik wrote: > > > > > > 7) > > > Shall we rename 'max_conflict_retention_duration'

Re: Proposal: Conflict log history table for Logical Replication

2025-08-07 Thread shveta malik
On Thu, Aug 7, 2025 at 12:25 PM shveta malik wrote: > > On Tue, Aug 5, 2025 at 5:54 PM Dilip Kumar wrote: > > > > Currently we log conflicts to the server's log file and updates, this > > approach has limitations, 1) Difficult to query and analyze, parsing > >

Re: Proposal: Conflict log history table for Logical Replication

2025-08-06 Thread shveta malik
On Tue, Aug 5, 2025 at 5:54 PM Dilip Kumar wrote: > > Currently we log conflicts to the server's log file and updates, this > approach has limitations, 1) Difficult to query and analyze, parsing > plain text log files for conflict details is inefficient. 2) Lack of > structured data, key conflict

Re: Logical Replication of sequences

2025-08-06 Thread shveta malik
On Wed, Aug 6, 2025 at 4:29 PM shveta malik wrote: > > On Wed, Aug 6, 2025 at 2:28 PM vignesh C wrote: > > > > The attached v20250806 version patch has the changes for the same. > > > > Thank You for the patches. Please find a few comments: > > 1) >

Re: Logical Replication of sequences

2025-08-06 Thread shveta malik
On Wed, Aug 6, 2025 at 2:28 PM vignesh C wrote: > > The attached v20250806 version patch has the changes for the same. > Thank You for the patches. Please find a few comments: 1) * If 'resync_all_sequences' is false: * Add or remove tables and sequences that have been added to or removed

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-08-05 Thread shveta malik
Please find a few comments on v6: 1) +/* + * Initialize logical decoding status on shmem at server startup. This + * must be called ONCE during postmaster or standalone-backend startup, + * before initializing replication slots. + */ +void +StartupLogicalDecodingStatus(bool last_status) The comme

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-08-05 Thread shveta malik
On Wed, Aug 6, 2025 at 6:18 AM Masahiko Sawada wrote: > > > I've attached the updated version patch. > Thank You for the patch. The patch does not apply to the latest head due to conflict with slot-sync fix (commit-Id: 4614d53d). thanks Shveta

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-08-05 Thread shveta malik
On Wed, Aug 6, 2025 at 7:35 AM Ajin Cherian wrote: > > On Tue, Aug 5, 2025 at 4:22 PM Amit Kapila wrote: > > > > On Tue, Aug 5, 2025 at 9:28 AM shveta malik wrote: > > > > > > On Mon, Aug 4, 2025 at 3:41 PM Amit Kapila > > > wrote: > > >

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-08-05 Thread shveta malik
On Tue, Aug 5, 2025 at 5:14 AM Masahiko Sawada wrote: > > On Mon, Aug 4, 2025 at 3:38 AM shveta malik wrote: > > > > On Sat, Aug 2, 2025 at 4:53 AM Masahiko Sawada > > wrote: > > > > > > On Thu, Jul 31, 2025 at 5:00 AM Hayato Kuroda (Fujitsu) >

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-08-04 Thread shveta malik
On Mon, Aug 4, 2025 at 3:41 PM Amit Kapila wrote: > > On Mon, Aug 4, 2025 at 12:19 PM shveta malik wrote: > > > > On Mon, Aug 4, 2025 at 11:31 AM Amit Kapila wrote: > > > > > > On Fri, Aug 1, 2025 at 2:50 PM shveta malik > > > wrote: > > >

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-08-04 Thread shveta malik
On Sat, Aug 2, 2025 at 4:53 AM Masahiko Sawada wrote: > > On Thu, Jul 31, 2025 at 5:00 AM Hayato Kuroda (Fujitsu) > wrote: > > > > Dear Sawada-san, > > > > > I thought we could fix this issue by checking the number of in-use > > > logical slots while holding ReplicationSlotControlLock and > > > L

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-08-03 Thread shveta malik
On Mon, Aug 4, 2025 at 11:31 AM Amit Kapila wrote: > > On Fri, Aug 1, 2025 at 2:50 PM shveta malik wrote: > > > > 5) > > I tried a test where there were 4 slots on the publisher, where one > > was getting used while the others were not. Initiated > > pg_sync_

Re: Conflict detection for update_deleted in logical replication

2025-08-03 Thread shveta malik
On Fri, Aug 1, 2025 at 9:16 PM Zhijie Hou (Fujitsu) wrote: > > > Thanks for confirming. Here is V56 patch set which addressed all the > comments including the comments from Amit[1] and Shveta[2]. > > I have merged V55-0002 into 0001 and updated the list of author > and reviewers based on my knowle

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-08-01 Thread shveta malik
On Fri, Aug 1, 2025 at 12:02 PM shveta malik wrote: > > On Thu, Jul 31, 2025 at 3:11 PM Ajin Cherian wrote: > > > > > > Patch v3 attached. > > > > Thanks for the patch. I tested it, please find a few comments: > > > 1) > it hit

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-07-31 Thread shveta malik
On Thu, Jul 31, 2025 at 3:11 PM Ajin Cherian wrote: > > > Patch v3 attached. > Thanks for the patch. I tested it, please find a few comments: 1) it hits an assert (slotsync_reread_config()-->Assert(sync_replication_slots)) when API is trying to sync and is in wait loop while in another session,

Re: Conflict detection for update_deleted in logical replication

2025-07-31 Thread shveta malik
On Thu, Jul 31, 2025 at 3:49 PM Zhijie Hou (Fujitsu) wrote: > > On Thursday, July 31, 2025 5:26 PM shveta malik > wrote: > > > > On Tue, Jul 29, 2025 at 10:51 AM Zhijie Hou (Fujitsu) > > > > wrote: > > > > > > > > > This is th

Re: Conflict detection for update_deleted in logical replication

2025-07-31 Thread shveta malik
On Tue, Jul 29, 2025 at 10:51 AM Zhijie Hou (Fujitsu) wrote: > > > This is the V54 patch set, with only patch 0001 updated to address the latest > comments. > Thanks for the patch. While performing tests on the latest patch, I found an assert in tablesync worker in FindDeletedTupleInLocalRel (se

Re: Logical Replication of sequences

2025-07-30 Thread shveta malik
On Wed, Jul 30, 2025 at 11:16 AM shveta malik wrote: > > On Mon, Jul 28, 2025 at 3:37 PM vignesh C wrote: > > > Thanks for the comments, the attached v20250728 version patch has the > > changes for the same. > > > Thanks for the patches, please find a few comments:

Re: Logical Replication of sequences

2025-07-29 Thread shveta malik
On Mon, Jul 28, 2025 at 3:37 PM vignesh C wrote: > Thanks for the comments, the attached v20250728 version patch has the > changes for the same. > Thanks for the patches, please find a few comments: 1) WARNING: WITH clause parameters do not affect sequence synchronization a) How about: WITH cl

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-28 Thread shveta malik
On Fri, Jul 25, 2025 at 11:45 AM Masahiko Sawada wrote: > > Thank you for testing the patch! > > I've reworked the locking part in the patch. The attached v4 patch > should address all review comments including your previous > comments[1]. > Thank You for the patch. I have not reviewed fully, but

Re: Conflict detection for update_deleted in logical replication

2025-07-28 Thread shveta malik
On Mon, Jul 28, 2025 at 4:38 PM shveta malik wrote: > > On Fri, Jul 25, 2025 at 4:38 PM Zhijie Hou (Fujitsu) > wrote: > > > > > > The V53-0001 also includes Shveta's comments in [1]. > > > > Thanks, I have not yet completed the review, bu

Re: Conflict detection for update_deleted in logical replication

2025-07-28 Thread shveta malik
On Fri, Jul 25, 2025 at 4:38 PM Zhijie Hou (Fujitsu) wrote: > > > The V53-0001 also includes Shveta's comments in [1]. > Thanks, I have not yet completed the review, but please find a few comments on 001: 1) IsIndexUsableForFindingDeletedTuple() We first have: + /* + * A frozen transaction ID in

Re: Conflict detection for update_deleted in logical replication

2025-07-25 Thread shveta malik
On Fri, Jul 25, 2025 at 2:31 PM Amit Kapila wrote: > > On Fri, Jul 25, 2025 at 12:37 PM shveta malik wrote: > > > > On Thu, Jul 24, 2025 at 9:12 AM shveta malik wrote: > > > > > > > > > 2) > > >

Re: Logical replication launcher did not automatically restart when got SIGKILL

2025-07-25 Thread shveta malik
On Fri, Jul 25, 2025 at 7:17 AM Fujii Masao wrote: > > On Thu, Jul 24, 2025 at 6:46 PM shveta malik wrote: > > Sounds reasonable. > > Thinking out loud, when cleaning up after a backend or background > > worker crash, process_pm_child_exit() is invoked, which sub

Re: Conflict detection for update_deleted in logical replication

2025-07-25 Thread shveta malik
On Thu, Jul 24, 2025 at 9:12 AM shveta malik wrote: > > > 2) > + if (MySubscription->retaindeadtuples && > + FindMostRecentlyDeletedTupleInfo(localrel, remoteslot, > + > &conflicttuple.xmin, > + &g

Re: Logical replication launcher did not automatically restart when got SIGKILL

2025-07-24 Thread shveta malik
On Thu, Jul 24, 2025 at 2:39 PM Fujii Masao wrote: > > On Thu, Jul 17, 2025 at 6:58 PM shveta malik wrote: > > > > On Wed, Jul 16, 2025 at 8:51 AM cca5507 wrote: > > > > > > Hi, > > > > > > The v1-0002 in [1] will call ReportBa

Re: Conflict detection for update_deleted in logical replication

2025-07-23 Thread shveta malik
On Wed, Jul 23, 2025 at 12:53 PM Zhijie Hou (Fujitsu) wrote: > > On Wednesday, July 23, 2025 12:08 PM Amit Kapila > wrote: > > > > On Wed, Jul 23, 2025 at 3:51 AM Masahiko Sawada > > wrote: > > > > > > I've reviewed the 0001 patch and it looks good to me. > > > > > > > Thanks, I have pushed the

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-22 Thread shveta malik
On Tue, Jul 22, 2025 at 5:03 AM Masahiko Sawada wrote: > > Yes, I agree. The main patch focuses on the part where we > automatically change the effective WAL level upon the logical slot > creation and deletion (and potentially remove 'logical' from > wal_level), and other things are implemented as

Re: Skipping schema changes in publication

2025-07-22 Thread shveta malik
I further tested inherited tables flow as well wrt ONLY and EXCEPT, it works well. But while reading docs for the saem, I have few concerns. 1) While explaining ONLY for EXCEPT, create-publication doc says this + This does not apply to a partitioned table, however. The partitions of +

Re: Skipping schema changes in publication

2025-07-22 Thread shveta malik
Shlok, I was trying to validate the interaction of 'publish_via_partition_root' with 'EXCEPT". Found some unexpected behaviour, can you please review: Pub: - CREATE TABLE tab_root (range_col int,i int,j int) PARTITION BY RANGE (range_col); CREATE TABLE tab_part_1 PARTITION OF tab_root FOR

Re: Skipping schema changes in publication

2025-07-22 Thread shveta malik
On Sat, Jul 19, 2025 at 4:17 PM Shlok Kyal wrote: > > On Mon, 30 Jun 2025 at 16:25, shveta malik wrote: > > > > Few more comments on 002: > > > > 5) > > +GetAllTablesPublicationRelations(Oid pubid, bool pubviaroot) > > { > > &

Re: Skipping schema changes in publication

2025-07-21 Thread shveta malik
On Sat, Jul 19, 2025 at 4:17 PM Shlok Kyal wrote: > > On Mon, 30 Jun 2025 at 16:25, shveta malik wrote: > > > > Few more comments on 002: > > > > 5) > > +GetAllTablesPublicationRelations(Oid pubid, bool pubviaroot) > > { > > &

Re: Logical Replication of sequences

2025-07-21 Thread shveta malik
On Mon, Jul 21, 2025 at 2:55 PM Amit Kapila wrote: > > On Mon, Jul 21, 2025 at 2:36 PM vignesh C wrote: > > > > On Mon, 21 Jul 2025 at 11:15, Dilip Kumar wrote: > > > > > > > > 3. Some of the syntaxes works for sequence which doesn't make sense to > > > me, as listed below, I think there are mor

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-21 Thread shveta malik
On Mon, Jul 21, 2025 at 12:23 PM Amit Kapila wrote: > > I am slightly hesitant to introduce multiple ways to enable logical > decoding/replication unless that is the only path as giving multiple > options to achieve the same thing can confuse users as to which one is > preferable and pros/cons of

Re: Logical Replication of sequences

2025-07-20 Thread shveta malik
On Mon, Jul 21, 2025 at 11:15 AM Dilip Kumar wrote: > > 3. Some of the syntaxes works for sequence which doesn't make sense to > me, as listed below, I think there are more > > postgres[154731]=# CREATE PUBLICATION insert_only FOR ALL SEQUENCES > WITH (publish = 'insert'); > CREATE PUBLICATION > >

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-20 Thread shveta malik
On Mon, Jul 21, 2025 at 10:48 AM shveta malik wrote: > > On Fri, Jul 18, 2025 at 3:03 PM Amit Kapila wrote: > > > > On Fri, Jul 18, 2025 at 2:27 PM shveta malik wrote: > > > > > > On Thu, Jul 17, 2025 at 3:06 PM Amit Kapila > > > wrote: > >

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-20 Thread shveta malik
On Fri, Jul 18, 2025 at 3:03 PM Amit Kapila wrote: > > On Fri, Jul 18, 2025 at 2:27 PM shveta malik wrote: > > > > On Thu, Jul 17, 2025 at 3:06 PM Amit Kapila wrote: > > > > > > On Wed, Jun 18, 2025 at 3:23 PM shveta malik > > > wrote: > > &

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-07-20 Thread shveta malik
On Sat, Jul 19, 2025 at 5:10 PM Amit Kapila wrote: > > On Fri, Jul 18, 2025 at 11:31 AM Dilip Kumar wrote: > > > > On Fri, Jul 18, 2025 at 11:25 AM shveta malik > > wrote: > > > > > > Okay. I see your point. Yes, it was non-blocking earlier but

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-18 Thread shveta malik
On Thu, Jul 17, 2025 at 3:06 PM Amit Kapila wrote: > > On Wed, Jun 18, 2025 at 3:23 PM shveta malik wrote: > > > > On Wed, Jun 18, 2025 at 2:39 PM Bertrand Drouvot > > wrote: > > > > > > Hi, > > > > > > On Tue, Jun 10, 2025 at 02:

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-07-17 Thread shveta malik
On Fri, Jul 18, 2025 at 10:52 AM Dilip Kumar wrote: > > On Fri, Jul 18, 2025 at 10:45 AM shveta malik wrote: > > > > On Fri, Jul 18, 2025 at 10:14 AM Dilip Kumar wrote: > > > > > > On Thu, Jul 17, 2025 at 9:34 AM shveta malik > > > wrote: > &g

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-07-17 Thread shveta malik
On Fri, Jul 18, 2025 at 10:14 AM Dilip Kumar wrote: > > On Thu, Jul 17, 2025 at 9:34 AM shveta malik wrote: > > > > On Wed, Jul 16, 2025 at 3:47 PM Ajin Cherian wrote: > > > > > > > I am not able to apply the patch to the latest head or even to a week >

Re: Conflict detection for update_deleted in logical replication

2025-07-17 Thread shveta malik
On Thu, Jul 17, 2025 at 9:56 AM Dilip Kumar wrote: > > On Fri, Jul 11, 2025 at 4:28 PM Amit Kapila wrote: > > > > On Thu, Jul 10, 2025 at 6:46 PM Masahiko Sawada > > wrote: > > > > > > On Wed, Jul 9, 2025 at 9:09 PM Amit Kapila > > > wrote: > > > > > > > > > > > > I think that even with retai

Re: Logical replication launcher did not automatically restart when got SIGKILL

2025-07-17 Thread shveta malik
On Wed, Jul 16, 2025 at 8:51 AM cca5507 wrote: > > Hi, > > The v1-0002 in [1] will call ReportBackgroundWorkerExit() which will send > SIGUSR1 to 'bgw_notify_pid', but it may already exit in HandleChildCrash(), > is this ok? > Shall ReportBackgroundWorkerExit() be skipped for 'crashed' backgrou

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-07-16 Thread shveta malik
On Wed, Jul 16, 2025 at 3:47 PM Ajin Cherian wrote: > > > I am not able to apply the patch to the latest head or even to a week > > back version. Can you please check and rebase? > > > > thanks > > Shveta > > Rebased. > Thanks. Please find a few comments: 1) /* Any slot with NULL in these field

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-07-16 Thread shveta malik
On Wed, Jul 16, 2025 at 3:00 PM Ajin Cherian wrote: > > On Wed, Jul 2, 2025 at 7:56 PM shveta malik wrote: > > > > Few comments: > > > > 1) > > When the API is waiting for the primary to advance, standby fails to > > handle promotion requests. Promot

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-15 Thread shveta malik
On Tue, Jul 15, 2025 at 10:37 PM Masahiko Sawada wrote: > > I've attached updated patches that implement the idea we've discussed. > The patches still need to be polished but the implemented ideas seem > good. Feedback is very welcome. > Thank You for the patches. I just tried my hands on ptach00

Re: Logical replication launcher did not automatically restart when got SIGKILL

2025-07-15 Thread shveta malik
On Tue, Jul 15, 2025 at 2:56 PM cca5507 wrote: > > Hi, hackers > > I found the $SUBJECT, the main reason is that RegisteredBgWorker::rw_pid has > not been cleaned. > > Attach a patch to fix it. > Thank You for reporting this. The problem exists and the patch works as expected. In the patch, we

Re: Logical Replication of sequences

2025-07-14 Thread shveta malik
On Mon, Jul 14, 2025 at 10:03 AM vignesh C wrote: > > On Fri, 11 Jul 2025 at 14:26, shveta malik wrote: > > > > On Wed, Jul 9, 2025 at 4:11 PM vignesh C wrote: > > > > > > > > > 3) > > > > SyncFetchRelationStates: > > > >

Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

2025-07-14 Thread shveta malik
On Mon, Jul 14, 2025 at 2:08 PM Peter Smith wrote: > > Hi Shveta, > > Thanks for your README questions. > > On Fri, Jul 11, 2025 at 1:46 PM shveta malik wrote: > > > > Thank You for working on this. I started going through the README and > > tried run

Re: Logical Replication of sequences

2025-07-11 Thread shveta malik
On Wed, Jul 9, 2025 at 4:11 PM vignesh C wrote: > > > 3) > > SyncFetchRelationStates: > > Earlier the name was FetchTableStates. If we really want to use the > > 'Sync' keyword, we can name it FetchRelationSyncStates, as we are > > fetching sync-status only. Thoughts? > > Instead of FetchRelatio

Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

2025-07-10 Thread shveta malik
Thank You for working on this. I started going through the README and tried running simple tests, have few concerns: 1) I am not able to understand section 4.2 'WOS-to-ROS conversion'. When whiteout-WOS says 'delete 4', what does that mean? 4 is CRID, TXID? And when does delete-vector X represents

Re: Logical Replication of sequences

2025-07-09 Thread shveta malik
Please find a few more comments on July4 patch 6) + + To synchronize sequences from a publisher to a subscriber, first publish + them using + CREATE PUBLICATION ... FOR ALL SEQUENCES. + This sentence looks odd, as we have 'first' but no follow-up sentence after that. Can we please comb

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-07-08 Thread shveta malik
Please find few more comments: 1) In pg_sync_replication_slots() doc, we have this: "Note that this function is primarily intended for testing and debugging purposes and should be used with caution. Additionally, this function cannot be executed if " We can get rid of this info as well and

Re: Using failover slots for PG-non_PG logical replication

2025-07-08 Thread shveta malik
On Tue, Jul 8, 2025 at 7:29 PM Ashutosh Bapat wrote: > > On Tue, Jul 8, 2025 at 4:03 PM shveta malik wrote: > > > > On Mon, Jul 7, 2025 at 7:00 PM Ashutosh Bapat > > wrote: > > > > > > On Mon, Jul 7, 2025 at 9:53 AM shveta malik > &g

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-08 Thread shveta malik
On Sun, Jul 6, 2025 at 8:33 PM Masahiko Sawada wrote: > > On Thu, Jul 3, 2025 at 3:32 PM shveta malik wrote: > > > > On Wed, Jul 2, 2025 at 9:46 PM Masahiko Sawada > > wrote: > > > > > > On Wed, Jun 18, 2025 at 1:07 PM shveta malik > > > wro

Re: Using failover slots for PG-non_PG logical replication

2025-07-08 Thread shveta malik
On Mon, Jul 7, 2025 at 7:00 PM Ashutosh Bapat wrote: > > On Mon, Jul 7, 2025 at 9:53 AM shveta malik wrote: > > > > Thanks for the patch. > > > > > I couldn't figure out whether the query on primary to fetch all the > > > slots to be synchron

Re: Logical Replication of sequences

2025-07-07 Thread shveta malik
On Mon, Jul 7, 2025 at 2:37 PM shveta malik wrote: > > On Fri, Jul 4, 2025 at 3:53 PM vignesh C wrote: > > > > On Tue, 1 Jul 2025 at 15:20, shveta malik wrote: > > > On Mon, Jun 30, 2025 at 3:21 PM Nisha Moond > > > wrote: > > > > >

Re: Logical Replication of sequences

2025-07-07 Thread shveta malik
On Fri, Jul 4, 2025 at 3:53 PM vignesh C wrote: > > On Tue, 1 Jul 2025 at 15:20, shveta malik wrote: > > On Mon, Jun 30, 2025 at 3:21 PM Nisha Moond > > wrote: > > > > > > Please find the attached v20250630 patch set addressing above comments > >

Re: Using failover slots for PG-non_PG logical replication

2025-07-06 Thread shveta malik
On Fri, Jul 4, 2025 at 7:42 PM Ashutosh Bapat wrote: > > On Fri, Jul 4, 2025 at 9:23 AM Amit Kapila wrote: > > > > > > > > > > How about this: > > > We change the following sentence in the third paragraph > > > To confirm that the standby server is indeed ready for failover > > addition> so that

Re: Logical Replication of sequences

2025-07-03 Thread shveta malik
Few more concerns: 4) In UpdateSubscriptionRelState(): if (!HeapTupleIsValid(tup)) elog(ERROR, "subscription table %u in subscription %u does not exist", relid, subid); table-->relation as now it can be hit for both sequence and table. 5) In Logic

Re: Using failover slots for PG-non_PG logical replication

2025-07-02 Thread shveta malik
On Thu, Jul 3, 2025 at 9:32 AM Amit Kapila wrote: > > On Wed, Jul 2, 2025 at 5:50 PM Ashutosh Bapat > wrote: > > > > On Wed, Jul 2, 2025 at 12:36 PM shveta malik wrote: > > > > > > On Wed, Jul 2, 2025 at 10:50 AM Ashutosh Bapat > > > wrote: >

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-02 Thread shveta malik
On Wed, Jul 2, 2025 at 9:46 PM Masahiko Sawada wrote: > > On Wed, Jun 18, 2025 at 1:07 PM shveta malik wrote: > > > > On Wed, Jun 18, 2025 at 6:06 AM Masahiko Sawada > > wrote: > > > > > > Thank you for the comments! > > > > > > &g

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-07-02 Thread shveta malik
On Tue, Jun 24, 2025 at 4:11 PM Ajin Cherian wrote: > > Hello, > > Creating this thread for a POC based on discussions in thread [1]. > Hou-san had created this patch, and I just cleaned up some documents, > did some testing and now sharing the patch here. > > In this patch, the pg_sync_replicatio

Re: Using failover slots for PG-non_PG logical replication

2025-07-02 Thread shveta malik
On Wed, Jul 2, 2025 at 10:50 AM Ashutosh Bapat wrote: > > Hi All, > > The failover slots documentation [1] is good for PG - PG logical > replication, but the first two queries require pg_subscription which > may not be present in non-PG downstream. Somebody looking to setup > failover slots for no

Re: Logical Replication of sequences

2025-07-01 Thread shveta malik
On Mon, Jun 30, 2025 at 3:21 PM Nisha Moond wrote: > > Please find the attached v20250630 patch set addressing above comments > and other comments in [1],[2],[3] and [4]. Thanks for the patches. I am still in process of reviewing it but please find few comments: 1) + if (pset.sversion >= 18)

Re: Logical Replication of sequences

2025-06-30 Thread shveta malik
On Mon, Jun 30, 2025 at 3:21 PM Nisha Moond wrote: > > Tab-completion is not supported after a comma (,) in any other cases. > For example, the following commands are valid, but tab-completion does > not work after the comma: > > CREATE PUBLICATION pub7 FOR TABLE t1, TABLES IN SCHEMA public; > CRE

Correct comment atop PublicationPartOpt

2025-06-30 Thread shveta malik
Hi, I found the wrong function name mentioned in the comment atop PublicationPartOpt. Comment wrongly mentioned the function name GetRelationPublications() for PublicationPartOpt usage instead of GetPublicationRelations(). Corrected the comment. thanks Shveta v1-0001-Comment-correction-atop-Pu

Re: Skipping schema changes in publication

2025-06-30 Thread shveta malik
Few more comments on 002: 5) +GetAllTablesPublicationRelations(Oid pubid, bool pubviaroot) { + List*exceptlist; + + exceptlist = GetPublicationRelations(pubid, PUBLICATION_PART_ALL); a) Here, we are assuming that the list provided by GetPublicationRelations() will be except-tables list onl

Re: Skipping schema changes in publication

2025-06-29 Thread shveta malik
On Fri, Jun 27, 2025 at 3:44 PM Shlok Kyal wrote: > > On Thu, 26 Jun 2025 at 15:27, shveta malik wrote: > > > > On Tue, Jun 24, 2025 at 9:48 AM Shlok Kyal wrote: > > > > > > I have included the changes for > > > it in v14-0003 patch. > > >

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-29 Thread shveta malik
On Mon, Jun 30, 2025 at 11:16 AM Masahiko Sawada wrote: > > On Thu, Jun 26, 2025 at 5:50 PM shveta malik wrote: > > > > On Wed, Jun 25, 2025 at 12:20 PM Bertrand Drouvot > > wrote: > > > > > > Hi, > > > > > > On Wed, Jun 25, 2025 at

Re: Conflict detection for update_deleted in logical replication

2025-06-27 Thread shveta malik
On Fri, Jun 27, 2025 at 7:58 AM Zhijie Hou (Fujitsu) wrote: > > On Thu, Jun 26, 2025 at 4:28 PM shveta malik wrote: > > > > On Thu, Jun 26, 2025 at 8:31 AM Zhijie Hou (Fujitsu) > > wrote: > > > > > > Thanks for the comments. All of them look good

Re: Logical Replication of sequences

2025-06-26 Thread shveta malik
On Wed, Jun 25, 2025 at 7:42 PM Shlok Kyal wrote: > > On Sun, 22 Jun 2025 at 08:05, vignesh C wrote: > > > > On Thu, 19 Jun 2025 at 11:26, Nisha Moond wrote: > > > > > > Hi, > > > > > > Here are my review comments for v20250610 patches: > > > > > > Patch-0005:sequencesync.c > > > > > > 1) report

Re: Skipping schema changes in publication

2025-06-26 Thread shveta malik
On Tue, Jun 24, 2025 at 9:48 AM Shlok Kyal wrote: > > I have included the changes for > it in v14-0003 patch. > Thanks for the patches. I have reviewed patch001 alone, please find few comments: 1) + + The RESET clause will reset the publication to the + default state which includes resetti

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-26 Thread shveta malik
On Wed, Jun 25, 2025 at 12:20 PM Bertrand Drouvot wrote: > > Hi, > > On Wed, Jun 25, 2025 at 09:15:04AM +0530, shveta malik wrote: > > On Wed, Jun 25, 2025 at 9:12 AM shveta malik wrote: > > > > > > On Tue, Jun 24, 2025 at 2:12 PM Bertrand Drouvot > > &g

Re: Conflict detection for update_deleted in logical replication

2025-06-26 Thread shveta malik
On Thu, Jun 26, 2025 at 8:31 AM Zhijie Hou (Fujitsu) wrote: > > Thanks for the comments. All of them look good to me and > have been addressed in V42. > Thank You for the patches. Few comments. t/035_conflicts.pl: 1) Both the subscriptions subname_BA and subname_AB have rci enabled during CREAT

Re: Conflict detection for update_deleted in logical replication

2025-06-25 Thread shveta malik
On Mon, Jun 23, 2025 at 4:20 PM Zhijie Hou (Fujitsu) wrote: > > > Here is the V40 patch set Thanks for the patches. Few comments: 1) In get_subscription_info(), we are doing COUNT of rci-subscriptions using below query: SELECT count(*) AS nsub, COUNT(CASE WHEN subretainconflictinfo THEN 1 END) A

Re: Conflict detection for update_deleted in logical replication

2025-06-24 Thread shveta malik
> > Here is the V41 patch set which includes the following changes: > Thanks for the patches. Few trivial things: 1) In ReplicationSlotAcquire(), does it make more sense to move the error after checking the slot's existence first? If a user is trying to use a slot which does not exist, he should

Re: Logical Replication of sequences

2025-06-24 Thread shveta malik
On Tue, Jun 24, 2025 at 6:44 PM Shlok Kyal wrote: > > > 1. Initially, I have created a publication on sequence s1. > postgres=# CREATE PUBLICATION pub1 FOR ALL SEQUENCES; > CREATE PUBLICATION > postgres=# ALTER PUBLICATION pub1 SET TABLE t1; > ALTER PUBLICATION > postgres=# \d s1 >

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-24 Thread shveta malik
On Wed, Jun 25, 2025 at 9:12 AM shveta malik wrote: > > On Tue, Jun 24, 2025 at 2:12 PM Bertrand Drouvot > wrote: > > > > Hi, > > > > On Tue, Jun 24, 2025 at 12:13:32AM +0900, Masahiko Sawada wrote: > > > On Mon, Jun 23, 2025 at 7:01 PM Bertran

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-24 Thread shveta malik
On Tue, Jun 24, 2025 at 2:12 PM Bertrand Drouvot wrote: > > Hi, > > On Tue, Jun 24, 2025 at 12:13:32AM +0900, Masahiko Sawada wrote: > > On Mon, Jun 23, 2025 at 7:01 PM Bertrand Drouvot > > wrote: > > > > > > Hi, > > > > > > On Mon, Jun 23, 2025 at 05:10:37PM +0900, Masahiko Sawada wrote: > > > >

Re: Logical Replication of sequences

2025-06-24 Thread shveta malik
> > Thanks for the comment, the attached v20250622 version patch has the > changes for the same. > Thanks for the patches, I am not done with review yet, but please find the feedback so far: 1) + if (!OidIsValid(seq_relid)) + ereport(ERROR, + errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), +

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-23 Thread shveta malik
On Mon, Jun 23, 2025 at 1:41 PM Masahiko Sawada wrote: > > Given the discussion so far, it seems we might want to have a > safeguard to prevent the effective_wal_level from being dropped to > 'replica' if the last logical slot is accidentally dropped. Yes, needed for cases where standby or cascad

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-23 Thread shveta malik
On Fri, Jun 20, 2025 at 12:15 PM Bertrand Drouvot wrote: > > Hi, > > On Fri, Jun 20, 2025 at 09:48:47AM +0530, shveta malik wrote: > > On Thu, Jun 19, 2025 at 2:30 PM Bertrand Drouvot > > wrote: > > > > > > I wonder if a way to address the concerns th

Re: Conflict detection for update_deleted in logical replication

2025-06-23 Thread shveta malik
> > Here is the V39 patch set which includes the following changes: > Few trivial comments: 1) Currently we have this error and detail: ERROR: Enabling retain_conflict_info requires "wal_level" >= "replica" DETAIL: A replication slot must be created to retain conflict information. Shall we ch

Re: Conflict detection for update_deleted in logical replication

2025-06-20 Thread shveta malik
On Thu, Jun 19, 2025 at 4:34 PM shveta malik wrote: > > > > > Here is the V38 patch set which includes the following changes: > > > > Thank You for the patches. Few comments: > > 1) > + > +Note that commit timestamps and origin data retain

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-19 Thread shveta malik
On Thu, Jun 19, 2025 at 2:30 PM Bertrand Drouvot wrote: > > I wonder if a way to address the concerns that we shared above is to use a > mixed approach like: > > - Forget the immediately_reserve idea > - If a user creates a logical slot then we automatically switch to wal_level = > logical (if not

Re: Conflict detection for update_deleted in logical replication

2025-06-19 Thread shveta malik
> > Here is the V38 patch set which includes the following changes: > Thank You for the patches. Few comments: 1) + +Note that commit timestamps and origin data retained by enabling the +retain_conflict_info +option will not be preserved during the upgrade. As a +result, the up

Re: Fix slot synchronization with two_phase decoding enabled

2025-06-18 Thread shveta malik
On Tue, Jun 17, 2025 at 5:17 AM Masahiko Sawada wrote: > > On Wed, Jun 11, 2025 at 9:50 PM shveta malik wrote: > > > > On Wed, Jun 11, 2025 at 11:31 PM Masahiko Sawada > > wrote: > > > > > > > > BTW have we addressed the point Amit mentioned befo

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-18 Thread shveta malik
On Wed, Jun 18, 2025 at 2:39 PM Bertrand Drouvot wrote: > > Hi, > > On Tue, Jun 10, 2025 at 02:00:55PM -0700, Masahiko Sawada wrote: > > > > > > > 0001 patch allows us to create a logical slot without WAL > > > > > > > reservation. > > Thanks for the patch and sorry to be late in this conversatio

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-17 Thread shveta malik
On Wed, Jun 18, 2025 at 6:06 AM Masahiko Sawada wrote: > > Thank you for the comments! > > > > > 2) > > I see that when primary switches back its effective wal_level to > > replica while standby has wal_level=logical in conf file, then standby > > has this status: > > > > postgres=# show wal_level

Re: Replication slot is not able to sync up

2025-06-17 Thread shveta malik
On Tue, Jun 17, 2025 at 12:01 PM Amit Kapila wrote: > > This whole paragraph sounds like a duplicate of its previous section, > and the line alignment in the first paragraph has some issues. > Sorry for the wrong upload, duplicacy was the merge issue. Removed the duplicate paragraph and corrected

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-16 Thread shveta malik
On Wed, Jun 11, 2025 at 2:31 AM Masahiko Sawada wrote: > > I think it's the user's responsibility to keep at least one logical > slot. It seems that setting wal_level to 'logical' would be the most > reliable solution for this case. We might want to provide a way to > keep 'logical' WAL level some

  1   2   3   4   5   6   >