RE: Replica Identity check of partition table on subscriber

2022-06-14 Thread shiy.f...@fujitsu.com
On Tue, Jun 14, 2022 8:57 PM Amit Kapila wrote: > > > v4-0002 looks good btw, except the bitpick about test comment similar > > to my earlier comment regarding v5-0001: > > > > +# Change the column order of table on publisher > > > > I think it might be better to say something specific to

RE: Replica Identity check of partition table on subscriber

2022-06-14 Thread shiy.f...@fujitsu.com
On Tue, Jun 14, 2022 2:18 PM Amit Langote wrote: > > On Mon, Jun 13, 2022 at 9:26 PM Amit Kapila > wrote: > > On Mon, Jun 13, 2022 at 1:03 PM houzj.f...@fujitsu.com > > wrote: > > > On Monday, June 13, 2022 1:53 PM Amit Kapila > wrote: > > > I have separated out the bug-fix for the

RE: tablesync copy ignores publication actions

2022-06-13 Thread shiy.f...@fujitsu.com
On Wed, Jun 8, 2022 12:10 PM Amit Kapila wrote: > > On Tue, Jun 7, 2022 at 7:08 PM Euler Taveira wrote: > > > > On Tue, Jun 7, 2022, at 1:10 AM, Peter Smith wrote: > > > > The logical replication tablesync ignores the publication 'publish' > > operations during the initial data copy. > > > >

RE: Replica Identity check of partition table on subscriber

2022-06-10 Thread shiy.f...@fujitsu.com
On Thu, June 9, 2022 7:02 PM Amit Kapila wrote: > > > I think one approach to fix it is to check the target partition in this > > case, > > instead of the partitioned table. > > > > This approach sounds reasonable to me. One minor point: > +/* > + * Check that replica identity matches. > + * >

Replica Identity check of partition table on subscriber

2022-06-08 Thread shiy.f...@fujitsu.com
Hi hackers, I saw a problem in logical replication, when the target table on subscriber is a partitioned table, it only checks whether the Replica Identity of partitioned table is consistent with the publisher, and doesn't check Replica Identity of the partition. For example: -- publisher --

RE: Handle infinite recursion in logical replication setup

2022-06-06 Thread shiy.f...@fujitsu.com
On Mon, Jun 6, 2022 1:14 PM vignesh C wrote: > > The attached v18 patch has the fixes for the same. > Thanks for updating the patch, here are some comments. 0002 patch == 1. + +origin (string) + + It maybe better if the type of "origin" parameter

RE: Hash index build performance tweak from sorting

2022-05-30 Thread shiy.f...@fujitsu.com
On Tue, May 10, 2022 5:43 PM Simon Riggs wrote: > > On Sat, 30 Apr 2022 at 12:12, Amit Kapila > wrote: > > > > Few comments on the patch: > > 1. I think it is better to use DatumGetUInt32 to fetch the hash key as > > the nearby code is using. > > 2. You may want to change the below comment in

RE: Handle infinite recursion in logical replication setup

2022-05-27 Thread shiy.f...@fujitsu.com
On Wed, May 25, 2022 7:55 PM vignesh C wrote: > > The attached v16 patch has the changes for the same. > Thanks for updating the patch. Some comments for the document in 0002 patch. 1. + +Lock the required tables in node1 and +node2 till the setup is completed. + + + +

RE: Handle infinite recursion in logical replication setup

2022-05-23 Thread shiy.f...@fujitsu.com
On Tue, May 24, 2022 1:34 AM vignesh C wrote: > > Thanks for the comments, the attached v15 patch has the fixes for the same. > Thanks for updating the patch. I have a comment on the document in 0002 patch. @@ -300,6 +310,11 @@ CREATE SUBSCRIPTION subscription_namefalse. +

RE: Handle infinite recursion in logical replication setup

2022-05-19 Thread shiy.f...@fujitsu.com
On Wed, May 4, 2022 2:47 PM vignesh C wrote: > > Thanks for the comments, the attached v13 patch has the changes for the same. > Here are some comments on v13-0002 patch. 1) +* Throw an error so that the user can take care of the initial data +* copying and

RE: Handle infinite recursion in logical replication setup

2022-05-18 Thread shiy.f...@fujitsu.com
On Wed, May 4, 2022 2:47 PM vignesh C wrote: > > Thanks for the comments, the attached v13 patch has the changes for the same. > Thanks for your patch. Here are some comments on v13-0001 patch. 1) @@ -152,6 +152,18 @@ CREATE SUBSCRIPTION subscription_name + +

RE: Skipping schema changes in publication

2022-05-17 Thread shiy.f...@fujitsu.com
On Sat, May 14, 2022 9:33 PM vignesh C wrote: > > Thanks for the comments, the attached v5 patch has the changes for the > same. Also I have made the changes for SKIP Table based on the new > syntax, the changes for the same are available in >

RE: bogus: logical replication rows/cols combinations

2022-05-16 Thread shiy.f...@fujitsu.com
On Mon, May 16, 2022 8:34 PM houzj.f...@fujitsu.com wrote: > > Attach the new version patch. > Thanks for your patch. Here are some comments: 1. (0001 patch) /* * Returns Oids of tables in a publication. */ Datum pg_get_publication_tables(PG_FUNCTION_ARGS) Should we modify the comment of

RE: Data is copied twice when specifying both child and parent table in publication

2022-04-27 Thread shiy.f...@fujitsu.com
On Sun, Apr 24, 2022 2:16 PM Wang, Wei/王 威 wrote: > > Attach the new patches.[suggestions by Amit-San] > The patch for HEAD: > 1. Add a new function to get tables info by a publications array. > The patch for REL14: > 1. Use an alias to make the statement understandable. BTW, I adjusted the >

RE: Data is copied twice when specifying both child and parent table in publication

2022-04-19 Thread shiy.f...@fujitsu.com
On Tue, Apr 19, 2022 3:05 PM houzj.f...@fujitsu.com wrote: > > > -Original Message- > > From: Wang, Wei/王 威 > On Thursday, April 7, 2022 11:08 AM > > > > On Thur, Mar 10, 2021 at 10:08 AM houzj.f...@fujitsu.com wrote: > > > Hi, > > > > > > When reviewing some logical replication related

RE: Skipping schema changes in publication

2022-04-14 Thread shiy.f...@fujitsu.com
On Tue, Apr 12, 2022 2:23 PM vignesh C wrote: > > The patch does not apply on top of HEAD because of the recent commit, > attached patch is rebased on top of HEAD. > Thanks for your patch. Here are some comments for 0001 patch. 1. doc/src/sgml/catalogs.sgml @@ -6438,6 +6438,15 @@

RE: Logical replication timeout problem

2022-03-30 Thread shiy.f...@fujitsu.com
On Wed, Mar 30, 2022 3:54 PM wangw.f...@fujitsu.com wrote: > > Rebase the patch because the commit d5a9d86d in current HEAD. > Thanks for your patch, I tried this patch and confirmed that there is no timeout problem after applying this patch, and I could reproduce this problem on HEAD.

RE: logical replication empty transactions

2022-03-29 Thread shiy.f...@fujitsu.com
On Tue, Mar 29, 2022 5:15 PM Hou, Zhijie/侯 志杰 wrote: > > Thanks for the comment. > Attach the new version patch with this change. > Hi, I did a performance test for this patch to see if it affects performance when publishing empty transactions, based on the v32 patch. In this test, I use

RE: logical replication empty transactions

2022-03-23 Thread shiy.f...@fujitsu.com
On Thursday, March 24, 2022 11:19 AM Hou, Zhijie/侯 志杰 wrote: > > Attach the new version patch which include the following changes: > > - Fix a typo > - Change the requestreply flag of the newly added WalSndKeepalive to false, > because the subscriber can judge whether it's necessary to post

RE: Skipping logical replication transactions on subscriber side

2022-03-16 Thread shiy.f...@fujitsu.com
On Wed, Mar 16, 2022 4:23 PM Masahiko Sawada wrote: > > I've attached an updated version patch. > Thanks for updating the patch. Here are some comments for the v15 patch. 1. src/backend/replication/logical/worker.c + * to skip applying the changes when starting to apply changes. The

RE: Column Filtering in Logical Replication

2022-03-14 Thread shiy.f...@fujitsu.com
On Mon, Mar 14, 2022 5:08 AM Tomas Vondra wrote: > > On 3/12/22 05:30, Amit Kapila wrote: > >> ... > > > > Okay, please find attached. I have done basic testing of this, if we > > agree with this approach then this will require some more testing. > > > > Thanks, the proposed changes seem like a

RE: Skipping logical replication transactions on subscriber side

2022-03-14 Thread shiy.f...@fujitsu.com
On Fri, Mar 11, 2022 4:20 PM Masahiko Sawada wrote: > > I've attached an updated version patch. This patch can be applied on > top of the latest disable_on_error patch[1]. > Thanks for your patch. Here are some comments for the v13 patch. 1. doc/src/sgml/ref/alter_subscription.sgml +

RE: logical replication empty transactions

2022-03-07 Thread shiy.f...@fujitsu.com
On Fri, Mar 4, 2022 9:41 AM Ajin Cherian wrote: > > I have split the patch into two. I have kept the logic of skipping > streaming changes in the second patch. > I will work on the second patch once we can figure out a solution for > the COMMIT PREPARED after restart problem. > Thanks for

RE: Optionally automatically disable logical replication subscriptions on error

2022-03-06 Thread shiy.f...@fujitsu.com
On Wed, Mar 2, 2022 5:39 PM osumi.takami...@fujitsu.com wrote: > > Attached an updated patch v26. > Thanks for your patch. A comment on the document. @@ -7771,6 +7771,16 @@ SCRAM-SHA-256$iteration count: + subdisableonerr bool + + + If true, the

RE: row filtering for logical replication

2022-03-02 Thread shiy.f...@fujitsu.com
On Thu, Mar 3, 2022 10:40 AM Amit Kapila wrote: > > On Wed, Mar 2, 2022 at 5:42 PM Euler Taveira wrote: > > > > On Wed, Mar 2, 2022, at 8:45 AM, Tomas Vondra wrote: > > > > While working on the column filtering patch, which touches about the > > same places, I noticed two minor gaps in testing:

RE: Failed transaction statistics to measure the logical replication progress

2022-03-02 Thread shiy.f...@fujitsu.com
Hi, A comments on the v26 patch. The following document about pg_stat_subscription_stats view only says that "showing statistics about errors", should we add something about transactions here? pg_stat_subscription_statspg_stat_subscription_stats One row per subscription,

RE: logical replication empty transactions

2022-03-01 Thread shiy.f...@fujitsu.com
Hi, Here are some comments on the v21 patch. 1. + WalSndKeepalive(false, 0); Maybe we can use InvalidXLogRecPtr here, instead of 0. 2. + pq_sendint64(_message, writePtr ? writePtr : sentPtr); Similarly, should we use XLogRecPtrIsInvalid()? 3. @@ -1183,6 +1269,20

RE: Data is copied twice when specifying both child and parent table in publication

2021-10-21 Thread shiy.f...@fujitsu.com
On Tuesday, October 19, 2021 10:47 AM houzj.f...@fujitsu.com wrote: > > On Monday, October 18, 2021 5:03 PM Amit Langote > wrote: > > I can imagine that the behavior seen here may look surprising, but not > > sure if I would call it a bug as such. I do remember thinking about > > this case

RE: Truncate in synchronous logical replication failed

2021-04-22 Thread shiy.f...@fujitsu.com
> I don't think here we need to restart to get a stable list of indexes > as we do in RelationGetIndexAttrBitmap. The reason is here we build > the cache entry using a historic snapshot and all the later changes > are absorbed while decoding WAL. I have updated that and modified few > comments in

撤回: Could you help testing logical replication?

2021-04-12 Thread shiy.f...@fujitsu.com
Shi, Yu/侍 雨 将撤回邮件“Could you help testing logical replication?”。

撤回: Could you help testing logical replication?

2021-04-12 Thread shiy.f...@fujitsu.com
Shi, Yu/侍 雨 将撤回邮件“Could you help testing logical replication?”。

RE: Could you help testing logical replication?

2021-04-12 Thread shiy.f...@fujitsu.com
Sorry for sending a wrong mail. Please ignore it. > -Original Message- > From: Shi, Yu/侍 雨 > Sent: Monday, April 12, 2021 6:51 PM > To: Tang, Haiying/唐 海英 > Cc: pgsql-hackers@lists.postgresql.org > Subject: RE: Could you help testing logical replication? > > > Then I get timeout error

RE: Could you help testing logical replication?

2021-04-12 Thread shiy.f...@fujitsu.com
> Then I get timeout error occurs and the subscriber worker keep re-launching > over and over (you did not mention see such errors?) I test again and get errors, too. I didn't check log after timeout in the previous test. Regards, Tang

RE: psql - add SHOW_ALL_RESULTS option

2021-04-07 Thread shiy.f...@fujitsu.com
> Attached a patch which attempts to fix this by moving the cancellation > cancelling request after processing results. Thank you for your fixing. I tested and the problem has been solved after applying your patch. Regards, Shi yu

RE: psql - add SHOW_ALL_RESULTS option

2021-04-07 Thread shiy.f...@fujitsu.com
Hi I met a problem after commit 3a51306722. While executing a SQL statement with psql, I can't interrupt it by pressing ctrl+c. For example: postgres=# insert into test select generate_series(1,1000); ^C^CINSERT 0 1000 Press ctrl+c before finishing INSERT, and psql still continuing

RE: Table refer leak in logical replication

2021-04-05 Thread shiy.f...@fujitsu.com
> BTW, it seems better to add a testcase for this ? I think the test for it can be added in src/test/subscription/t/003_constraints.pl, which is like what in my patch. Regards, Shi yu tests_for_table_refer_leak.diff Description: tests_for_table_refer_leak.diff

<    1   2