Re: bogus: logical replication rows/cols combinations

2022-06-15 Thread Amit Kapila
On Mon, Jun 13, 2022 at 8:54 AM Amit Kapila wrote: > > I would like to close the Open item listed corresponding to this > thread [1] as the fix for the reported issue is committed > (fd0b9dcebd). Do let me know if you or others think otherwise? > Seeing no objections, I have closed this item.

Re: bogus: logical replication rows/cols combinations

2022-06-12 Thread Amit Kapila
On Wed, Jun 8, 2022 at 11:05 AM Peter Smith wrote: > > On Wed, Jun 8, 2022 at 1:25 PM Justin Pryzby wrote: > > > > On Mon, Jun 06, 2022 at 03:42:31PM +1000, Peter Smith wrote: > > > I noticed the patch "0001-language-fixes-on-HEAD-from-Justin.patch" says: > > > > > > @@ -11673,7 +11673,7 @@ > >

Re: bogus: logical replication rows/cols combinations

2022-06-07 Thread Peter Smith
On Wed, Jun 8, 2022 at 1:25 PM Justin Pryzby wrote: > > On Mon, Jun 06, 2022 at 03:42:31PM +1000, Peter Smith wrote: > > I noticed the patch "0001-language-fixes-on-HEAD-from-Justin.patch" says: > > > > @@ -11673,7 +11673,7 @@ > >prosrc => 'pg_show_replication_origin_status' }, > > > > #

Re: bogus: logical replication rows/cols combinations

2022-06-07 Thread Justin Pryzby
On Mon, Jun 06, 2022 at 03:42:31PM +1000, Peter Smith wrote: > I noticed the patch "0001-language-fixes-on-HEAD-from-Justin.patch" says: > > @@ -11673,7 +11673,7 @@ >prosrc => 'pg_show_replication_origin_status' }, > > # publications > -{ oid => '6119', descr => 'get information of tables

Re: bogus: logical replication rows/cols combinations

2022-06-05 Thread Peter Smith
On Thu, Jun 2, 2022 at 9:58 PM Amit Kapila wrote: > > On Fri, May 27, 2022 at 1:04 PM houzj.f...@fujitsu.com > wrote: > > > > On Friday, May 27, 2022 1:54 PM Justin Pryzby wrote: > > > > > > On Fri, May 27, 2022 at 11:17:00AM +0530, Amit Kapila wrote: > > > > On Tue, May 24, 2022 at 11:03 AM

Re: bogus: logical replication rows/cols combinations

2022-06-02 Thread Amit Kapila
On Fri, May 27, 2022 at 1:04 PM houzj.f...@fujitsu.com wrote: > > On Friday, May 27, 2022 1:54 PM Justin Pryzby wrote: > > > > On Fri, May 27, 2022 at 11:17:00AM +0530, Amit Kapila wrote: > > > On Tue, May 24, 2022 at 11:03 AM houzj.f...@fujitsu.com > > wrote: > > > > > > > > On Friday, May 20,

RE: bogus: logical replication rows/cols combinations

2022-05-27 Thread houzj.f...@fujitsu.com
On Friday, May 27, 2022 1:54 PM Justin Pryzby wrote: > > On Fri, May 27, 2022 at 11:17:00AM +0530, Amit Kapila wrote: > > On Tue, May 24, 2022 at 11:03 AM houzj.f...@fujitsu.com > wrote: > > > > > > On Friday, May 20, 2022 11:06 AM Amit Kapila > wrote: > > > > > > Thanks for pointing it out.

Re: bogus: logical replication rows/cols combinations

2022-05-26 Thread Justin Pryzby
On Fri, May 27, 2022 at 11:17:00AM +0530, Amit Kapila wrote: > On Tue, May 24, 2022 at 11:03 AM houzj.f...@fujitsu.com > wrote: > > > > On Friday, May 20, 2022 11:06 AM Amit Kapila > > wrote: > > > > Thanks for pointing it out. Here is the new version patch which add this > > version check. >

Re: bogus: logical replication rows/cols combinations

2022-05-26 Thread Amit Kapila
On Tue, May 24, 2022 at 11:03 AM houzj.f...@fujitsu.com wrote: > > On Friday, May 20, 2022 11:06 AM Amit Kapila wrote: > > Thanks for pointing it out. Here is the new version patch which add this > version check. > I have added/edited a few comments and ran pgindent. The attached looks good to

Re: bogus: logical replication rows/cols combinations

2022-05-25 Thread Amit Kapila
On Tue, May 24, 2022 at 3:19 PM Amit Kapila wrote: > > On Fri, May 20, 2022 at 8:36 AM Amit Kapila wrote: > > > > On Thu, May 19, 2022 at 7:54 PM Tom Lane wrote: > > > > > > Justin Pryzby writes: > > > > On Thu, May 19, 2022 at 10:33:13AM +0530, Amit Kapila wrote: > > > >> I have committed the

Re: bogus: logical replication rows/cols combinations

2022-05-24 Thread Amit Kapila
On Fri, May 20, 2022 at 8:36 AM Amit Kapila wrote: > > On Thu, May 19, 2022 at 7:54 PM Tom Lane wrote: > > > > Justin Pryzby writes: > > > On Thu, May 19, 2022 at 10:33:13AM +0530, Amit Kapila wrote: > > >> I have committed the first patch after fixing this part. It seems Tom > > >> is not very

RE: bogus: logical replication rows/cols combinations

2022-05-23 Thread houzj.f...@fujitsu.com
On Friday, May 20, 2022 11:06 AM Amit Kapila wrote: > > On Thu, May 19, 2022 at 7:54 PM Tom Lane wrote: > > > Even more to the point, how can we > > have a subscriber do that by relying on view columns that don't exist > > in older versions? > > > > We need a version check like (if >

Re: bogus: logical replication rows/cols combinations

2022-05-19 Thread Amit Kapila
On Thu, May 19, 2022 at 7:54 PM Tom Lane wrote: > > Justin Pryzby writes: > > On Thu, May 19, 2022 at 10:33:13AM +0530, Amit Kapila wrote: > >> I have committed the first patch after fixing this part. It seems Tom > >> is not very happy doing this after beta-1 [1]. The reason we get this > >>

Re: bogus: logical replication rows/cols combinations

2022-05-19 Thread Tom Lane
Justin Pryzby writes: > On Thu, May 19, 2022 at 10:33:13AM +0530, Amit Kapila wrote: >> I have committed the first patch after fixing this part. It seems Tom >> is not very happy doing this after beta-1 [1]. The reason we get this >> information via this view (and underlying function) is that it

Re: bogus: logical replication rows/cols combinations

2022-05-19 Thread Justin Pryzby
On Thu, May 19, 2022 at 10:33:13AM +0530, Amit Kapila wrote: > I have committed the first patch after fixing this part. It seems Tom > is not very happy doing this after beta-1 [1]. The reason we get this > information via this view (and underlying function) is that it > simplifies the queries on

Re: bogus: logical replication rows/cols combinations

2022-05-18 Thread Amit Kapila
On Mon, May 16, 2022 at 6:50 PM Alvaro Herrera wrote: > > On 2022-May-16, Amit Kapila wrote: > > > Few comments: > > = > > 1. > > postgres=# select * from pg_publication_tables; > > pubname | schemaname | tablename | columnlist | rowfilter > >

Re: bogus: logical replication rows/cols combinations

2022-05-17 Thread Amit Kapila
On Tue, May 17, 2022 at 2:40 PM houzj.f...@fujitsu.com wrote: > > Attach the new version patch which addressed all the above comments and > comments from Shi yu[1] and Osumi-san[2]. > Thanks, your first patch looks good to me. I'll commit that tomorrow unless there are more comments on the same.

RE: bogus: logical replication rows/cols combinations

2022-05-17 Thread houzj.f...@fujitsu.com
On Tuesday, May 17, 2022 2:53 PM Amit Kapila wrote: > > Few minor comments: > == > 1. > + > + Names of table columns included in the publication. This contains all > + the columns of table when user didn't specify column list for the > + table. > + >

Re: bogus: logical replication rows/cols combinations

2022-05-17 Thread Amit Kapila
On Mon, May 16, 2022 at 6:04 PM houzj.f...@fujitsu.com wrote: > > Attach the new version patch. > Few minor comments: == 1. + + Names of table columns included in the publication. This contains all + the columns of table when user didn't specify column list for

RE: bogus: logical replication rows/cols combinations

2022-05-17 Thread osumi.takami...@fujitsu.com
On Monday, May 16, 2022 9:34 PM houzj.f...@fujitsu.com wrote: > Attach the new version patch. Hi, I have few minor comments. For v2-0001. (1) Unnecessary period at the end of column explanation + + + rowfilter text + + + Expression for the table's

Re: bogus: logical replication rows/cols combinations

2022-05-16 Thread Amit Kapila
On Mon, May 16, 2022 at 6:50 PM Alvaro Herrera wrote: > > On 2022-May-16, Amit Kapila wrote: > > > > Agreed. If we can get columnlist and rowfilter from > > > pg_publication_tables, it > > > will be more convenient. And I think users that want to fetch the > > > columnlist > > > and rowfilter

RE: bogus: logical replication rows/cols combinations

2022-05-16 Thread shiy.f...@fujitsu.com
On Mon, May 16, 2022 8:34 PM houzj.f...@fujitsu.com wrote: > > Attach the new version patch. > Thanks for your patch. Here are some comments: 1. (0001 patch) /* * Returns Oids of tables in a publication. */ Datum pg_get_publication_tables(PG_FUNCTION_ARGS) Should we modify the comment of

Re: bogus: logical replication rows/cols combinations

2022-05-16 Thread Alvaro Herrera
On 2022-May-16, Amit Kapila wrote: > > Agreed. If we can get columnlist and rowfilter from pg_publication_tables, > > it > > will be more convenient. And I think users that want to fetch the columnlist > > and rowfilter of table can also benefit from it. > > After the change for this, we will

RE: bogus: logical replication rows/cols combinations

2022-05-16 Thread houzj.f...@fujitsu.com
On Monday, May 16, 2022 2:10 PM Amit Kapila > > On Fri, May 13, 2022 at 11:32 AM houzj.f...@fujitsu.com > wrote: > > > > On Thursday, May 12, 2022 2:45 PM Amit Kapila > wrote: > > > > > > On Wed, May 11, 2022 at 12:55 PM houzj.f...@fujitsu.com > > > wrote: > > > > > > Few comments: > > >

Re: bogus: logical replication rows/cols combinations

2022-05-16 Thread Amit Kapila
On Fri, May 13, 2022 at 11:32 AM houzj.f...@fujitsu.com wrote: > > On Thursday, May 12, 2022 2:45 PM Amit Kapila wrote: > > > > On Wed, May 11, 2022 at 12:55 PM houzj.f...@fujitsu.com > > wrote: > > > > Few comments: > > === > > 1. > > initStringInfo(); > > -

RE: bogus: logical replication rows/cols combinations

2022-05-13 Thread houzj.f...@fujitsu.com
On Thursday, May 12, 2022 2:45 PM Amit Kapila wrote: > > On Wed, May 11, 2022 at 12:55 PM houzj.f...@fujitsu.com > wrote: > > > > On Wednesday, May 11, 2022 11:33 AM Amit Kapila > wrote: > > > > > > Fair enough, then we should go with a simpler approach to detect it > > > in pgoutput.c

Re: bogus: logical replication rows/cols combinations

2022-05-12 Thread Amit Kapila
On Thu, May 12, 2022 at 12:15 PM Amit Kapila wrote: > > On Wed, May 11, 2022 at 12:55 PM houzj.f...@fujitsu.com > wrote: > > > > On Wednesday, May 11, 2022 11:33 AM Amit Kapila > > wrote: > > > > > > Fair enough, then we should go with a simpler approach to detect it in > > > pgoutput.c

Re: bogus: logical replication rows/cols combinations

2022-05-12 Thread Amit Kapila
On Wed, May 11, 2022 at 12:55 PM houzj.f...@fujitsu.com wrote: > > On Wednesday, May 11, 2022 11:33 AM Amit Kapila > wrote: > > > > Fair enough, then we should go with a simpler approach to detect it in > > pgoutput.c (get_rel_sync_entry). > > OK, here is the patch that try to check column list

RE: bogus: logical replication rows/cols combinations

2022-05-11 Thread houzj.f...@fujitsu.com
On Wednesday, May 11, 2022 11:33 AM Amit Kapila wrote: > On Wed, May 11, 2022 at 12:35 AM Tomas Vondra > wrote: > > > > On 5/9/22 05:45, Amit Kapila wrote: > > > On Sun, May 8, 2022 at 11:41 PM Tomas Vondra > > > wrote: > > >> > > >> On 5/7/22 07:36, Amit Kapila wrote: > > >>> On Mon, May 2,

Re: bogus: logical replication rows/cols combinations

2022-05-10 Thread Amit Kapila
On Wed, May 11, 2022 at 12:35 AM Tomas Vondra wrote: > > On 5/9/22 05:45, Amit Kapila wrote: > > On Sun, May 8, 2022 at 11:41 PM Tomas Vondra > > wrote: > >> > >> On 5/7/22 07:36, Amit Kapila wrote: > >>> On Mon, May 2, 2022 at 6:11 PM Alvaro Herrera > >>> wrote: > > On 2022-May-02,

Re: bogus: logical replication rows/cols combinations

2022-05-10 Thread Tomas Vondra
On 5/9/22 05:45, Amit Kapila wrote: > On Sun, May 8, 2022 at 11:41 PM Tomas Vondra > wrote: >> >> On 5/7/22 07:36, Amit Kapila wrote: >>> On Mon, May 2, 2022 at 6:11 PM Alvaro Herrera >>> wrote: On 2022-May-02, Amit Kapila wrote: > I think it is possible to expose a

Re: bogus: logical replication rows/cols combinations

2022-05-08 Thread Amit Kapila
On Sun, May 8, 2022 at 11:41 PM Tomas Vondra wrote: > > On 5/7/22 07:36, Amit Kapila wrote: > > On Mon, May 2, 2022 at 6:11 PM Alvaro Herrera > > wrote: > >> > >> On 2022-May-02, Amit Kapila wrote: > >> > >> > >>> I think it is possible to expose a list of publications for each > >>> walsender

Re: bogus: logical replication rows/cols combinations

2022-05-08 Thread Tomas Vondra
On 5/7/22 07:36, Amit Kapila wrote: > On Mon, May 2, 2022 at 6:11 PM Alvaro Herrera wrote: >> >> On 2022-May-02, Amit Kapila wrote: >> >> >>> I think it is possible to expose a list of publications for each >>> walsender as it is stored in each walsenders >>>

Re: bogus: logical replication rows/cols combinations

2022-05-06 Thread Amit Kapila
On Mon, May 2, 2022 at 6:11 PM Alvaro Herrera wrote: > > On 2022-May-02, Amit Kapila wrote: > > > > I think it is possible to expose a list of publications for each > > walsender as it is stored in each walsenders > > LogicalDecodingContext->output_plugin_private. AFAIK, each walsender > > can

Re: bogus: logical replication rows/cols combinations

2022-05-06 Thread Tomas Vondra
On 5/6/22 15:40, Amit Kapila wrote: > On Fri, May 6, 2022 at 5:56 PM Tomas Vondra > wrote: >> I could think of below two options: 1. Forbid any case where column list is different for the same table when combining publications. 2. Forbid if the column list and row

Re: bogus: logical replication rows/cols combinations

2022-05-06 Thread Amit Kapila
On Fri, May 6, 2022 at 5:56 PM Tomas Vondra wrote: > > >> > >> I could think of below two options: > >> 1. Forbid any case where column list is different for the same table > >> when combining publications. > >> 2. Forbid if the column list and row filters for a table are different > >> in the

Re: bogus: logical replication rows/cols combinations

2022-05-06 Thread Tomas Vondra
On 5/6/22 05:23, houzj.f...@fujitsu.com wrote: > On Tuesday, May 3, 2022 11:31 AM Amit Kapila wrote: >> >> On Tue, May 3, 2022 at 12:10 AM Tomas Vondra >> wrote: >>> >>> On 5/2/22 19:51, Alvaro Herrera wrote: > Why would we need to know publications replicated by other >> walsenders?

RE: bogus: logical replication rows/cols combinations

2022-05-05 Thread houzj.f...@fujitsu.com
On Tuesday, May 3, 2022 11:31 AM Amit Kapila wrote: > > On Tue, May 3, 2022 at 12:10 AM Tomas Vondra > wrote: > > > > On 5/2/22 19:51, Alvaro Herrera wrote: > > >> Why would we need to know publications replicated by other > walsenders? > > >> And what if the subscriber is not connected at the

Re: bogus: logical replication rows/cols combinations

2022-05-04 Thread Peter Eisentraut
On 03.05.22 21:40, Tomas Vondra wrote: So what's wrong with merging the column lists as implemented in the v2 patch, posted a couple days ago? Merging the column lists is ok if all other publication attributes match. Otherwise, I think not. I don't think triggers are a suitable

Re: bogus: logical replication rows/cols combinations

2022-05-03 Thread Tomas Vondra
On 5/2/22 22:34, Peter Eisentraut wrote: > On 01.05.22 23:42, Tomas Vondra wrote: >> Imagine have a table with customers from different regions, and you want >> to replicate the data somewhere else, but for some reason you can only >> replicate details for one particular region, and subset of

Re: bogus: logical replication rows/cols combinations

2022-05-02 Thread Amit Kapila
On Mon, May 2, 2022 at 6:11 PM Alvaro Herrera wrote: > > On 2022-May-02, Amit Kapila wrote: > > > I think it is possible to expose a list of publications for each > > walsender as it is stored in each walsenders > > LogicalDecodingContext->output_plugin_private. AFAIK, each walsender > > can have

Re: bogus: logical replication rows/cols combinations

2022-05-02 Thread Amit Kapila
On Tue, May 3, 2022 at 12:10 AM Tomas Vondra wrote: > > On 5/2/22 19:51, Alvaro Herrera wrote: > >> Why would we need to know publications replicated by other walsenders? > >> And what if the subscriber is not connected at the moment? In that case > >> there'll be no walsender. > > > > Sure, if

Re: bogus: logical replication rows/cols combinations

2022-05-02 Thread Peter Eisentraut
On 01.05.22 23:42, Tomas Vondra wrote: Imagine have a table with customers from different regions, and you want to replicate the data somewhere else, but for some reason you can only replicate details for one particular region, and subset of columns for everyone else. So you'd do something like

Re: bogus: logical replication rows/cols combinations

2022-05-02 Thread Tomas Vondra
On 5/2/22 19:51, Alvaro Herrera wrote: > On 2022-May-02, Tomas Vondra wrote: > >> pgoutput.c is relies on relcache callbacks to get notified of changes. >> See the stuff that touches replicate_valid and publications_valid. So >> the walsender should notice the changes immediately. > > Hmm, I

Re: bogus: logical replication rows/cols combinations

2022-05-02 Thread Tomas Vondra
On 5/2/22 13:23, Amit Kapila wrote: > On Mon, May 2, 2022 at 3:05 PM Tomas Vondra > wrote: >> >> On 5/2/22 07:31, Amit Kapila wrote: >>> On Mon, May 2, 2022 at 3:27 AM Tomas Vondra >>> wrote: >> The second option has the annoying consequence that it makes this useless for the

Re: bogus: logical replication rows/cols combinations

2022-05-02 Thread Alvaro Herrera
On 2022-May-02, Tomas Vondra wrote: > pgoutput.c is relies on relcache callbacks to get notified of changes. > See the stuff that touches replicate_valid and publications_valid. So > the walsender should notice the changes immediately. Hmm, I suppose that makes any changes easy enough to detect.

Re: bogus: logical replication rows/cols combinations

2022-05-02 Thread Tomas Vondra
On 5/2/22 13:44, Alvaro Herrera wrote: > On 2022-May-02, Amit Kapila wrote: > >> We don't do that currently but we can as mentioned in my previous >> email [1]. Let me write the relevant part again. We need to expose all >> publications for a walsender, and then we can find the exact set of >>

Re: bogus: logical replication rows/cols combinations

2022-05-02 Thread Alvaro Herrera
On 2022-Apr-28, Tomas Vondra wrote: > SELECT > (CASE WHEN (a < 0) OR (a > 0) THEN a ELSE NULL END) AS a, > (CASE WHEN (a > 0) THEN b ELSE NULL END) AS b, > (CASE WHEN (a < 0) THEN c ELSE NULL END) AS c > FROM uno WHERE (a < 0) OR (a > 0) BTW, looking at the new COPY commands, the

Re: bogus: logical replication rows/cols combinations

2022-05-02 Thread Alvaro Herrera
On 2022-May-02, Amit Kapila wrote: > We don't do that currently but we can as mentioned in my previous > email [1]. Let me write the relevant part again. We need to expose all > publications for a walsender, and then we can find the exact set of > publications where the current publication is

Re: bogus: logical replication rows/cols combinations

2022-05-02 Thread Amit Kapila
On Mon, May 2, 2022 at 3:05 PM Tomas Vondra wrote: > > On 5/2/22 07:31, Amit Kapila wrote: > > On Mon, May 2, 2022 at 3:27 AM Tomas Vondra > > wrote: > >> > > >> The second option has the annoying consequence that it makes this > >> useless for the "data redaction" use case I described in [2],

Re: bogus: logical replication rows/cols combinations

2022-05-02 Thread Amit Kapila
On Mon, May 2, 2022 at 3:53 PM Tomas Vondra wrote: > > On 5/2/22 12:17, Alvaro Herrera wrote: > > On 2022-May-02, Tomas Vondra wrote: > >> On 5/2/22 07:31, Amit Kapila wrote: > > > >>> Yeah, or don't allow to define such publications in the first place so > >>> that different subscriptions can't

Re: bogus: logical replication rows/cols combinations

2022-05-02 Thread Alvaro Herrera
On 2022-May-02, Tomas Vondra wrote: > On 5/2/22 12:17, Alvaro Herrera wrote: > > The latter ultimately means that we aren't sure that a combined > > subscription is safe. And in turn this means that a pg_dump of such a > > database cannot be restored (because the CREATE SUBSCRIPTION will be > >

Re: bogus: logical replication rows/cols combinations

2022-05-02 Thread Tomas Vondra
On 5/2/22 12:17, Alvaro Herrera wrote: > On 2022-May-02, Tomas Vondra wrote: >> On 5/2/22 07:31, Amit Kapila wrote: > >>> Yeah, or don't allow to define such publications in the first place so >>> that different subscriptions can't combine them but I guess that might >>> forbid some useful

Re: bogus: logical replication rows/cols combinations

2022-05-02 Thread Alvaro Herrera
On 2022-May-02, Tomas Vondra wrote: > On 5/2/22 07:31, Amit Kapila wrote: > > Yeah, or don't allow to define such publications in the first place so > > that different subscriptions can't combine them but I guess that might > > forbid some useful cases as well where publication may not get > >

Re: bogus: logical replication rows/cols combinations

2022-05-02 Thread Tomas Vondra
On 5/2/22 07:31, Amit Kapila wrote: > On Mon, May 2, 2022 at 3:27 AM Tomas Vondra > wrote: >> >> On 4/30/22 12:11, Amit Kapila wrote: >>> On Sat, Apr 30, 2022 at 3:01 PM Alvaro Herrera >>> wrote: My proposal is that if users want to define multiple publications, and their

Re: bogus: logical replication rows/cols combinations

2022-05-01 Thread Amit Kapila
On Mon, May 2, 2022 at 11:01 AM Amit Kapila wrote: > > On Mon, May 2, 2022 at 3:27 AM Tomas Vondra > wrote: > > > > On 4/30/22 12:11, Amit Kapila wrote: > > > On Sat, Apr 30, 2022 at 3:01 PM Alvaro Herrera > > > wrote: > > >> > > >> My proposal is that if users want to define multiple

Re: bogus: logical replication rows/cols combinations

2022-05-01 Thread Amit Kapila
On Mon, May 2, 2022 at 3:27 AM Tomas Vondra wrote: > > On 4/30/22 12:11, Amit Kapila wrote: > > On Sat, Apr 30, 2022 at 3:01 PM Alvaro Herrera > > wrote: > >> > >> My proposal is that if users want to define multiple publications, and > >> their definitions conflict in a way that would behave

Re: bogus: logical replication rows/cols combinations

2022-05-01 Thread Tomas Vondra
On 4/30/22 12:11, Amit Kapila wrote: > On Sat, Apr 30, 2022 at 3:01 PM Alvaro Herrera > wrote: >> >> On 2022-Apr-30, Amit Kapila wrote: >> >>> On Sat, Apr 30, 2022 at 2:02 AM Tomas Vondra >>> wrote: >> That seems to deal with a circular replication, i.e. two logical replication links

Re: bogus: logical replication rows/cols combinations

2022-05-01 Thread Tomas Vondra
On 4/29/22 07:05, Amit Kapila wrote: > On Thu, Apr 28, 2022 at 5:56 PM Peter Eisentraut > wrote: >> >> On 27.04.22 12:33, Amit Kapila wrote: >>> Currently, when the subscription has multiple publications, we combine >>> the objects, and actions of those publications. It happens for >>>

Re: bogus: logical replication rows/cols combinations

2022-05-01 Thread Tomas Vondra
On 4/30/22 11:28, Alvaro Herrera wrote: > On 2022-Apr-28, Tomas Vondra wrote: > >> Attached is a patch doing the same thing in tablesync. The overall idea >> is to generate copy statement with CASE expressions, applying filters to >> individual columns. For Alvaro's example, this generates

Re: bogus: logical replication rows/cols combinations

2022-04-30 Thread Alvaro Herrera
On 2022-Apr-30, Amit Kapila wrote: > I agree with throwing errors for obvious/known bogus cases but do we > want to throw errors or restrict the combining of column lists when > row filters are present in all cases? See some examples [1 ] where it > may be valid to combine them. I agree we

Re: bogus: logical replication rows/cols combinations

2022-04-30 Thread Amit Kapila
On Sat, Apr 30, 2022 at 3:01 PM Alvaro Herrera wrote: > > On 2022-Apr-30, Amit Kapila wrote: > > > On Sat, Apr 30, 2022 at 2:02 AM Tomas Vondra > > wrote: > > > > That seems to deal with a circular replication, i.e. two logical > > > replication links - a bit like a multi-master. Not sure how is

Re: bogus: logical replication rows/cols combinations

2022-04-30 Thread Alvaro Herrera
On 2022-Apr-30, Amit Kapila wrote: > On Sat, Apr 30, 2022 at 2:02 AM Tomas Vondra > wrote: > > That seems to deal with a circular replication, i.e. two logical > > replication links - a bit like a multi-master. Not sure how is that > > related to the issue we're discussing here? > > It is not

Re: bogus: logical replication rows/cols combinations

2022-04-30 Thread Alvaro Herrera
On 2022-Apr-28, Tomas Vondra wrote: > Attached is a patch doing the same thing in tablesync. The overall idea > is to generate copy statement with CASE expressions, applying filters to > individual columns. For Alvaro's example, this generates something like > > SELECT > (CASE WHEN (a < 0)

Re: bogus: logical replication rows/cols combinations

2022-04-29 Thread Amit Kapila
On Sat, Apr 30, 2022 at 2:02 AM Tomas Vondra wrote: > > On 4/29/22 06:48, Amit Kapila wrote: > > On Thu, Apr 28, 2022 at 11:00 PM Tomas Vondra > > I think such issues due to ALTER of the publication are somewhat > expected, and I think users will understand they might need to resync > the

Re: bogus: logical replication rows/cols combinations

2022-04-29 Thread Tomas Vondra
On 4/29/22 06:48, Amit Kapila wrote: > On Thu, Apr 28, 2022 at 11:00 PM Tomas Vondra > wrote: >> >> On 4/28/22 14:26, Peter Eisentraut wrote: >>> On 27.04.22 12:33, Amit Kapila wrote: >>> >>> I wonder how we handle the combination of >>> >>> pub1: publish=insert WHERE (foo) >>> pub2:

Re: bogus: logical replication rows/cols combinations

2022-04-28 Thread Amit Kapila
On Thu, Apr 28, 2022 at 5:56 PM Peter Eisentraut wrote: > > On 27.04.22 12:33, Amit Kapila wrote: > > Currently, when the subscription has multiple publications, we combine > > the objects, and actions of those publications. It happens for > > 'publish_via_partition_root', publication actions,

Re: bogus: logical replication rows/cols combinations

2022-04-28 Thread Amit Kapila
On Thu, Apr 28, 2022 at 11:00 PM Tomas Vondra wrote: > > On 4/28/22 14:26, Peter Eisentraut wrote: > > On 27.04.22 12:33, Amit Kapila wrote: > > > > I wonder how we handle the combination of > > > > pub1: publish=insert WHERE (foo) > > pub2: publish=update WHERE (bar) > > > > I think it would be

Re: bogus: logical replication rows/cols combinations

2022-04-28 Thread Tomas Vondra
On 4/28/22 14:26, Peter Eisentraut wrote: > On 27.04.22 12:33, Amit Kapila wrote: >> Currently, when the subscription has multiple publications, we combine >> the objects, and actions of those publications. It happens for >> 'publish_via_partition_root', publication actions, tables, column >>

Re: bogus: logical replication rows/cols combinations

2022-04-28 Thread Tomas Vondra
On 4/28/22 05:17, Amit Kapila wrote: > On Thu, Apr 28, 2022 at 3:26 AM Tomas Vondra > wrote: >> >> so I've been looking at tweaking the code so that the behavior matches >> Alvaro's expectations. It passes check-world but I'm not claiming it's >> nowhere near commitable - the purpose is mostly

Re: bogus: logical replication rows/cols combinations

2022-04-28 Thread Peter Eisentraut
On 27.04.22 12:33, Amit Kapila wrote: Currently, when the subscription has multiple publications, we combine the objects, and actions of those publications. It happens for 'publish_via_partition_root', publication actions, tables, column lists, or row filters. I think the whole design works on

Re: bogus: logical replication rows/cols combinations

2022-04-28 Thread Peter Eisentraut
On 27.04.22 11:53, Alvaro Herrera wrote: Now, another possibility is to say "naah, this is too hard", or even "naah, there's no time to write all that for this release". That might be okay, but in that case let's add an implementation restriction to ensure that we don't paint ourselves in a

Re: bogus: logical replication rows/cols combinations

2022-04-27 Thread Amit Kapila
On Thu, Apr 28, 2022 at 3:26 AM Tomas Vondra wrote: > > so I've been looking at tweaking the code so that the behavior matches > Alvaro's expectations. It passes check-world but I'm not claiming it's > nowhere near commitable - the purpose is mostly to give better idea of > how invasive the

Re: bogus: logical replication rows/cols combinations

2022-04-27 Thread Tomas Vondra
Hi, so I've been looking at tweaking the code so that the behavior matches Alvaro's expectations. It passes check-world but I'm not claiming it's nowhere near commitable - the purpose is mostly to give better idea of how invasive the change is etc. As described earlier, this abandons the idea of

Re: bogus: logical replication rows/cols combinations

2022-04-27 Thread Amit Kapila
On Wed, Apr 27, 2022 at 4:27 PM Alvaro Herrera wrote: > > On 2022-Apr-27, Amit Kapila wrote: > > > On Wed, Apr 27, 2022 at 3:13 PM Alvaro Herrera > > wrote: > > > > > Changing this to behave the way you expect would be quite difficult, > > > > because at the moment we build a single OR

Re: bogus: logical replication rows/cols combinations

2022-04-27 Thread Alvaro Herrera
On 2022-Apr-27, Amit Kapila wrote: > On Wed, Apr 27, 2022 at 3:13 PM Alvaro Herrera > wrote: > > > Changing this to behave the way you expect would be quite difficult, > > > because at the moment we build a single OR expression from all the row > > > filters. We'd have to keep the individual

Re: bogus: logical replication rows/cols combinations

2022-04-27 Thread Amit Kapila
On Wed, Apr 27, 2022 at 3:13 PM Alvaro Herrera wrote: > > On 2022-Apr-26, Tomas Vondra wrote: > > > I'm not quite sure which of the two behaviors is more "desirable". In a > > way, it's somewhat similar to publish_as_relid, which is also calculated > > not considering which of the row filters

RE: bogus: logical replication rows/cols combinations

2022-04-27 Thread houzj.f...@fujitsu.com
On Wednesday, April 27, 2022 12:56 PM From: Amit Kapila wrote: > On Tue, Apr 26, 2022 at 4:00 AM Tomas Vondra > wrote: > > > > On 4/25/22 17:48, Alvaro Herrera wrote: > > > > > The desired result on subscriber is: > > > > > > table uno; > > > a │ b │ c > > > ┼───┼─── > > > 1 │ 2 │ > > >

Re: bogus: logical replication rows/cols combinations

2022-04-27 Thread Alvaro Herrera
On 2022-Apr-27, Amit Kapila wrote: > On Tue, Apr 26, 2022 at 4:00 AM Tomas Vondra > wrote: > > I can take a stab at it, but it seems strange to not apply the same > > logic to evaluation of publish_as_relid. > > Yeah, the current behavior seems to be consistent with what we already > do.

Re: bogus: logical replication rows/cols combinations

2022-04-27 Thread Alvaro Herrera
On 2022-Apr-26, Tomas Vondra wrote: > I'm not quite sure which of the two behaviors is more "desirable". In a > way, it's somewhat similar to publish_as_relid, which is also calculated > not considering which of the row filters match? I grepped doc/src/sgml for `publish_as_relid` and found no

Re: bogus: logical replication rows/cols combinations

2022-04-27 Thread Michael Paquier
On Wed, Apr 27, 2022 at 10:25:50AM +0530, Amit Kapila wrote: > I feel we can explain a bit more about this in docs. We already have > some explanation of how row filters are combined [1]. We can probably > add a few examples for column lists. I am not completely sure exactly what we should do

Re: bogus: logical replication rows/cols combinations

2022-04-26 Thread Amit Kapila
On Tue, Apr 26, 2022 at 4:00 AM Tomas Vondra wrote: > > On 4/25/22 17:48, Alvaro Herrera wrote: > > > The desired result on subscriber is: > > > > table uno; > > a │ b │ c > > ┼───┼─── > > 1 │ 2 │ > > -1 │ │ 4 > > > > > > Thoughts? > > > > I'm not quite sure which of the two behaviors

Re: bogus: logical replication rows/cols combinations

2022-04-25 Thread Tomas Vondra
On 4/25/22 17:48, Alvaro Herrera wrote: > I just noticed that publishing tables on multiple publications with > different row filters and column lists has somewhat surprising behavior. > To wit: if a column is published in any row-filtered publication, then > the values for that column are sent to

bogus: logical replication rows/cols combinations

2022-04-25 Thread Alvaro Herrera
I just noticed that publishing tables on multiple publications with different row filters and column lists has somewhat surprising behavior. To wit: if a column is published in any row-filtered publication, then the values for that column are sent to the subscriber even for rows that don't match