Re: [PATCH] fix a performance issue with multiple logical-decoding walsenders

2020-01-06 Thread Tom Lane
Julien Rouhaud writes: > On Mon, Jan 6, 2020 at 7:57 PM Pierre Ducroquet wrote: >> My deepest apologies for the patch being broken, I messed up when >> transferring >> it between my computers after altering the comments. The verbatim one >> attached >> to this email applies with no issue on

Re: [PATCH] fix a performance issue with multiple logical-decoding walsenders

2020-01-06 Thread Marc Cousin
Hi, I spent a little bit of time trying to explain the problem we are facing clearly, and provide a reproducible benchmark. So here it is. What triggered our investigation is that we have a PostgreSQL cluster containing about 15 databases, most of them being used as sources for logical

Re: [PATCH] fix a performance issue with multiple logical-decoding walsenders

2020-01-06 Thread Julien Rouhaud
On Mon, Jan 6, 2020 at 7:57 PM Pierre Ducroquet wrote: > > On Monday, January 6, 2020 6:57:33 PM CET Tom Lane wrote: > > Pierre Ducroquet writes: > > > Attached to this email is a patch with better comments regarding the > > > XLogSendLogical change. > > > > Hi, > > This patch entirely fails

Re: [PATCH] fix a performance issue with multiple logical-decoding walsenders

2020-01-06 Thread Pierre Ducroquet
On Monday, January 6, 2020 6:57:33 PM CET Tom Lane wrote: > Pierre Ducroquet writes: > > Attached to this email is a patch with better comments regarding the > > XLogSendLogical change. > > Hi, > This patch entirely fails to apply for me (and for the cfbot too). > It looks like (a) it's

Re: [PATCH] fix a performance issue with multiple logical-decoding walsenders

2020-01-06 Thread Tom Lane
Pierre Ducroquet writes: > Attached to this email is a patch with better comments regarding the > XLogSendLogical change. Hi, This patch entirely fails to apply for me (and for the cfbot too). It looks like (a) it's missing a final newline and (b) all the tabs have been mangled into spaces,

Re: [PATCH] fix a performance issue with multiple logical-decoding walsenders

2019-12-30 Thread Pierre Ducroquet
On Sunday, December 29, 2019 1:32:31 PM CET Julien Rouhaud wrote: > On Sat, Dec 28, 2019 at 1:56 PM Pierre Ducroquet wrote: > > Thank you for your comments. > > Attached to this email is a patch with better comments regarding the > > XLogSendLogical change. > > Arguably the first test to

Re: [PATCH] fix a performance issue with multiple logical-decoding walsenders

2019-12-29 Thread Julien Rouhaud
On Sat, Dec 28, 2019 at 1:56 PM Pierre Ducroquet wrote: > > Thank you for your comments. > Attached to this email is a patch with better comments regarding the > XLogSendLogical change. Arguably the first test to compare to InvalidXLogRecPtr is unneeded, as any value of EndRecPtr is greater or

Re: [PATCH] fix a performance issue with multiple logical-decoding walsenders

2019-12-28 Thread Pierre Ducroquet
On Thursday, December 26, 2019 8:18:46 PM CET Julien Rouhaud wrote: > Hello Pierre, > > On Thu, Dec 26, 2019 at 5:43 PM Pierre Ducroquet wrote: > > The second one was tested on PG 10 and PG 12 (with 48 lines offset). It > > has on PG12 the same effect it has on a PG10+isAlive patch. Instead of

Re: [PATCH] fix a performance issue with multiple logical-decoding walsenders

2019-12-26 Thread Julien Rouhaud
Hello Pierre, On Thu, Dec 26, 2019 at 5:43 PM Pierre Ducroquet wrote: > > The second one was tested on PG 10 and PG 12 (with 48 lines offset). It has on > PG12 the same effect it has on a PG10+isAlive patch. Instead of calling each > time GetFlushRecPtr, we call it only if we notice we have

[PATCH] fix a performance issue with multiple logical-decoding walsenders

2019-12-26 Thread Pierre Ducroquet
Hello Our current setup uses logical replication to build a BI replication server along our primary clusters (running PG 10.10 so far). This implies having one logical replication slot per database. After some analysis, we identified two hot spots behind this issue. Fixing them gave us a 10