Re: [PATCH] Improve performance of NOTIFY over many databases (issue blocking on AccessExclusiveLock on object 0 of class 1262 of database 0)

2019-06-05 Thread Martijn van Oosterhout
the actual optimisation here is that the async queue tail pointer is only updated once per SLRU page instead of every message. This would require a significantly larger patch, but shouldn't be too difficult. Thoughts? Have a nice day, Martijn On Tue, 4 Jun 2019 at 09:08, Martijn van Oosterhout wrote

[PATCH] Improve performance of NOTIFY over many databases (issue blocking on AccessExclusiveLock on object 0 of class 1262 of database 0)

2019-06-04 Thread Martijn van Oosterhout
/cadwg95t0j9zf0uwdcmh81kmndsitavhxmbvgyqrrjcd-ilw...@mail.gmail.com -- Martijn van Oosterhout http://svana.org/kleptog/ From 17c241a2e307e70465c235248c17ac70d34fd175 Mon Sep 17 00:00:00 2001 From: Martijn van Oosterhout Date: Mon, 3 Jun 2019 17:13:31 +0200 Subject: [PATCH 1/2] Only try advancing tail

[PATCH] Improve performance of NOTIFY over many databases (v2)

2019-08-02 Thread Martijn van Oosterhout
16:42:48.683 [9991] martijn@test_042 DEBUG: PreCommit_Notify 16:42:48.683 [9991] martijn@test_042 DEBUG: NOTIFY QUEUE = (75,7744)...(79,32) 16:42:48.683 [9991] martijn@test_042 DEBUG: AtCommit_Notify - Have a nice weekend. -- Martijn van Oosterhout http://svana.org/kleptog

Re: [PATCH] Improve performance of NOTIFY over many databases (v2)

2019-09-14 Thread Martijn van Oosterhout
afterwards whether it's slow enough to merit a kick. Adjusted this. I'm not sure it's actually clearer this way, but it is less work inside the loop. A small change is that now it won't signal anyone if this backend is the slowest, which more correct. Thanks for the feedback. Attached is versi

Re: [PATCH] Improve performance of NOTIFY over many databases (v2)

2019-09-15 Thread Martijn van Oosterhout
On Sat, 14 Sep 2019 at 17:08, Tom Lane wrote: > Martijn van Oosterhout writes: > > On Fri, 13 Sep 2019 at 22:04, Tom Lane wrote: > >> But, really ... do we need the backendTryAdvanceTail flag at all? > None of this seems to respond to my point: it looks to me like it wou

Re: [PATCH] Improve performance of NOTIFY over many databases (v2)

2019-09-17 Thread Martijn van Oosterhout
Hoi Tom, On Mon, 16 Sep 2019 at 15:33, Tom Lane wrote: > > Martijn van Oosterhout writes: > > I think I like the idea of having SignalBackend do the waking up a > > slow backend but I'm not enthused by the "lets wake up (at once) > > everyone that is behind

Re: [PATCH] Improve performance of NOTIFY over many databases (v2)

2019-09-16 Thread Martijn van Oosterhout
r to catch up, thus holding the lock longer. It's not worth making it configurable so we have to guess, but 16 is perhaps a good compromise. Have a nice day, -- Martijn van Oosterhout http://svana.org/kleptog/

Re: [PATCH] Improve performance of NOTIFY over many databases (v2)

2019-09-11 Thread Martijn van Oosterhout
002 is now going to need a rebase, so please do that. > > Thanks for this, and good catch. Looks like I didn't test the first patch by itself very well. Here is the rebased second patch. Thanks in advance, -- Martijn van Oosterhout http://svana.org/kleptog/ From bc4b1b458564f758b7fa1c1f7b

Re: [PATCH] Improve performance of NOTIFY over many databases (issue blocking on AccessExclusiveLock on object 0 of class 1262 of database 0)

2019-07-23 Thread Martijn van Oosterhout
, or even thousands of messages. Do 2 & 3 seem like a good direction to go? I can probably work something up. Thanks in advance, Martijn > Martijn van Oosterhout writes: > > Please find attached updated versions of the patches, I've now tested > > them. Also attached is a

Re: [PATCH] Improve performance of NOTIFY over many databases (issue blocking on AccessExclusiveLock on object 0 of class 1262 of database 0)

2019-07-23 Thread Martijn van Oosterhout
On Tue, 23 Jul 2019 at 19:21, Tom Lane wrote: > > Martijn van Oosterhout writes: > > 2. Add a field to AsyncQueueEntry which points to the next listening > > backend. This would allow the loops over all listening backends to > > complete much faster, especially

Re: [PATCH] Improve performance of NOTIFY over many databases (issue blocking on AccessExclusiveLock on object 0 of class 1262 of database 0)

2019-07-24 Thread Martijn van Oosterhout
On Tue, 23 Jul 2019 at 23:32, Tom Lane wrote: > > Martijn van Oosterhout writes: > > I mean tracking the listening backends specifically, so you can > > replace the loops: > > for (i=0; i < MaxBackends; i++) > > with > > for

Re: LISTEN/NOTIFY testing woes

2019-11-24 Thread Martijn van Oosterhout
one that makes the made changes invisible. Arguably not doing this means we'd have to document the values are possibly out of date. So I think this patch does the right thing. Have a nice day, -- Martijn van Oosterhout http://svana.org/kleptog/

IDEA: pg_stat_statements tracking utility statements by tag?

2020-07-29 Thread Martijn van Oosterhout
ively. But it would also catch DDL. Does this sound like something for which a patch would be accepted? Have a nice day, -- Martijn van Oosterhout http://svana.org/kleptog/

Re: IDEA: pg_stat_statements tracking utility statements by tag?

2020-07-29 Thread Martijn van Oosterhout
e query. That said, I think "aggregate by tag" has value all by itself. Being able to collapse all CREATE TABLES into a single line can be useful in some situations. Ideally the results of FETCH "cursor" should be combined with the DECLARE, but I really don't know how to go abo