Hello
On 2021-Apr-13, Andres Freund wrote:
> > The concerns that had come to my mind were more along the lines
> > of things like pg_dump requiring a larger footprint in the shared
> > lock table. We could alleviate that by increasing the default
> > value of max_locks_per_transaction, perhaps.
Hi,
On 2021-04-13 23:04:50 -0400, Tom Lane wrote:
> Andres Freund writes:
> > On 2021-04-13 19:16:46 -0400, Tom Lane wrote:
> >> Like this? This passes check-world, modulo the one very-unsurprising
> >> regression test change. I've not tried to do any performance testing.
>
> > I wonder if
Andres Freund writes:
> On 2021-04-13 19:16:46 -0400, Tom Lane wrote:
>> Like this? This passes check-world, modulo the one very-unsurprising
>> regression test change. I've not tried to do any performance testing.
> I wonder if there's a realistic chance it could create additional
> deadlocks
Hi,
On 2021-04-13 19:16:46 -0400, Tom Lane wrote:
> > Maybe the same thing we do with user tables, ie not give up the lock
> > when we close a toast rel? As long as the internal lock counters
> > are 64-bit, we'd not have to worry about overflowing them.
Well, I was assuming we'd not want to do
I wrote:
> Andres Freund writes:
>> Cool. And damn: I can't immediately think of a way to optimize this to
>> not require this kind of hack in the future.
> Maybe the same thing we do with user tables, ie not give up the lock
> when we close a toast rel? As long as the internal lock counters
>
: paul.fried...@streetlightdata.com
-Original Message-
From: Andres Freund
Sent: Tuesday, April 13, 2021 1:48 PM
To: Paul Friedman
Cc: pgsql-performance@lists.postgresql.org
Subject: Re: LWLocks by LockManager slowing large DB
Hi,
On 2021-04-13 11:47:06 -0700, Paul Friedman wrote:
> YES!!! This comple
Andres Freund writes:
> On 2021-04-13 11:47:06 -0700, Paul Friedman wrote:
>> YES!!! This completely alleviates the bottleneck and I'm able to run the
>> queries full-throttle. Thank you SO much for your help+insight.
> Cool. And damn: I can't immediately think of a way to optimize this to
>
On 2021-Apr-13, Andres Freund wrote:
> > Sounds like I should file this as a requested improvement?
>
> The ability to lock a toast table? Yea, it might be worth doing that. I
> seem to recall this being discussed not too long ago...
Yep, commit 59ab4ac32460 reverted by eeda7f633809. There
Hi,
On 2021-04-13 11:47:06 -0700, Paul Friedman wrote:
> YES!!! This completely alleviates the bottleneck and I'm able to run the
> queries full-throttle. Thank you SO much for your help+insight.
Cool. And damn: I can't immediately think of a way to optimize this to
not require this kind of
reund
Sent: Tuesday, April 13, 2021 11:17 AM
To: Paul Friedman
Cc: pgsql-performance@lists.postgresql.org
Subject: Re: LWLocks by LockManager slowing large DB
Hi,
On 2021-04-13 09:33:48 -0700, Paul Friedman wrote:
> I've attached the 2 perf reports. From the 2nd one, I can see lots of
>
Hi,
On 2021-04-13 09:33:48 -0700, Paul Friedman wrote:
> I've attached the 2 perf reports. From the 2nd one, I can see lots of
> time waiting for TOAST table locks on the geometry column, but I
> definitely don't fully understand the implications or why LockManager
> would be struggling here.
...@streetlightdata.com
*From:* Nikolay Samokhvalov
*Sent:* Monday, April 12, 2021 4:34 PM
*To:* Andres Freund
*Cc:* Paul Friedman ;
pgsql-performance@lists.postgresql.org
*Subject:* Re: LWLocks by LockManager slowing large DB
On Mon, Apr 12, 2021 at 14:57 Andres Freund wrote:
Without knowing
: LWLocks by LockManager slowing large DB
Hi,
On 2021-04-12 15:56:08 -0700, Paul Friedman wrote:
> Also, I didn't understand your comment about a 'futex profile', could
> you point me in the right direction here?
My earlier mail included a section about running a perf profile showing
the c
On Mon, Apr 12, 2021 at 14:57 Andres Freund wrote:
> Without knowing the proportion of LockManager wait events compared to
> the rest it's hard to know what to make of it.
These OSS tools can be useful to understand the proportion:
- pgCenter
https://github.com/lesovsky/pgcenter
-
Hi,
On 2021-04-12 15:56:08 -0700, Paul Friedman wrote:
> Also, I didn't understand your comment about a 'futex profile', could you
> point me in the right direction here?
My earlier mail included a section about running a perf profile showing
the callers of the futex() system call, which in turn
Friedman
CTO
677 Harrison St | San Francisco, CA 94107
M: (650) 270-7676
E-mail: paul.fried...@streetlightdata.com
-Original Message-
From: Andres Freund
Sent: Monday, April 12, 2021 3:22 PM
To: Paul Friedman
Cc: pgsql-performance@lists.postgresql.org
Subject: Re: LWLocks by LockManager
Hi,
On 2021-04-12 15:15:05 -0700, Paul Friedman wrote:
> Thanks again for any advice you have.
I think we'd need the perf profiles to be able to dig into this
further.
It's odd that there are a meaningful amount of LockManager contention in
your case - assuming the stats you collected weren't
) 270-7676
E-mail: paul.fried...@streetlightdata.com
-Original Message-
From: Andres Freund
Sent: Monday, April 12, 2021 2:58 PM
To: Paul Friedman
Cc: pgsql-performance@lists.postgresql.org
Subject: Re: LWLocks by LockManager slowing large DB
Hi,
On 2021-04-12 12:37:42 -0700
Hi,
On 2021-04-12 12:37:42 -0700, Paul Friedman wrote:
> Boiling the complex queries down to their simplest form, we test running 60
> of this query simultaneously:
How long does one execution of these queries take (on average)? The
likely bottlenecks are very different between running 60
19 matches
Mail list logo
