Hello
On 2021-Apr-13, Andres Freund wrote:
> > The concerns that had come to my mind were more along the lines
> > of things like pg_dump requiring a larger footprint in the shared
> > lock table. We could alleviate that by increasing the default
> > value of max_locks_per_transaction, perhaps.
Hi,
On 2021-04-13 23:04:50 -0400, Tom Lane wrote:
> Andres Freund writes:
> > On 2021-04-13 19:16:46 -0400, Tom Lane wrote:
> >> Like this? This passes check-world, modulo the one very-unsurprising
> >> regression test change. I've not tried to do any performance testing.
>
> > I wonder if the
Andres Freund writes:
> On 2021-04-13 19:16:46 -0400, Tom Lane wrote:
>> Like this? This passes check-world, modulo the one very-unsurprising
>> regression test change. I've not tried to do any performance testing.
> I wonder if there's a realistic chance it could create additional
> deadlocks
Hi,
On 2021-04-13 19:16:46 -0400, Tom Lane wrote:
> > Maybe the same thing we do with user tables, ie not give up the lock
> > when we close a toast rel? As long as the internal lock counters
> > are 64-bit, we'd not have to worry about overflowing them.
Well, I was assuming we'd not want to do
I wrote:
> Andres Freund writes:
>> Cool. And damn: I can't immediately think of a way to optimize this to
>> not require this kind of hack in the future.
> Maybe the same thing we do with user tables, ie not give up the lock
> when we close a toast rel? As long as the internal lock counters
> a
: paul.fried...@streetlightdata.com
-Original Message-
From: Andres Freund
Sent: Tuesday, April 13, 2021 1:48 PM
To: Paul Friedman
Cc: pgsql-performance@lists.postgresql.org
Subject: Re: LWLocks by LockManager slowing large DB
Hi,
On 2021-04-13 11:47:06 -0700, Paul Friedman wrote:
> YES!!! This comple
Andres Freund writes:
> On 2021-04-13 11:47:06 -0700, Paul Friedman wrote:
>> YES!!! This completely alleviates the bottleneck and I'm able to run the
>> queries full-throttle. Thank you SO much for your help+insight.
> Cool. And damn: I can't immediately think of a way to optimize this to
> no
On 2021-Apr-13, Andres Freund wrote:
> > Sounds like I should file this as a requested improvement?
>
> The ability to lock a toast table? Yea, it might be worth doing that. I
> seem to recall this being discussed not too long ago...
Yep, commit 59ab4ac32460 reverted by eeda7f633809. There were
Hi,
On 2021-04-13 11:47:06 -0700, Paul Friedman wrote:
> YES!!! This completely alleviates the bottleneck and I'm able to run the
> queries full-throttle. Thank you SO much for your help+insight.
Cool. And damn: I can't immediately think of a way to optimize this to
not require this kind of hac
res Freund
Sent: Tuesday, April 13, 2021 11:17 AM
To: Paul Friedman
Cc: pgsql-performance@lists.postgresql.org
Subject: Re: LWLocks by LockManager slowing large DB
Hi,
On 2021-04-13 09:33:48 -0700, Paul Friedman wrote:
> I've attached the 2 perf reports. From the 2nd one, I can see lots
Hi,
On 2021-04-13 09:33:48 -0700, Paul Friedman wrote:
> I've attached the 2 perf reports. From the 2nd one, I can see lots of
> time waiting for TOAST table locks on the geometry column, but I
> definitely don't fully understand the implications or why LockManager
> would be struggling here.
Oh
...@streetlightdata.com
*From:* Nikolay Samokhvalov
*Sent:* Monday, April 12, 2021 4:34 PM
*To:* Andres Freund
*Cc:* Paul Friedman ;
pgsql-performance@lists.postgresql.org
*Subject:* Re: LWLocks by LockManager slowing large DB
On Mon, Apr 12, 2021 at 14:57 Andres Freund wrote:
Without knowing the
ubject: Re: LWLocks by LockManager slowing large DB
Hi,
On 2021-04-12 15:56:08 -0700, Paul Friedman wrote:
> Also, I didn't understand your comment about a 'futex profile', could
> you point me in the right direction here?
My earlier mail included a section about running a perf
On Mon, Apr 12, 2021 at 14:57 Andres Freund wrote:
> Without knowing the proportion of LockManager wait events compared to
> the rest it's hard to know what to make of it.
These OSS tools can be useful to understand the proportion:
- pgCenter
https://github.com/lesovsky/pgcenter
- pg_wait_samp
Hi,
On 2021-04-12 15:56:08 -0700, Paul Friedman wrote:
> Also, I didn't understand your comment about a 'futex profile', could you
> point me in the right direction here?
My earlier mail included a section about running a perf profile showing
the callers of the futex() system call, which in turn
?
Thanks.
---Paul
Paul Friedman
CTO
677 Harrison St | San Francisco, CA 94107
M: (650) 270-7676
E-mail: paul.fried...@streetlightdata.com
-Original Message-
From: Andres Freund
Sent: Monday, April 12, 2021 3:22 PM
To: Paul Friedman
Cc: pgsql-performance@lists.postgresql.org
Subject: R
Hi,
On 2021-04-12 15:15:05 -0700, Paul Friedman wrote:
> Thanks again for any advice you have.
I think we'd need the perf profiles to be able to dig into this
further.
It's odd that there are a meaningful amount of LockManager contention in
your case - assuming the stats you collected weren't ju
) 270-7676
E-mail: paul.fried...@streetlightdata.com
-Original Message-
From: Andres Freund
Sent: Monday, April 12, 2021 2:58 PM
To: Paul Friedman
Cc: pgsql-performance@lists.postgresql.org
Subject: Re: LWLocks by LockManager slowing large DB
Hi,
On 2021-04-12 12:37:42 -0700
Hi,
On 2021-04-12 12:37:42 -0700, Paul Friedman wrote:
> Boiling the complex queries down to their simplest form, we test running 60
> of this query simultaneously:
How long does one execution of these queries take (on average)? The
likely bottlenecks are very different between running 60 concurr
Hello, apologies for the long post, but I want to make sure I’ve got enough
details to describe the problem for y’all.
I’ve got a 64-core (Ubuntu 18.04 – 240GB RAM running at GCP) instance
running PG 13.2 and PostGIS 3.1.1 and we’re having troubles getting it to
run more than 30 or so large quer
20 matches
Mail list logo
