On Sun, Jun 14, 2015 at 9:52 AM, Jan Wieck j...@wi3ck.info wrote:
The whole thing turns out to be based on wrong baseline data, taken with a
pgbench client running from a remote machine. It all started out from an
investigation against 9.3. Curiously enough, the s_lock() problem that
existed
The whole thing turns out to be based on wrong baseline data, taken with
a pgbench client running from a remote machine. It all started out from
an investigation against 9.3. Curiously enough, the s_lock() problem
that existed in 9.3 has a very similar effect on throughput as a network
Hi,
I think I may have found one of the problems, PostgreSQL has on machines
with many NUMA nodes. I am not yet sure what exactly happens on the NUMA
bus, but there seems to be a tipping point at which the spinlock
concurrency wreaks havoc and the performance of the database collapses.
On a
On 06/10/2015 09:28 AM, Andres Freund wrote:
On 2015-06-10 09:18:56 -0400, Jan Wieck wrote:
On a machine with 8 sockets, 64 cores, Hyperthreaded 128 threads total, a
pgbench -S peaks with 50-60 clients around 85,000 TPS. The throughput then
takes a very sharp dive and reaches around 20,000 TPS
On 10/06/15 16:05, Andres Freund wrote:
it'll nearly always be beneficial to spin
Trouble is that postgres cannot know if the process holding the lock actually
does run, so if it doesn't, all we're doing is burn cycles and make the problem
worse.
Contrary to that, the kernel does know, so for
On 06/10/2015 10:07 AM, Nils Goroll wrote:
On larger Linux machines, we have been running with spin locks replaced by
generic posix mutexes for years now. I personally haven't look at the code for
ages, but we maintain a patch which pretty much does the same thing still:
Ref:
Jan Wieck j...@wi3ck.info writes:
The attached patch demonstrates that less aggressive spinning and (much)
more often delaying improves the performance on this type of machine.
Hm. One thing worth asking is why the code didn't converge to a good
value of spins_per_delay without help. The
On 2015-06-10 16:12:05 +0200, Nils Goroll wrote:
On 10/06/15 16:05, Andres Freund wrote:
it'll nearly always be beneficial to spin
Trouble is that postgres cannot know if the process holding the lock actually
does run, so if it doesn't, all we're doing is burn cycles and make the
problem
Andres Freund and...@anarazel.de writes:
Unfortunately there's no portable futex support. That's what stopped us
from adopting them so far. And even futexes can be significantly more
heavyweight under moderate contention than our spinlocks - It's rather
easy to reproduce scenarios where
On Wed, Jun 10, 2015 at 09:18:56AM -0400, Jan Wieck wrote:
The attached patch demonstrates that less aggressive spinning and
(much) more often delaying improves the performance on this type of
machine. The 8 socket machine in question scales to over 350,000
TPS.
The patch is meant to
Hi,
On 2015-06-10 09:54:00 -0400, Jan Wieck wrote:
model name : Intel(R) Xeon(R) CPU E7- 8830 @ 2.13GHz
numactl --hardware shows the distance to the attached memory as 10, the
distance to every other node as 21. I interpret that as the machine having
one NUMA bus with all cpu packages
On 2015-06-10 09:18:56 -0400, Jan Wieck wrote:
On a machine with 8 sockets, 64 cores, Hyperthreaded 128 threads total, a
pgbench -S peaks with 50-60 clients around 85,000 TPS. The throughput then
takes a very sharp dive and reaches around 20,000 TPS at 120 clients. It
never recovers from
On larger Linux machines, we have been running with spin locks replaced by
generic posix mutexes for years now. I personally haven't look at the code for
ages, but we maintain a patch which pretty much does the same thing still:
Ref:
As in 200%+ slower.
Have you tried PTHREAD_MUTEX_ADAPTIVE_NP ?
Yes.
Ok, if this can be validated, we might have a new case now for which my
suggestion would not be helpful. Reviewed, optimized code with short critical
sections and no hotspots by design could indeed be an exception where to
On 2015-06-10 11:51:06 -0400, Jan Wieck wrote:
ret = pg_atomic_fetch_sub_u32(buf-state, 1);
if (ret BM_PIN_COUNT_WAITER)
{
pg_atomic_fetch_sub_u32(buf-state, BM_PIN_COUNT_WAITER);
/* XXX: deal with race that another backend has set BM_PIN_COUNT_WAITER
*/
}
There are atomic
On Wed, Jun 10, 2015 at 11:58 AM, Andres Freund and...@anarazel.de wrote:
I think we should just gank spinlocks asap. The hard part is removing
them from lwlock.c's slow path and the buffer headers imo. After that we
should imo be fine replacing them with lwlocks.
Mmmph. I'm not convinced
On 2015-06-10 13:52:14 -0400, Robert Haas wrote:
On Wed, Jun 10, 2015 at 1:39 PM, Andres Freund and...@anarazel.de wrote:
Well, not necessarily. If you can write your algorithm in a way that
xadd etc are used, instead of a lock cmpxchg, you're actually never
spinning on x86 as it's
On Wed, Jun 10, 2015 at 1:39 PM, Andres Freund and...@anarazel.de wrote:
In the uncontended case lwlocks are just as fast as spinlocks now, with
the exception of the local tracking array. They're faster if there's
differences with read/write lockers.
If nothing else, the spinlock calls are
On 2015-06-10 13:19:14 -0400, Robert Haas wrote:
On Wed, Jun 10, 2015 at 11:58 AM, Andres Freund and...@anarazel.de wrote:
I think we should just gank spinlocks asap. The hard part is removing
them from lwlock.c's slow path and the buffer headers imo. After that we
should imo be fine
19 matches
Mail list logo
