On Fri, Apr 15, 2011 at 12:42 AM, Scott Carey sc...@richrelevance.com wrote:
I do know that dual-pivot quicksort provably causes fewer swaps (but the
same # of compares) as the usual single-pivot quicksort. And swaps are a
lot slower than you would expect due to the effects on processor
* Jesper Krogh:
If you have a 1 socket system, all of your data can be fetched from
local ram seen from you cpu, on a 2 socket, 50% of your accesses
will be way slower, 4 socket even worse.
There are non-NUMA multi-socket systems, so this doesn't apply in all
cases. (The E5320-based system
2011/4/14 Florian Weimer fwei...@bfk.de:
* Jesper Krogh:
If you have a 1 socket system, all of your data can be fetched from
local ram seen from you cpu, on a 2 socket, 50% of your accesses
will be way slower, 4 socket even worse.
There are non-NUMA multi-socket systems, so this doesn't
On 4/13/11 9:23 PM, Greg Smith g...@2ndquadrant.com wrote:
Scott Carey wrote:
If postgres is memory bandwidth constrained, what can be done to reduce
its bandwidth use?
Huge Pages could help some, by reducing page table lookups and making
overall access more efficient.
Compressed pages
On Thu, Apr 14, 2011 at 10:05 PM, Scott Carey sc...@richrelevance.com wrote:
Huge Pages helps caches.
Dual-Pivot quicksort is more cache friendly and is _always_ equal to or
faster than traditional quicksort (its a provably improved algorithm).
If you want a cache-friendly sorting algorithm,
On 4/14/11 1:19 PM, Claudio Freire klaussfre...@gmail.com wrote:
On Thu, Apr 14, 2011 at 10:05 PM, Scott Carey sc...@richrelevance.com
wrote:
Huge Pages helps caches.
Dual-Pivot quicksort is more cache friendly and is _always_ equal to or
faster than traditional quicksort (its a provably
--- On Tue, 12/4/11, Greg Smith g...@2ndquadrant.com wrote:
From: Greg Smith g...@2ndquadrant.com
Subject: Re: [PERFORM] Linux: more cores = less concurrency.
To: Kevin Grittner kevin.gritt...@wicourts.gov
Cc: da...@lang.hm, Steve Clark scl...@netwolves.com, Glyn Astill
glynast
If postgres is memory bandwidth constrained, what can be done to reduce
its bandwidth use?
Huge Pages could help some, by reducing page table lookups and making
overall access more efficient.
Compressed pages (speedy / lzo) in memory can help trade CPU cycles for
memory usage for certain memory
Scott Carey wrote:
If postgres is memory bandwidth constrained, what can be done to reduce
its bandwidth use?
Huge Pages could help some, by reducing page table lookups and making
overall access more efficient.
Compressed pages (speedy / lzo) in memory can help trade CPU cycles for
memory usage
On 11-4-2011 22:04 da...@lang.hm wrote:
in your case, try your new servers without hyperthreading. you will end
up with a 4x4 core system, which should handily outperform the 2x4 core
system you are replacing.
the limit isn't 8 cores, it's that the hyperthreaded cores don't work
well with the
--- On Tue, 12/4/11, Merlin Moncure mmonc...@gmail.com wrote:
The issue I'm seeing is that 8 real cores
outperform 16 real
cores, which outperform 32 real cores under high
concurrency.
With every benchmark I've done of PostgreSQL, the
knee in the
performance graph comes right around
--- On Tue, 12/4/11, Scott Marlowe scott.marl...@gmail.com wrote:
From: Scott Marlowe scott.marl...@gmail.com
Subject: Re: [PERFORM] Linux: more cores = less concurrency.
To: Glyn Astill glynast...@yahoo.co.uk
Cc: pgsql-performance@postgresql.org
Date: Tuesday, 12 April, 2011, 6:55
On Mon
--- On Mon, 11/4/11, Kevin Grittner kevin.gritt...@wicourts.gov wrote:
From: Kevin Grittner kevin.gritt...@wicourts.gov
Subject: Re: [PERFORM] Linux: more cores = less concurrency.
To: da...@lang.hm, Steve Clark scl...@netwolves.com, Kevin Grittner
kevin.gritt...@wicourts.gov, Glyn Astill
On Tue, Apr 12, 2011 at 3:54 AM, Glyn Astill glynast...@yahoo.co.uk wrote:
--- On Tue, 12/4/11, Merlin Moncure mmonc...@gmail.com wrote:
The issue I'm seeing is that 8 real cores
outperform 16 real
cores, which outperform 32 real cores under high
concurrency.
With every benchmark I've
On Tue, Apr 12, 2011 at 8:23 AM, Merlin Moncure mmonc...@gmail.com wrote:
On Tue, Apr 12, 2011 at 3:54 AM, Glyn Astill glynast...@yahoo.co.uk wrote:
--- On Tue, 12/4/11, Merlin Moncure mmonc...@gmail.com wrote:
The issue I'm seeing is that 8 real cores
outperform 16 real
cores, which
Glyn Astill glynast...@yahoo.co.uk wrote:
Tried tweeking LOG2_NUM_LOCK_PARTITIONS between 5 and 7. My
results took a dive when I changed to 32 partitions, and improved
as I increaced to 128, but appeared to be happiest at the default
of 16.
Good to know.
Also, if you can profile
--- On Tue, 12/4/11, Merlin Moncure mmonc...@gmail.com wrote:
Can we see some iobound and cpubound pgbench
runs on both
servers?
Of course, I'll post when I've gotten to that.
Ok, there's no writing going on -- so the i/o tets
aren't necessary.
Context switches are also not too
--- On Tue, 12/4/11, Kevin Grittner kevin.gritt...@wicourts.gov wrote:
Wow, zero idle and zero wait, and single digit for
system. Did you
ever run those RAM speed tests? (I don't remember
seeing results
for that -- or failed to recognize them.) At this
point, my best
guess at this point
On Tue, Apr 12, 2011 at 11:01 AM, Glyn Astill glynast...@yahoo.co.uk wrote:
--- On Tue, 12/4/11, Merlin Moncure mmonc...@gmail.com wrote:
Can we see some iobound and cpubound pgbench
runs on both
servers?
Of course, I'll post when I've gotten to that.
Ok, there's no writing going
Glyn Astill glynast...@yahoo.co.uk wrote:
Results from Greg Smiths stream_scaling test are here:
http://www.privatepaste.com/4338aa1196
Well, that pretty much clinches it. Your RAM access tops out at 16
processors. It appears that your processors are spending most of
their time waiting
On Tue, Apr 12, 2011 at 6:40 PM, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
Well, that pretty much clinches it. Your RAM access tops out at 16
processors. It appears that your processors are spending most of
their time waiting for and contending for the RAM bus.
It tops, but it
Hi,
I think that a NUMA architecture machine can solve the problem
A +
Le 11/04/2011 15:04, Glyn Astill a écrit :
Hi Guys,
I'm just doing some tests on a new server running one of our heavy select
functions (the select part of a plpgsql function to allocate seats)
concurrently. We do
Kevin Grittner wrote:
Glyn Astill glynast...@yahoo.co.uk wrote:
Results from Greg Smiths stream_scaling test are here:
http://www.privatepaste.com/4338aa1196
Well, that pretty much clinches it. Your RAM access tops out at 16
processors. It appears that your processors are
Scott Marlowe wrote:
Have you tried running the memory stream benchmark Greg Smith had
posted here a while back? It'll let you know if you're memory is
bottlenecking. Right now my 48 core machines are the king of that
benchmark with something like 70+Gig a second.
The big Opterons are
On Tue, Apr 12, 2011 at 12:00 PM, Greg Smith g...@2ndquadrant.com wrote:
Kevin Grittner wrote:
Glyn Astill glynast...@yahoo.co.uk wrote:
Results from Greg Smiths stream_scaling test are here:
http://www.privatepaste.com/4338aa1196
Well, that pretty much clinches it. Your RAM access
; Steve Clark; Glyn Astill; Joshua D. Drake;
Scott Marlowe; pgsql-performance@postgresql.org
Subject: Re: [PERFORM] Linux: more cores = less concurrency.
On Tue, Apr 12, 2011 at 12:00 PM, Greg Smith g...@2ndquadrant.com wrote:
Kevin Grittner wrote:
Glyn Astill glynast...@yahoo.co.uk wrote
Hi Guys,
I'm just doing some tests on a new server running one of our heavy select
functions (the select part of a plpgsql function to allocate seats)
concurrently. We do use connection pooling and split out some selects to slony
slaves, but the tests here are primeraly to test what an
Glyn Astill glynast...@yahoo.co.uk wrote:
The new server uses 4 x 8 core Xeon X7550 CPUs at 2Ghz
Which has hyperthreading.
our current servers are 2 x 4 core Xeon E5320 CPUs at 2Ghz.
Which doesn't have hyperthreading.
PostgreSQL often performs worse with hyperthreading than without.
On Mon, 11 Apr 2011 13:09:15 -0500, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
Glyn Astill glynast...@yahoo.co.uk wrote:
The new server uses 4 x 8 core Xeon X7550 CPUs at 2Ghz
Which has hyperthreading.
our current servers are 2 x 4 core Xeon E5320 CPUs at 2Ghz.
Which
--- On Mon, 11/4/11, Joshua D. Drake j...@commandprompt.com wrote:
From: Joshua D. Drake j...@commandprompt.com
Subject: Re: [PERFORM] Linux: more cores = less concurrency.
To: Kevin Grittner kevin.gritt...@wicourts.gov
Cc: pgsql-performance@postgresql.org, Glyn Astill glynast
On Mon, Apr 11, 2011 at 12:12 PM, Joshua D. Drake j...@commandprompt.com
wrote:
On Mon, 11 Apr 2011 13:09:15 -0500, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
Glyn Astill glynast...@yahoo.co.uk wrote:
The new server uses 4 x 8 core Xeon X7550 CPUs at 2Ghz
Which has hyperthreading.
On Mon, Apr 11, 2011 at 12:23 PM, Glyn Astill glynast...@yahoo.co.uk wrote:
--- On Mon, 11/4/11, Joshua D. Drake j...@commandprompt.com wrote:
From: Joshua D. Drake j...@commandprompt.com
Subject: Re: [PERFORM] Linux: more cores = less concurrency.
To: Kevin Grittner kevin.gritt
--- On Mon, 11/4/11, Scott Marlowe scott.marl...@gmail.com wrote:
Just FYI, in synthetic pgbench type benchmarks, a 48 core
AMD Magny
Cours with LSI HW RAID and 34 15k6 Hard drives scales
almost linearly
up to 48 or so threads, getting into the 7000+ tps
range. With SW
RAID it gets into
On 04/11/2011 02:32 PM, Scott Marlowe wrote:
On Mon, Apr 11, 2011 at 12:12 PM, Joshua D. Drakej...@commandprompt.com
wrote:
On Mon, 11 Apr 2011 13:09:15 -0500, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
Glyn Astillglynast...@yahoo.co.uk wrote:
The new server uses 4 x 8 core Xeon
On 2011-04-11 21:42, Glyn Astill wrote:
I'll have to try with the synthetic benchmarks next then, but somethings
definately going off here. I'm seeing no disk activity at all as they're
selects and all pages are in ram.
Well, if you dont have enough computations to be bottlenecked on the
--- On Mon, 11/4/11, da...@lang.hm da...@lang.hm wrote:
From: da...@lang.hm da...@lang.hm
Subject: Re: [PERFORM] Linux: more cores = less concurrency.
To: Steve Clark scl...@netwolves.com
Cc: Scott Marlowe scott.marl...@gmail.com, Joshua D. Drake
j...@commandprompt.com, Kevin Grittner
On Mon, 11 Apr 2011, Steve Clark wrote:
On 04/11/2011 02:32 PM, Scott Marlowe wrote:
On Mon, Apr 11, 2011 at 12:12 PM, Joshua D. Drakej...@commandprompt.com
wrote:
On Mon, 11 Apr 2011 13:09:15 -0500, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
Glyn Astillglynast...@yahoo.co.uk wrote:
On Mon, Apr 11, 2011 at 1:42 PM, Glyn Astill glynast...@yahoo.co.uk wrote:
A wild guess is something like multiple cores contending for cpu cache, cpu
affinity, or some kind of contention in the kernel, alas a little out of my
depth.
It's pretty sickening to think I can't get anything else
Glyn Astill glynast...@yahoo.co.uk wrote:
The issue I'm seeing is that 8 real cores outperform 16 real
cores, which outperform 32 real cores under high concurrency.
With every benchmark I've done of PostgreSQL, the knee in the
performance graph comes right around ((2 * cores) +
--- On Mon, 11/4/11, Scott Marlowe scott.marl...@gmail.com wrote:
From: Scott Marlowe scott.marl...@gmail.com
Subject: Re: [PERFORM] Linux: more cores = less concurrency.
To: Glyn Astill glynast...@yahoo.co.uk
Cc: Kevin Grittner kevin.gritt...@wicourts.gov, Joshua D. Drake
j
GA == Glyn Astill glynast...@yahoo.co.uk writes:
GA I was hoping someone had seen this sort of behaviour before,
GA and could offer some sort of explanation or advice.
Jesper's reply is probably most on point as to the reason.
I know that recent Opterons use some of their cache to better
Kevin Grittner kevin.gritt...@wicourts.gov wrote:
I don't know why you were hitting the knee sooner than I've seen
in my benchmarks
If you're compiling your own executable, you might try boosting
LOG2_NUM_LOCK_PARTITIONS (defined in lwlocks.h) to 5 or 6. The
current value of 4 means that
On Mon, Apr 11, 2011 at 6:04 AM, Glyn Astill glynast...@yahoo.co.uk wrote:
The new server uses 4 x 8 core Xeon X7550 CPUs at 2Ghz, our current servers
are 2 x 4 core Xeon E5320 CPUs at 2Ghz.
What I'm seeing is when the number of clients is greater than the number of
cores, the new servers
: [PERFORM] Linux: more cores = less concurrency.
On Mon, Apr 11, 2011 at 12:23 PM, Glyn Astill glynast...@yahoo.co.uk
wrote:
--- On Mon, 11/4/11, Joshua D. Drake j...@commandprompt.com wrote:
From: Joshua D. Drake j...@commandprompt.com
Subject: Re: [PERFORM] Linux: more cores = less
On Mon, Apr 11, 2011 at 6:05 PM, mark dvlh...@gmail.com wrote:
Just wondering, which LSI card ?
Was this 32 drives in Raid 1+0 with a two drive raid 1 for logs or some
other config?
We were using teh LSI but I'll be switching back to Areca when we
go back to HW RAID. The LSI only
On Mon, Apr 11, 2011 at 6:18 PM, Scott Marlowe scott.marl...@gmail.com wrote:
On Mon, Apr 11, 2011 at 6:05 PM, mark dvlh...@gmail.com wrote:
Just wondering, which LSI card ?
Was this 32 drives in Raid 1+0 with a two drive raid 1 for logs or some
other config?
We were using teh LSI but
-Original Message-
From: Scott Marlowe [mailto:scott.marl...@gmail.com]
Sent: Monday, April 11, 2011 6:18 PM
To: mark
Cc: Glyn Astill; Kevin Grittner; Joshua D. Drake; pgsql-
performa...@postgresql.org
Subject: Re: [PERFORM] Linux: more cores = less concurrency.
On Mon, Apr 11
On Mon, Apr 11, 2011 at 5:06 PM, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
Glyn Astill glynast...@yahoo.co.uk wrote:
The issue I'm seeing is that 8 real cores outperform 16 real
cores, which outperform 32 real cores under high concurrency.
With every benchmark I've done of
On Mon, Apr 11, 2011 at 6:50 PM, mark dvlh...@gmail.com wrote:
Interesting, thanks for sharing.
I guess I have never gotten to the point where I felt I needed more than 2
drives for my xlogs. Maybe I have been dismissing that as a possibility
something. (my biggest array is only 24 SFF
On 2011-04-11 22:39, James Cloos wrote:
GA == Glyn Astillglynast...@yahoo.co.uk writes:
GA I was hoping someone had seen this sort of behaviour before,
GA and could offer some sort of explanation or advice.
Jesper's reply is probably most on point as to the reason.
I know that recent
On Mon, Apr 11, 2011 at 7:04 AM, Glyn Astill glynast...@yahoo.co.uk wrote:
Hi Guys,
I'm just doing some tests on a new server running one of our heavy select
functions (the select part of a plpgsql function to allocate seats)
concurrently. We do use connection pooling and split out some
51 matches
Mail list logo
