[PERFORM] opportunity to benchmark a quad core Xeon

2006-12-15 Thread Jeff Frost
I have the opportunity to benchmark a system is based on Supermicro 6015B-8V. It has 2x Quad Xeon E5320 1.86GHZ, 4GB DDR2 533, 1x 73GB 10k SCSI. http://www.supermicro.com/products/system/1U/6015/SYS-6015B-8V.cfm We can add more RAM and drives for testing purposes. Can someone suggest what ben

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-15 Thread Michael Stone
On Fri, Dec 15, 2006 at 12:24:46PM -0500, Ron wrote: ATM, the most we can say is that in a number of systems with modest physical IO subsystems So I reran it on a 3.2GHz xeon with 6G RAM off a ramdisk; I/O ain't the bottleneck on that one. Results didn't show didn't show any signficant gain

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-15 Thread Alexander Staubo
On Dec 15, 2006, at 17:53 , Ron wrote: At 09:50 AM 12/15/2006, Greg Smith wrote: On Fri, 15 Dec 2006, Merlin Moncure wrote: The slower is probably due to the unroll loops switch which can actually hurt code due to the larger footprint (less cache coherency). The cache issues are so impor

Re: [PERFORM] File Systems Compared

2006-12-15 Thread Bruno Wolff III
On Fri, Dec 15, 2006 at 10:34:15 -0600, Bruno Wolff III <[EMAIL PROTECTED]> wrote: > The reply wasn't (directly copied to the performance list, but I will > copy this one back. Sorry about this one, I meant to intersperse my replies and hit the 'y' key at the wrong time. (And there ended up bein

Re: [PERFORM] Insertion to temp table deteriorating over time

2006-12-15 Thread Tom Lane
"Steven Flatt" <[EMAIL PROTECTED]> writes: > I've been trying to reproduce the problem for days now :). I've done pretty > much exactly what you describe below, but I can't reproduce the problem on > any of our lab machines. Something is indeed special in this environment. Yuck. You could try s

Re: [PERFORM] Insertion to temp table deteriorating over time

2006-12-15 Thread Steven Flatt
I've been trying to reproduce the problem for days now :). I've done pretty much exactly what you describe below, but I can't reproduce the problem on any of our lab machines. Something is indeed special in this environment. Thanks for all your help, Steve On 12/15/06, Tom Lane <[EMAIL PROTE

Re: [PERFORM] Insertion to temp table deteriorating over time

2006-12-15 Thread Tom Lane
"Steven Flatt" <[EMAIL PROTECTED]> writes: > Are the removable rows in pg_class even an issue? So what if 5000-6000 dead > tuples are generated every hour then vacuumed? Performance continues to > steadily decline over a few days time. Memory usage does not appear to be > bloating. Open file ha

Re: [PERFORM] Insertion to temp table deteriorating over time

2006-12-15 Thread Steven Flatt
Good question, and I agree with your point. Are the removable rows in pg_class even an issue? So what if 5000-6000 dead tuples are generated every hour then vacuumed? Performance continues to steadily decline over a few days time. Memory usage does not appear to be bloating. Open file handles

Re: [PERFORM] Insertion to temp table deteriorating over time

2006-12-15 Thread Tom Lane
"Steven Flatt" <[EMAIL PROTECTED]> writes: > Our application is such that there is a great deal of activity at the > beginning of the hour and minimal activity near the end of the hour. OK ... > The truncate and re-fill process is done once per hour, at the end of the > high-load cycle, so I doub

Re: [PERFORM] Insertion to temp table deteriorating over time

2006-12-15 Thread Steven Flatt
Our application is such that there is a great deal of activity at the beginning of the hour and minimal activity near the end of the hour. Those 3 vacuums were done at (approximately) 30 minutes past, 40 minutes past, and 50 minutes past the hour, during low activity. Vacuums of pg_class look li

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-15 Thread Ron
At 10:55 AM 12/15/2006, Merlin Moncure wrote: On 12/15/06, Ron <[EMAIL PROTECTED]> wrote: There are many instances of x86 compatible code that get 30-40% speedups just because they get access to 16 rather than 8 GPRs when recompiled for x84-64. ...We benchmarked PostgreSQL internally here and

Re: [PERFORM] Insertion to temp table deteriorating over time

2006-12-15 Thread Tom Lane
"Steven Flatt" <[EMAIL PROTECTED]> writes: > Here's the output of "VACUUM VERBOSE pg_class". I think it looks fine. I > even did it three times in a row, each about 10 minutes apart, just to see > what was changing: Hm, look at the numbers of rows removed: > INFO: "pg_class": found 5680 remova

Re: [HACKERS] [PERFORM] EXPLAIN ANALYZE on 8.2

2006-12-15 Thread Ron
At 10:45 AM 12/15/2006, Tom Lane wrote: Gregory Stark <[EMAIL PROTECTED]> writes: > There are various attempts at providing better timing infrastructure at low > overhead but I'm not sure what's out there currently. I expect to do this what > we'll have to do is invent a pg_* abstraction that ha

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-15 Thread Ron
At 09:50 AM 12/15/2006, Greg Smith wrote: On Fri, 15 Dec 2006, Merlin Moncure wrote: The slower is probably due to the unroll loops switch which can actually hurt code due to the larger footprint (less cache coherency). The cache issues are so important with current processors that I'd sugge

Re: [PERFORM] File Systems Compared

2006-12-15 Thread Bruno Wolff III
On Thu, Dec 14, 2006 at 13:21:11 -0800, Ron Mayer <[EMAIL PROTECTED]> wrote: > Bruno Wolff III wrote: > > On Thu, Dec 14, 2006 at 01:39:00 -0500, > > Jim Nasby <[EMAIL PROTECTED]> wrote: > >> On Dec 11, 2006, at 12:54 PM, Bruno Wolff III wrote: > >>> This appears to be changing under Linux. Rec

Re: [PERFORM] File Systems Compared

2006-12-15 Thread Bruno Wolff III
The reply wasn't (directly copied to the performance list, but I will copy this one back. On Thu, Dec 14, 2006 at 13:21:11 -0800, Ron Mayer <[EMAIL PROTECTED]> wrote: > Bruno Wolff III wrote: > > On Thu, Dec 14, 2006 at 01:39:00 -0500, > > Jim Nasby <[EMAIL PROTECTED]> wrote: > >> On Dec 11, 2

Re: [PERFORM] Insertion to temp table deteriorating over time

2006-12-15 Thread Steven Flatt
Here's the output of "VACUUM VERBOSE pg_class". I think it looks fine. I even did it three times in a row, each about 10 minutes apart, just to see what was changing: INFO: vacuuming "pg_catalog.pg_class" INFO: index "pg_class_oid_index" now contains 3263 row versions in

Re: [HACKERS] [PERFORM] EXPLAIN ANALYZE on 8.2

2006-12-15 Thread Gregory Stark
Tom Lane <[EMAIL PROTECTED]> writes: > Gregory Stark <[EMAIL PROTECTED]> writes: > > There are various attempts at providing better timing infrastructure at low > > overhead but I'm not sure what's out there currently. I expect to do this > > what > > we'll have to do is invent a pg_* abstraction

Re: [HACKERS] [PERFORM] EXPLAIN ANALYZE on 8.2

2006-12-15 Thread Tom Lane
Peter Eisentraut <[EMAIL PROTECTED]> writes: > Am Freitag, 15. Dezember 2006 11:28 schrieb Simon Riggs: >> Until we work out a better solution we can fix this in two ways: >> >> 1. EXPLAIN ANALYZE [ [ WITH | WITHOUT ] TIME STATISTICS ] ... >> 2. enable_analyze_timer = off | on (default) (USERSET)

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-15 Thread Merlin Moncure
On 12/15/06, Ron <[EMAIL PROTECTED]> wrote: At 09:23 AM 12/15/2006, Merlin Moncure wrote: >On 12/15/06, Ron <[EMAIL PROTECTED]> wrote: >> >>It seems unusual that code generation options which give access to >>more registers would ever result in slower code... > >The slower is probably due to the

Re: [HACKERS] [PERFORM] EXPLAIN ANALYZE on 8.2

2006-12-15 Thread Martijn van Oosterhout
On Fri, Dec 15, 2006 at 09:56:57AM -0500, Tom Lane wrote: > Martijn van Oosterhout writes: > > On Fri, Dec 15, 2006 at 12:20:46PM +, Simon Riggs wrote: > >> Maybe sampling every 10 rows will bring things down to an acceptable > >> level (after the first N). You tried less than 10 didn't you? >

Re: [HACKERS] [PERFORM] EXPLAIN ANALYZE on 8.2

2006-12-15 Thread Tom Lane
Gregory Stark <[EMAIL PROTECTED]> writes: > There are various attempts at providing better timing infrastructure at low > overhead but I'm not sure what's out there currently. I expect to do this what > we'll have to do is invent a pg_* abstraction that has various implementations > on different ar

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-15 Thread Ron
At 09:23 AM 12/15/2006, Merlin Moncure wrote: On 12/15/06, Ron <[EMAIL PROTECTED]> wrote: It seems unusual that code generation options which give access to more registers would ever result in slower code... The slower is probably due to the unroll loops switch which can actually hurt code d

Re: [HACKERS] [PERFORM] EXPLAIN ANALYZE on 8.2

2006-12-15 Thread Tom Lane
Martijn van Oosterhout writes: > On Fri, Dec 15, 2006 at 12:20:46PM +, Simon Riggs wrote: >> Maybe sampling every 10 rows will bring things down to an acceptable >> level (after the first N). You tried less than 10 didn't you? > Yeah, it reduced the number of calls as the count got larger. It

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-15 Thread Greg Smith
On Fri, 15 Dec 2006, Merlin Moncure wrote: The slower is probably due to the unroll loops switch which can actually hurt code due to the larger footprint (less cache coherency). The cache issues are so important with current processors that I'd suggest throwing -Os (optimize for size) into th

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-15 Thread Merlin Moncure
On 12/15/06, Ron <[EMAIL PROTECTED]> wrote: I'm looking more closely into exactly what the various gcc -O optimizations do on Kx's as well. 64b vs 32b gets x86 compatible code access to ~ 2x as many registers; and MMX or SSE instructions get you access to not only more registers, but wider ones a

Re: [HACKERS] [PERFORM] EXPLAIN ANALYZE on 8.2

2006-12-15 Thread Florian Weimer
* Simon Riggs: >> I think the best option is setitimer(), but it's not POSIX so >> platform support is going to be patchy. > > Don't understand that. I thought that was to do with alarms and > signals. You could use it for sampling. Every few milliseconds, you record which code is executing (pos

Re: [HACKERS] [PERFORM] EXPLAIN ANALYZE on 8.2

2006-12-15 Thread Martijn van Oosterhout
On Fri, Dec 15, 2006 at 12:20:46PM +, Simon Riggs wrote: > > I > > wrote a patch that tried statistical sampling, but the figures were too > > far off for people's liking. > > Well, I like your ideas, so if you have any more... > > Maybe sampling every 10 rows will bring things down to an acc

Re: [HACKERS] [PERFORM] EXPLAIN ANALYZE on 8.2

2006-12-15 Thread Martijn van Oosterhout
On Fri, Dec 15, 2006 at 12:15:59PM +, Gregory Stark wrote: > There are various attempts at providing better timing infrastructure at low > overhead but I'm not sure what's out there currently. I expect to do this what > we'll have to do is invent a pg_* abstraction that has various implementati

Re: [HACKERS] [PERFORM] EXPLAIN ANALYZE on 8.2

2006-12-15 Thread Simon Riggs
On Fri, 2006-12-15 at 11:50 +0100, Martijn van Oosterhout wrote: > On Fri, Dec 15, 2006 at 10:28:08AM +, Simon Riggs wrote: > > Until we work out a better solution we can fix this in two ways: > > > > 1. EXPLAIN ANALYZE [ [ WITH | WITHOUT ] TIME STATISTICS ] ... > > > > 2. enable_analyze_time

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-15 Thread Ron
At 04:54 AM 12/15/2006, Alexander Staubo wrote: On Dec 15, 2006, at 04:09 , Ron wrote: At 07:27 PM 12/14/2006, Alexander Staubo wrote: Sorry, I neglected to include the pertinent graph: http://purefiction.net/paste/pgbench2.pdf In fact, your graph suggests that using arch specific options

Re: [HACKERS] [PERFORM] EXPLAIN ANALYZE on 8.2

2006-12-15 Thread Dimitri Fontaine
Hi list, Le vendredi 15 décembre 2006 11:50, Martijn van Oosterhout a écrit : > BTW, doing gettimeofday() without kernel entry is not really possible. > You could use the cycle counter but it has the problem that if you have > multiple CPUs you need to calibrate the result. If the CPU goes to > sl

Re: [HACKERS] [PERFORM] EXPLAIN ANALYZE on 8.2

2006-12-15 Thread Martijn van Oosterhout
On Fri, Dec 15, 2006 at 10:28:08AM +, Simon Riggs wrote: > Until we work out a better solution we can fix this in two ways: > > 1. EXPLAIN ANALYZE [ [ WITH | WITHOUT ] TIME STATISTICS ] ... > > 2. enable_analyze_timer = off | on (default) (USERSET) What exactly would this do? Only count actu

Re: [PERFORM] EXPLAIN ANALYZE on 8.2

2006-12-15 Thread Simon Riggs
On Thu, 2006-12-14 at 19:05 -0500, Tom Lane wrote: > "Kelly Burkhart" <[EMAIL PROTECTED]> writes: > > I hope this isn't a "crummy mainboard" but I can't seem to affect > > things by changing clock source (kernel 2.6.16 SLES10). I tried > > kernel command option clock=XXX where XXX in > > (cyclone,

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-15 Thread Steinar H. Gunderson
On Fri, Dec 15, 2006 at 10:53:25AM +0100, Alexander Staubo wrote: > The difference is very slight. I'm going to run without -funroll- > loops and -pipe (which are not arch-related) to get better data. -pipe does not matter for the generated code; it only affects compiler speed. (It simply means t

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-15 Thread Alexander Staubo
On Dec 15, 2006, at 04:09 , Ron wrote: At 07:27 PM 12/14/2006, Alexander Staubo wrote: Sorry, I neglected to include the pertinent graph: http://purefiction.net/paste/pgbench2.pdf In fact, your graph suggests that using arch specific options in addition to -O3 actually =hurts= performance