Re: [HACKERS] [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL

2006-07-23 Thread Robert Lor
Tom Lane wrote: Hmmm ... AFAICS this must mean that flushing the WAL data to disk at transaction commit time takes (most of) 20 msec on your hardware. Which still seems high --- on most modern disks that'd be at least two disk revolutions, maybe more. What's the disk hardware you're testing on,

Re: [HACKERS] [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL

2006-07-23 Thread Tom Lane
Robert Lor <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> Yeah, those seem plausible, although the hold time for >> CheckpointStartLock seems awfully high --- about 20 msec >> per transaction. Are you using a nonzero commit_delay? >> > I didn't change commit_delay which defaults to zero. Hmmm

Re: [HACKERS] [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL

2006-07-23 Thread Robert Lor
Tom Lane wrote: Yeah, those seem plausible, although the hold time for CheckpointStartLock seems awfully high --- about 20 msec per transaction. Are you using a nonzero commit_delay? I didn't change commit_delay which defaults to zero. Regards, -Robert ---(end

Re: [HACKERS] [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL

2006-07-23 Thread Tom Lane
Tatsuo Ishii <[EMAIL PROTECTED]> writes: >>> Interesting. We (some Japanese companies including SRA OSS, >>> Inc. Japan) did some PG scalability testing using a Unisys's big 16 >>> (physical) CPU machine and found PG scales up to 8 CPUs. However >>> beyond 8 CPU PG does not scale anymore. The resul

Re: [HACKERS] [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL

2006-07-23 Thread Tom Lane
Robert Lor <[EMAIL PROTECTED]> writes: > Here is the break down between exclusive & shared LWLocks. Do the > numbers look reasonable to you? Yeah, those seem plausible, although the hold time for CheckpointStartLock seems awfully high --- about 20 msec per transaction. Are you using a nonzero co

Re: [HACKERS] [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL

2006-07-21 Thread Robert Lor
Tom Lane wrote: Also, it'd be interesting to count time spent holding shared lock separately from time spent holding exclusive. Tom, Here is the break down between exclusive & shared LWLocks. Do the numbers look reasonable to you? Regards, -Robert bash-3.00# time ./Tom_lwlock_acquire.d

Re: [HACKERS] [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL

2006-07-21 Thread Robert Lor
Tom Lane wrote: Those numbers look a bit suspicious --- I'd expect to see some of the LWLocks being taken in both shared and exclusive modes, but you don't show any such cases. You sure your script is counting correctly? I'll double check to make sure no stupid mistakes were made! Also, it

Re: [HACKERS] [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL

2006-07-21 Thread Sven Geisler
Hi, Tom Lane schrieb: > Robert Lor <[EMAIL PROTECTED]> writes: >> I ran pgbench and fired up a DTrace script using the lwlock probes we've >> added, and it looks like BufMappingLock is the most contended lock, but >> CheckpointStartLocks are held for longer duration! > > Those numbers look a b

Re: [HACKERS] [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL

2006-07-21 Thread Jim C. Nasby
On Fri, Jul 21, 2006 at 12:56:56AM -0700, Robert Lor wrote: > I ran pgbench and fired up a DTrace script using the lwlock probes we've > added, and it looks like BufMappingLock is the most contended lock, but > CheckpointStartLocks are held for longer duration! Not terribly surprising given th

Re: [HACKERS] [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL

2006-07-21 Thread Tom Lane
Robert Lor <[EMAIL PROTECTED]> writes: > I ran pgbench and fired up a DTrace script using the lwlock probes we've > added, and it looks like BufMappingLock is the most contended lock, but > CheckpointStartLocks are held for longer duration! Those numbers look a bit suspicious --- I'd expect to

Re: [HACKERS] [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL

2006-07-21 Thread Robert Lor
Tom Lane wrote: Tatsuo Ishii <[EMAIL PROTECTED]> writes: 18% in s_lock is definitely bad :-(. Were you able to determine which LWLock(s) are accounting for the contention? Sorry for the delay. Finally I got the oprofile data. It's huge(34MB). If you are interested, I can put

Re: [HACKERS] [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL

2006-07-20 Thread Tom Lane
Tatsuo Ishii <[EMAIL PROTECTED]> writes: >>> 18% in s_lock is definitely bad :-(. Were you able to determine which >>> LWLock(s) are accounting for the contention? > Sorry for the delay. Finally I got the oprofile data. It's > huge(34MB). If you are interested, I can put somewhere. Please let me

Re: [HACKERS] [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL

2006-07-13 Thread Tom Lane
Tatsuo Ishii <[EMAIL PROTECTED]> writes: >>> 18% in s_lock is definitely bad :-(. Were you able to determine which >>> LWLock(s) are accounting for the contention? >> >> Yes. We were interested in that too. Some people did addtional tests >> to determin that. I don't have the report handy now. I

Re: [HACKERS] [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL

2006-07-13 Thread Tatsuo Ishii
> > Tatsuo Ishii <[EMAIL PROTECTED]> writes: > > > Interesting. We (some Japanese companies including SRA OSS, > > > Inc. Japan) did some PG scalability testing using a Unisys's big 16 > > > (physical) CPU machine and found PG scales up to 8 CPUs. However > > > beyond 8 CPU PG does not scale anymor

Re: [HACKERS] [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL

2006-06-22 Thread Arjen van der Meijden
On 22-6-2006 15:03, David Roussel wrote: Sureky the 'perfect' line ought to be linear? If the performance was perfectly linear, then the 'pages generated' ought to be G times the number (virtual) processors, where G is the gradient of the graph. In such a case the graph will go through the or

Re: [HACKERS] [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL

2006-06-22 Thread David Roussel
Arjen van der Meijden wrote: Here is a graph of our performance measured on PostgreSQL: http://achelois.tweakers.net/~acm/pgsql-t2000/T2000-schaling-postgresql.png ... The "perfect" line is based on the "Max" value for 1 core and then just multiplied by the amount of cores to have

Re: [HACKERS] [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL

2006-06-18 Thread Arjen van der Meijden
On 17-6-2006 1:24, Josh Berkus wrote: Arjen, I can already confirm very good scalability (with our workload) on postgresql on that machine. We've been testing a 32thread/16G-version and it shows near-linear scaling when enabling 1, 2, 4, 6 and 8 cores (with all four threads enabled). Keen.

Re: [HACKERS] [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL

2006-06-17 Thread Andrew Dunstan
Jim Nasby wrote: On Jun 16, 2006, at 12:01 PM, Josh Berkus wrote: First thing as soon as I have a login, of course, is to set up a Buildfarm instance. Keep in mind that buildfarm clients and benchmarking stuff don't usually mix well. On a fast machine like this a buildfarm run is

Re: [HACKERS] [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL community

2006-06-17 Thread Jim Nasby
On Jun 16, 2006, at 12:01 PM, Josh Berkus wrote: First thing as soon as I have a login, of course, is to set up a Buildfarm instance. Keep in mind that buildfarm clients and benchmarking stuff don't usually mix well. -- Jim C. Nasby, Sr. Engineering Consultant [EMAIL PROTECTED] Pervas

Re: [HACKERS] [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL community

2006-06-17 Thread Jim Nasby
On Jun 16, 2006, at 12:01 PM, Josh Berkus wrote: Folks, I am thrill to inform you all that Sun has just donated a fully loaded T2000 system to the PostgreSQL community, and it's being setup by Corey Shields at OSL (osuosl.org) and should be online probably early next week. The system has

Re: [HACKERS] [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL

2006-06-17 Thread Josh Berkus
Tom, > 18% in s_lock is definitely bad :-(.  Were you able to determine which > LWLock(s) are accounting for the contention? Gavin Sherry and Tom Daly (Sun) are currently working on identifying the problem lock using DLWLOCK_STATS. Any luck, Gavin? -- Josh Berkus PostgreSQL @ Sun San Francisc

Re: [HACKERS] [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL community

2006-06-16 Thread Robert Lor
Arjen van der Meijden wrote: I can already confirm very good scalability (with our workload) on postgresql on that machine. We've been testing a 32thread/16G-version and it shows near-linear scaling when enabling 1, 2, 4, 6 and 8 cores (with all four threads enabled). The threads are a bit

Re: [HACKERS] [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL

2006-06-16 Thread Tatsuo Ishii
> Tatsuo Ishii <[EMAIL PROTECTED]> writes: > > Interesting. We (some Japanese companies including SRA OSS, > > Inc. Japan) did some PG scalability testing using a Unisys's big 16 > > (physical) CPU machine and found PG scales up to 8 CPUs. However > > beyond 8 CPU PG does not scale anymore. The res

Re: [HACKERS] [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL

2006-06-16 Thread Tom Lane
Tatsuo Ishii <[EMAIL PROTECTED]> writes: > Interesting. We (some Japanese companies including SRA OSS, > Inc. Japan) did some PG scalability testing using a Unisys's big 16 > (physical) CPU machine and found PG scales up to 8 CPUs. However > beyond 8 CPU PG does not scale anymore. The result can be

Re: [HACKERS] [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL

2006-06-16 Thread Tatsuo Ishii
> I am thrill to inform you all that Sun has just donated a fully loaded > T2000 system to the PostgreSQL community, and it's being setup by Corey > Shields at OSL (osuosl.org) and should be online probably early next > week. The system has > > * 8 cores, 4 hw threads/core @ 1.2 GHz. Solaris se

Re: [HACKERS] [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL community

2006-06-16 Thread Josh Berkus
Arjen, > I can already confirm very good scalability (with our workload) on > postgresql on that machine. We've been testing a 32thread/16G-version > and it shows near-linear scaling when enabling 1, 2, 4, 6 and 8 cores > (with all four threads enabled). Keen. We're trying to keep the linear sc

Re: [HACKERS] [PERFORM] Sun Donated a Sun Fire T2000 to the PostgreSQL community

2006-06-16 Thread Arjen van der Meijden
On 16-6-2006 17:18, Robert Lor wrote: I think this system is well suited for PG scalability testing, among others. We did an informal test using an internal OLTP benchmark and noticed that PG can scale to around 8 CPUs. Would be really cool if all 32 virtual CPUs can be utilized!!! I can al