Re: [PERFORM] PostgreSQL performance issues

2006-08-30 Thread Willo van der Merwe
Merlin Moncure wrote: On 8/29/06, Willo van der Merwe [EMAIL PROTECTED] wrote: and it has 743321 rows and a explain analyze select count(*) from property_values; you have a number of options: All good ideas and I'll be sure to implement them later. I am curious why you need to query the

[PERFORM] performance problems.

2006-08-30 Thread Matthew Sullivan
All, Got a little bit of a performance problem I hope that can be resolved. All the files/info I believe you are going to ask for are here: http://www.au.sorbs.net/~matthew/postgres/30.8.06/ The odd thing was it originally was fast (1-2 seconds) which is all I need - the query is a

Re: [PERFORM] PostgreSQL performance issues

2006-08-30 Thread Luke Lonergan
Currently the load looks like this: Cpu0 : 96.8% us, 1.9% sy, 0.0% ni, 0.3% id, 0.0% wa, 0.0% hi, 1.0% si Cpu1 : 97.8% us, 1.6% sy, 0.0% ni, 0.3% id, 0.0% wa, 0.0% hi, 0.3% si Cpu2 : 96.8% us, 2.6% sy, 0.0% ni, 0.3% id, 0.0% wa, 0.0% hi, 0.3% si Cpu3 : 96.2% us,

Re: [PERFORM] PostgreSQL performance issues

2006-08-30 Thread Willo van der Merwe
Rusty Conover wrote: On Aug 29, 2006, at 7:52 AM, Willo van der Merwe wrote: Hi, We're running PostgreSQL 8.1.4 on CentOS 4 (Linux version 2.6.9-34.0.1.ELsmp). Hardware specs: 2x AMD Dual-Core Opteron 270 Italy 1Ghz HT 2 x 1MB L2 Cache Socket 940 4 GB Registered ECC PC3200 DDR RAM

Re: [PERFORM] PostgreSQL performance issues

2006-08-30 Thread Willo van der Merwe
Luke Lonergan wrote: Currently the load looks like this: Cpu0 : 96.8% us, 1.9% sy, 0.0% ni, 0.3% id, 0.0% wa, 0.0% hi, 1.0% si Cpu1 : 97.8% us, 1.6% sy, 0.0% ni, 0.3% id, 0.0% wa, 0.0% hi, 0.3% si Cpu2 : 96.8% us, 2.6% sy, 0.0% ni, 0.3% id, 0.0% wa, 0.0% hi, 0.3% si Cpu3

Re: [PERFORM] PostgreSQL performance issues

2006-08-30 Thread Luke Lonergan
Interesting - in this quick snapshot there is no I/O happening at all. What happens when you track the activity for a longer period of time? How about just capturing vmstat during a period when the queries are slow? Has the load average been this high forever or are you experiencing a growth in

Re: [PERFORM] PostgreSQL performance issues

2006-08-30 Thread Alex Hayward
On Wed, 30 Aug 2006, Willo van der Merwe wrote: Merlin Moncure wrote: On 8/29/06, Willo van der Merwe [EMAIL PROTECTED] wrote: and it has 743321 rows and a explain analyze select count(*) from property_values; you have a number of options: All good ideas and I'll be sure to

Re: [PERFORM] PostgreSQL performance issues

2006-08-30 Thread Willo van der Merwe
That's exactly what I'm experiencing. Everything was fine until yesterday, when we noticed a considerable site slow-down. Graphs showed the server suddenly spiking to a load of 67. At first I thought somebody executed a ran-away query, so I restarted postgres, but after it came back up, it

Re: [PERFORM] PostgreSQL performance issues

2006-08-30 Thread Willo van der Merwe
Alex Hayward wrote: On Wed, 30 Aug 2006, Willo van der Merwe wrote: Merlin Moncure wrote: On 8/29/06, Willo van der Merwe [EMAIL PROTECTED] wrote: and it has 743321 rows and a explain analyze select count(*) from property_values; you have a number of options:

Re: [PERFORM] PostgreSQL performance issues

2006-08-30 Thread Dave Cramer
On 30-Aug-06, at 7:35 AM, Willo van der Merwe wrote: Luke Lonergan wrote: Currently the load looks like this: Cpu0 : 96.8% us, 1.9% sy, 0.0% ni, 0.3% id, 0.0% wa, 0.0% hi, 1.0% si Cpu1 : 97.8% us, 1.6% sy, 0.0% ni, 0.3% id, 0.0% wa, 0.0% hi, 0.3% si Cpu2 : 96.8% us, 2.6%

Re: [PERFORM] PostgreSQL performance issues

2006-08-30 Thread Willo van der Merwe
Dave Cramer wrote: On 30-Aug-06, at 7:35 AM, Willo van der Merwe wrote: Luke Lonergan wrote: Currently the load looks like this: Cpu0 : 96.8% us, 1.9% sy, 0.0% ni, 0.3% id, 0.0% wa, 0.0% hi, 1.0% si Cpu1 : 97.8% us, 1.6% sy, 0.0% ni, 0.3% id, 0.0% wa,

Re: [PERFORM] PostgreSQL performance issues

2006-08-30 Thread Dave Dutcher
Title: Message That's an interesting situation. Your CPU's are pegged, and you're hardly doing any IO. I wonder if there is some ineficient query, or if its just very high query volume. Maybe you could try setting log_min_duration_statement to try to track down the slowest of the queries.

Re: [PERFORM] performance problems.

2006-08-30 Thread Vivek Khera
On Aug 30, 2006, at 5:29 AM, Matthew Sullivan wrote: The hardware is a Compaq 6400r with 4G of EDO RAM, 4x500MHz Xeons and a Compaq RAID 3200 in RAID 5 configuration running across 3 spindles (34G total space). The OS is FreeBSD 5.4-RELEASE-p14 The PG Version is 8.1.3 What else does

Re: [PERFORM] PostgreSQL performance issues

2006-08-30 Thread Willo van der Merwe
Dave Dutcher wrote: That's an interesting situation. Your CPU's are pegged, and you're hardly doing any IO. I wonder if there is some ineficient query, or if its just very high query volume. Maybe you could try setting log_min_duration_statement to try to track down the slowest of the

Re: [PERFORM] PostgreSQL performance issues

2006-08-30 Thread Merlin Moncure
On 8/30/06, Willo van der Merwe [EMAIL PROTECTED] wrote: This was just an example. All queries have slowed down. Could it be that I've reached some cut-off and now my disk is thrashing? Currently the load looks like this: Cpu0 : 96.8% us, 1.9% sy, 0.0% ni, 0.3% id, 0.0% wa, 0.0% hi, 1.0%

Re: [PERFORM] performance problems.

2006-08-30 Thread Jim C. Nasby
On Wed, Aug 30, 2006 at 10:10:28AM -0400, Vivek Khera wrote: effective_cache_size = 27462# `sysctl -n vfs.hibufspace` / 8192 (BLKSZ) random_page_cost = 2 You misunderstand how effective_cache_size is used. It's the *only* memory factor that plays a role in cost estimator

Re: [PERFORM] PostgreSQL performance issues

2006-08-30 Thread Alan Hodgson
On Wednesday 30 August 2006 03:48, Willo van der Merwe [EMAIL PROTECTED] wrote: Hi Rusty, Good ideas and I've implemented some of them, and gained about 10%. I'm still sitting on a load avg of about 60. Any ideas on optimizations on my postgresql.conf, that might have an effect? If all of

Re: [PERFORM] slow i/o

2006-08-30 Thread Jignesh K. Shah
The bgwriter parameters changed in 8.1 Try bgwriter_lru_maxpages=0 bgwriter_lru_percent=0 to turn off bgwriter and see if there is any change. -Jignesh Junaili Lie wrote: Hi Jignesh, Thank you for my reply. I have the setting just like what you described: wal_sync_method = fsync

Re: [PERFORM] performance problems.

2006-08-30 Thread Alex Hayward
On Wed, 30 Aug 2006, Jim C. Nasby wrote: On Wed, Aug 30, 2006 at 10:10:28AM -0400, Vivek Khera wrote: effective_cache_size = 27462# `sysctl -n vfs.hibufspace` / 8192 (BLKSZ) random_page_cost = 2 You misunderstand how effective_cache_size is used. It's the *only* memory

Re: [PERFORM] slow i/o

2006-08-30 Thread Junaili Lie
I have tried this to no avail. I have also tried changing the bg_writer_delay parameter to 10. The spike in i/o still occurs although not in a consistent basis and it is only happening for a few seconds. On 8/30/06, Jignesh K. Shah [EMAIL PROTECTED] wrote: The bgwriter parameters changed in

Re: [PERFORM] performance problems.

2006-08-30 Thread Matthew Sullivan
Vivek Khera wrote: On Aug 30, 2006, at 5:29 AM, Matthew Sullivan wrote: The hardware is a Compaq 6400r with 4G of EDO RAM, 4x500MHz Xeons and a Compaq RAID 3200 in RAID 5 configuration running across 3 spindles (34G total space). The OS is FreeBSD 5.4-RELEASE-p14 The PG Version is 8.1.3

Re: [PERFORM] performance problems.

2006-08-30 Thread Mark Kirkwood
Matthew Sullivan wrote: The OS is FreeBSD 5.4-RELEASE-p14 The PG Version is 8.1.3 Solutions/tips greatly appreciated. This won't help this particular query, but 6.1-RELEASE will possibly be a better performer generally, in particular for your SMP system - e.g. the vfs layer is no longer

Re: [PERFORM] performance problems.

2006-08-30 Thread Dave Cramer
On 30-Aug-06, at 10:10 AM, Vivek Khera wrote: On Aug 30, 2006, at 5:29 AM, Matthew Sullivan wrote: The hardware is a Compaq 6400r with 4G of EDO RAM, 4x500MHz Xeons and a Compaq RAID 3200 in RAID 5 configuration running across 3 spindles (34G total space). The OS is FreeBSD