Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-04 Thread Scott Otis
Robert Haas robertmh...@gmail.com: Do you by any chance have a bazillion databases in this cluster? Can you do these? select sum(1) from pg_database; 1555 select pg_relation_size('pg_database'); 221184 select sum(pg_column_size(d.*)) from pg_database; That gave me: ERROR:

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-04 Thread Claus Guttesen
max_fsm_pages = 280 max_fsm_relations = 16 What does the last couple of lines from a 'vacuum analyze verbose' say? I have max_fsm_pages = 400 and max_fsm_relations = 1500. You can also try to lower random_page_cost to a lower value like 1.2 but I doubt this will help in your

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-04 Thread Scott Otis
Claus Guttesen kome...@gmail.com: Would love to get some advice on how to change my conf settings / setup to get better I/O performance. Server Specs: 2x Intel Xeon Quad Core (@2 Ghz - Clovertown,L5335) 4GB RAM 4x Seagate 73GB SAS HDD 10k RPM - in RAID ( stripped and mirrored )

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-04 Thread Magnus Hagander
On Fri, Sep 4, 2009 at 08:54, Scott Otisscott.o...@intand.com wrote: Robert Haas robertmh...@gmail.com: Do you by any chance have a bazillion databases in this cluster?  Can you do these? select sum(1) from pg_database; 1555 Note that there are two features in 8.4 specifically designed to

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-04 Thread Scott Otis
So is there anything I can do in 8.3 to help this? I have tried setting ' track_activities', 'track_counts' and 'autovacuum' to 'off' (which has reduced CPU and I/O a bit) - but the stats collector process is still using up a good deal of CPU and I/O - is there any way to turn stats collecting

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-04 Thread Kevin Grittner
Scott Otis scott.o...@intand.com wrote: So is there anything I can do in 8.3 to help this? I have tried setting 'track_activities', 'track_counts' and 'autovacuum' to 'off' (which has reduced CPU and I/O a bit) You're going to regret that very soon, unless you are *very* sure you have

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-03 Thread Ivan Voras
Scott Otis wrote: 2x Intel Xeon Quad Core (@2 Ghz - Clovertown,L5335) 4GB RAM 4x Seagate 73GB SAS HDD 10k RPM – in RAID ( stripped and mirrored ) Would love to get some advice on how to change my conf settings / setup to get better I/O performance. ~1500 databases w/ ~60 tables each

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-03 Thread Andy Colson
Scott Otis wrote: Would love to get some advice on how to change my conf settings / setup to get better I/O performance. Total I/O (these number are pretty constant throughout the day): Reads: ~ 100 / sec for about 2.6 Mb/sec Writes: ~ 400 /sec for about 46.1Mb/sec Most of the SQL happening

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-03 Thread Andy Colson
Scott Otis wrote: I agree that they don't make sense - part of the reason I am looking for help :) I am using iostat to get those numbers ( which I specify to average over 5 min then collect to display in Cacti ). 2 processes are taking up a good deal of CPU - the postgres stats collector

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-03 Thread Scott Otis
Sorry about not responding to the whole list earlier - this is my first time posting to a mailing list. Would providing more information about the size and complexities of the databases help? I measure I/O stats with iostat - here is the command I use: iostat -d -x mfid0 -t 290 2 I tried

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-03 Thread Andy Colson
Scott Otis wrote: Sorry about not responding to the whole list earlier - this is my first time posting to a mailing list. Would providing more information about the size and complexities of the databases help? I measure I/O stats with iostat - here is the command I use: iostat -d -x mfid0

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-03 Thread Robert Haas
On Thu, Sep 3, 2009 at 4:16 PM, Scott Otisscott.o...@intand.com wrote: Sorry about not responding to the whole list earlier - this is my first time posting to a mailing list. Would providing more information about the size and complexities of the databases help? I measure I/O stats with

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-03 Thread Andy Colson
Robert Haas wrote: On Thu, Sep 3, 2009 at 4:16 PM, Scott Otisscott.o...@intand.com wrote: Sorry about not responding to the whole list earlier - this is my first time posting to a mailing list. Would providing more information about the size and complexities of the databases help? I measure

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-03 Thread Robert Haas
On Thu, Sep 3, 2009 at 5:27 PM, Andy Colsona...@squeakycode.net wrote: Robert Haas wrote: On Thu, Sep 3, 2009 at 4:16 PM, Scott Otisscott.o...@intand.com wrote: Sorry about not responding to the whole list earlier - this is my first time posting to a mailing list. Would providing more

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-03 Thread Ivan Voras
2009/9/3 Scott Otis scott.o...@intand.com: Sorry about not responding to the whole list earlier - this is my first time posting to a mailing list. Would providing more information about the size and complexities of the databases help? I measure I/O stats with iostat - here is the command

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-03 Thread Scott Otis
Simply do iostat mfid0 1 and post 10 lines of its output. tty mfid0 cpu tin tout KB/t tps MB/s us ni sy in id 0 152 108.54 335 35.51 43 0 30 1 27 0 525 85.73 759 63.55 14 0 12 0 74 0 86 67.72 520 34.39 13 0 12 0 75 0 86 86.89 746 63.26

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-03 Thread Scott Otis
Can you post to the list all the uncommented lines from your postgresql.conf file and attach the results of select * from pg_stat_all_tables as an attachment? I attached a CSV of select * from pg_stat_all_tables from one of our more heavily used databases. Note: I turned off stats collection

Re: [PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-03 Thread Robert Haas
On Thu, Sep 3, 2009 at 7:11 PM, Scott Otisscott.o...@intand.com wrote: Can you post to the list all the uncommented lines from your postgresql.conf file and attach the results of select * from pg_stat_all_tables as an attachment? I attached a CSV of select * from pg_stat_all_tables from one

[PERFORM] Seeking performance advice and explanation for high I/O on 8.3

2009-09-02 Thread Scott Otis
Would love to get some advice on how to change my conf settings / setup to get better I/O performance. Server Specs: 2x Intel Xeon Quad Core (@2 Ghz - Clovertown,L5335) 4GB RAM 4x Seagate 73GB SAS HDD 10k RPM - in RAID ( stripped and mirrored ) FreeBSD 6.4 Apache 2.2 PostgreSQL