[PERFORM] Slow queries / commits, mis-configuration or hardware issues?

2011-11-14 Thread Cody Caughlan
Hi, running Postgres 9.1.1 on an EC2 m1.xlarge instance. Machine is a dedicated master with 2 streaming replication nodes. The machine has 16GB of RAM and 4 cores. We're starting to see some slow queries, especially COMMITs that are happening more frequently. The slow queries are against seemingl

Re: [PERFORM] Slow queries / commits, mis-configuration or hardware issues?

2011-11-14 Thread Cody Caughlan
Thanks for your response. Please see below for answers to your questions. On Mon, Nov 14, 2011 at 11:22 AM, Tomas Vondra wrote: > On 14 Listopad 2011, 19:16, Cody Caughlan wrote: >> shared_buffers = 3584MB >> wal_buffers = 16MB >> checkpoint_segments = 32 &g

Re: [PERFORM] Slow queries / commits, mis-configuration or hardware issues?

2011-11-14 Thread Cody Caughlan
On Mon, Nov 14, 2011 at 2:57 PM, Tomas Vondra wrote: > On 14 Listopad 2011, 22:58, Cody Caughlan wrote: >>> Seems reasonable, although I'd bump up the checkpoint_timeout (the 5m is >>> usually too low). >> >> Ok, will do. > > Yes, but find out what th

[PERFORM] Large number of short lived connections - could a connection pool help?

2011-11-14 Thread Cody Caughlan
We have anywhere from 60-80 background worker processes connecting to Postgres, performing a short task and then disconnecting. The lifetime of these tasks averages 1-3 seconds. I know that there is some connection overhead to Postgres, but I dont know what would be the best way to measure this ov

Re: [PERFORM] Large number of short lived connections - could a connection pool help?

2011-11-14 Thread Cody Caughlan
On Mon, Nov 14, 2011 at 4:59 PM, Ben Chobot wrote: > On Nov 14, 2011, at 4:42 PM, Cody Caughlan wrote: > >> We have anywhere from 60-80 background worker processes connecting to >> Postgres, performing a short task and then disconnecting. The lifetime >> of these ta

Re: [PERFORM] Slow queries / commits, mis-configuration or hardware issues?

2011-11-15 Thread Cody Caughlan
On Tue, Nov 15, 2011 at 5:16 PM, Tomas Vondra wrote: > Dne 14.11.2011 22:58, Cody Caughlan napsal(a): >> I ran bonnie++ on a slave node, doing active streaming replication but >> otherwise idle: >> http://batch-files-test.s3.amazonaws.com/sql03.prod.html >> >> bon

Re: [PERFORM] Slow queries / commits, mis-configuration or hardware issues?

2011-11-16 Thread Cody Caughlan
On Nov 16, 2011, at 8:52 AM, Tomas Vondra wrote: > On 16 Listopad 2011, 2:21, Cody Caughlan wrote: >> How did you build your RAID array? Maybe I have a fundamental flaw / >> misconfiguration. I am doing it via: >> >> $ yes | mdadm --create /dev/md0 --level=10 -c256 --