Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-16 Thread Marti Raudsepp
On Tue, Feb 15, 2011 at 20:01, Scott Marlowe scott.marl...@gmail.com wrote: run htop and look for red.  if youi've got lots of red bar on each CPU but no io wait then it's waiting for memory access. I don't think this is true. AFAICT the red bar refers to system time, time that's spent in the

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-16 Thread Greg Smith
Kevin Grittner wrote: In fact, I wonder whether we shouldn't leave a couple items you've excluded, since they are sometimes germane to problems posted, like lc_collate and TimeZone. I pulled some of them out only because they're not really postgresql.conf settings; lc_collate and lc_ctype for

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-16 Thread Scott Marlowe
On Wed, Feb 16, 2011 at 6:44 AM, Marti Raudsepp ma...@juffo.org wrote: On Tue, Feb 15, 2011 at 20:01, Scott Marlowe scott.marl...@gmail.com wrote: run htop and look for red.  if youi've got lots of red bar on each CPU but no io wait then it's waiting for memory access. I don't think this is

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-16 Thread Scott Marlowe
Yeah, at max load we are. We're running quad 12 core AMD Magny Cours. Under max load all of our cores go about 20 to 30% red (i.e. kernel) and we wind up waiting on the kernel much more. Could be a mix of context switching and waiting on memory, so it's just a guesstimate I'm making based on

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-16 Thread Kevin Grittner
Justin Pitts justinpi...@gmail.com wrote: I think adding UNION ALL SELECT 'postgres version', version(); might be a good thing. Good point. Added. Greg Smith g...@2ndquadrant.com wrote: Kevin Grittner wrote: In fact, I wonder whether we shouldn't leave a couple items you've

Re: [PERFORM] Really really slow select count(*)

2011-02-16 Thread Ross J. Reedstrom
On Tue, Feb 08, 2011 at 03:52:31PM -0600, Kevin Grittner wrote: Scott Marlowe scott.marl...@gmail.com wrote: Greg Smith g...@2ndquadrant.com wrote: Kevin and I both suggested a fast plus timeout then immediate behavior is what many users seem to want. Are there any settings in

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-16 Thread Thomas Pöhler
Hi, we are using two instances of pgbouncer v1.4 for connection pooling. One for prepared statements (pool_mode session) and one without (pool_mode transaction). Pgbouncer.ini: [pgbouncer] pool_mode = transaction/session server_reset_query = DISCARD ALL; server_check_query = select 1

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-16 Thread Cédric Villemain
2011/2/16 Thomas Pöhler t...@turtle-entertainment.de: Hi, we are using two instances of pgbouncer v1.4 for connection pooling. One for prepared statements (pool_mode session) and one without (pool_mode transaction). Pgbouncer.ini: [pgbouncer] pool_mode = transaction/session

Re: [PERFORM] Really really slow select count(*)

2011-02-16 Thread Bob Lunney
Ross, Way off topic now, but from my time programming electrical meters I can tell you pulling the meter from its socket is potentially an extremely dangerous thing to do. If there is a load across the meter's poles the spark that results on disconnect could kill the puller instantly. (You

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-16 Thread Kevin Grittner
Thomas Pöhlert...@turtle-entertainment.de wrote: we are using two instances of pgbouncer v1.4 for connection pooling. One for prepared statements (pool_mode session) and one without (pool_mode transaction). max_client_conn = 1 default_pool_size = 450 Your best defense against the

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-16 Thread Greg Smith
Thomas Pöhler wrote: We are running a biweekly downtime where we do a complete reindex and vaccum full. We cannot identify certain queries causing this. If you feel that you need VACUUM FULL, either something terribly wrong has happened, or someone has gotten confused. In both cases it's

Re: [PERFORM] Estimating hot data size

2011-02-16 Thread Tomas Vondra
Dne 16.2.2011 21:51, Chris Hoover napsal(a): All, I'm trying to estimate the size of my hot data set, and wanted to get some validation that I'm doing this correctly. Basically, I'm using the sum(heap_blks_read + idx_blks_read) from pg_statio_all_tables, and diffing the numbers over a

Re: [PERFORM] Why we don't want hints Was: Slow count(*) again...

2011-02-16 Thread Bruce Momjian
Kevin Grittner wrote: Shaun Thomas stho...@peak6.com wrote: how difficult would it be to add that syntax to the JOIN statement, for example? Something like this syntax?: JOIN WITH (correlation_factor=0.3) Where 1.0 might mean that for each value on the left there was only one

[PERFORM] Does exclusive locking improve performance?

2011-02-16 Thread Jeremy Palmer
In normal circumstances does locking a table in access exclusive mode improve insert, update and delete operation performance on that table. Is MVCC disabled or somehow has less work to do? Cheers Jeremy