Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-17 Thread Pierre C
Thomas Pöhler wrote: I remember you said you were using nginx and php-fastcgi, how many web server boxes do you have, and what are the specs ? -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription:

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-17 Thread Justin Pitts
I think adding UNION ALL SELECT 'postgres version', version(); might be a good thing. On Wed, Feb 16, 2011 at 9:55 AM, Greg Smith g...@2ndquadrant.com wrote: Kevin Grittner wrote: In fact, I wonder whether we shouldn't leave a couple items you've excluded, since they are sometimes germane

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-17 Thread Strange, John W
Sent: 16 February 2011 15:43 To: Marti Raudsepp Cc: Thomas Pöhler; pgsql-performance@postgresql.org; Felix Feinhals; Verteiler_A-Team; Björn Metzdorf Subject: Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem On Wed, Feb 16, 2011 at 6:44 AM, Marti Raudsepp ma...@juffo.org wrote

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-16 Thread Marti Raudsepp
On Tue, Feb 15, 2011 at 20:01, Scott Marlowe scott.marl...@gmail.com wrote: run htop and look for red.  if youi've got lots of red bar on each CPU but no io wait then it's waiting for memory access. I don't think this is true. AFAICT the red bar refers to system time, time that's spent in the

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-16 Thread Greg Smith
Kevin Grittner wrote: In fact, I wonder whether we shouldn't leave a couple items you've excluded, since they are sometimes germane to problems posted, like lc_collate and TimeZone. I pulled some of them out only because they're not really postgresql.conf settings; lc_collate and lc_ctype for

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-16 Thread Scott Marlowe
On Wed, Feb 16, 2011 at 6:44 AM, Marti Raudsepp ma...@juffo.org wrote: On Tue, Feb 15, 2011 at 20:01, Scott Marlowe scott.marl...@gmail.com wrote: run htop and look for red.  if youi've got lots of red bar on each CPU but no io wait then it's waiting for memory access. I don't think this is

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-16 Thread Scott Marlowe
; Verteiler_A-Team; Björn Metzdorf Subject: Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem On Wed, Feb 16, 2011 at 6:44 AM, Marti Raudsepp ma...@juffo.org wrote: On Tue, Feb 15, 2011 at 20:01, Scott Marlowe scott.marl...@gmail.com wrote: run htop and look for red.  if youi've

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-16 Thread Kevin Grittner
Justin Pitts justinpi...@gmail.com wrote: I think adding UNION ALL SELECT 'postgres version', version(); might be a good thing. Good point. Added. Greg Smith g...@2ndquadrant.com wrote: Kevin Grittner wrote: In fact, I wonder whether we shouldn't leave a couple items you've

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-16 Thread Thomas Pöhler
Pöhler Betreff: Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem Justin Pitts justinpi...@gmail.com wrote: I think adding UNION ALL SELECT 'postgres version', version(); might be a good thing. Good point. Added. Greg Smith g...@2ndquadrant.com wrote: Kevin Grittner

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-16 Thread Cédric Villemain
: pgsql-performance@postgresql.org; Verteiler_A-Team; Björn Metzdorf; Felix Feinhals; Thomas Pöhler Betreff: Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem Justin Pitts justinpi...@gmail.com wrote: I think adding UNION ALL SELECT 'postgres version', version(); might

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-16 Thread Kevin Grittner
Thomas Pöhlert...@turtle-entertainment.de wrote: we are using two instances of pgbouncer v1.4 for connection pooling. One for prepared statements (pool_mode session) and one without (pool_mode transaction). max_client_conn = 1 default_pool_size = 450 Your best defense against the

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-16 Thread Greg Smith
Thomas Pöhler wrote: We are running a biweekly downtime where we do a complete reindex and vaccum full. We cannot identify certain queries causing this. If you feel that you need VACUUM FULL, either something terribly wrong has happened, or someone has gotten confused. In both cases it's

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-15 Thread Scott Marlowe
On Tue, Feb 15, 2011 at 10:19 AM, Thomas Pöhler t...@turtle-entertainment.de wrote: Since a few weeks we have really strange peaks on this system. User CPU is increasing up to 100% and we have lots of SELECTs running. Are you using pooling of some kind, or do you have LOTS of connections?

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-15 Thread Kevin Grittner
Thomas Pöhlert...@turtle-entertainment.de wrote: we have lots of SELECTs running. How many? Could you show your postgresql.conf file, with all comments removed? What does vmstat 1 (or similar) show at baseline and during your problem episodes? -Kevin -- Sent via pgsql-performance

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-15 Thread Strange, John W
You have also run analyze verbose, and checked to make sure you don't have a ton of bloated indexes? - check the process with strace -p PID - check the diskIO with iostat, not vmstat - run analyze verbose, and possible reindex the database, or cluster the larger tables. - dump from

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-15 Thread marcin mank
On Tue, Feb 15, 2011 at 6:19 PM, Thomas Pöhler t...@turtle-entertainment.de wrote: Hi list, See ganglia: http://dl.dropbox.com/u/183323/CPUloadprobsdb1.jpg What is the bottom graph? queries/minute? Looks like Your database is just getting hammered. Maybe there is a really badly coded page

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-15 Thread Ivan Voras
On 15/02/2011 18:19, Thomas Pöhler wrote: Hi list, first time for me here, hope you’re not dealing too severely with me regarding guidelines. Giving my best. We are running PostgreSQL 8.4.4 on x86_64-unknown-linux-gnu, compiled by GCC gcc (Debian 4.3.2-1.1) 4.3.2, 64-bit on a Supermicro

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-15 Thread Scott Marlowe
On Tue, Feb 15, 2011 at 6:00 PM, Ivan Voras ivo...@freebsd.org wrote: There is an old problem (which I've encountered so I'm replying but it may or may not be in your case) in which PostgreSQL starts behaving badly even for SELECT queries if the number of simultaneous queries exceeds the number

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-15 Thread Greg Smith
Kevin Grittner wrote: Could you show your postgresql.conf file, with all comments removed I just added a sample query to provide the data we always want here without people having to edit their config files, by querying pg_settings for it, to