Re: [PERFORM] degenerate performance on one server of 3

2009-06-05 Thread Robert Haas
On Thu, Jun 4, 2009 at 7:31 AM, Erik Aronesty wrote: > is there some way to view the level of "bloat that needs full" in each > table, so i could write a script that alerts me to the need of a > "vacuum full"  without waiting for random queries to "get slow"? > > looking at the results of the "bloa

Re: [PERFORM] Bad Plan for Questionnaire-Type Query

2009-06-05 Thread Robert Haas
On Fri, Jun 5, 2009 at 8:29 PM, David Blewett wrote: > On Fri, Jun 5, 2009 at 7:32 PM, Josh Berkus wrote: >> My first thought would be to increase statistics dramatically on the >> filtered columns in hopes of making PG realize there's a lot of rows there; >> it's off by 8x.  Correlations stats ar

Re: [PERFORM] degenerate performance on one server of 3

2009-06-05 Thread Erik Aronesty
> See ALTER TABLE and CREATE TABLE  (and the Index variants). > > ALTER TABLE foo SET (fillfactor=90); I'll try that. > This will leave on average, 10% of every 8k block empty and allow updates to > columns to more likely live within the same block. Good for the items table. Probably bad for th

Re: [PERFORM] Bad Plan for Questionnaire-Type Query

2009-06-05 Thread David Blewett
On Fri, Jun 5, 2009 at 7:32 PM, Josh Berkus wrote: > My first thought would be to increase statistics dramatically on the > filtered columns in hopes of making PG realize there's a lot of rows there; > it's off by 8x.  Correlations stats are an ongoing issue in PostgreSQL. I started at a stats_ta

Re: [PERFORM] Bad Plan for Questionnaire-Type Query

2009-06-05 Thread Josh Berkus
David, My first thought would be to increase statistics dramatically on the filtered columns in hopes of making PG realize there's a lot of rows there; it's off by 8x. Correlations stats are an ongoing issue in PostgreSQL. -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent v

Re: [PERFORM] Bad Plan for Questionnaire-Type Query

2009-06-05 Thread David Blewett
On Mon, May 25, 2009 at 11:22 AM, David Blewett wrote: > On Sun, May 24, 2009 at 2:42 PM, Tom Lane wrote: >> >> It still feels like this schema design is obscuring correlations that >> the planner needs to know about in order to make decent estimates. > > I'm not sure how to make the planner awar

Re: [PERFORM] Scalability in postgres

2009-06-05 Thread Robert Haas
On Fri, Jun 5, 2009 at 1:02 PM, Greg Smith wrote: > On Fri, 5 Jun 2009, Mark Mielke wrote: >> I disagree that profiling trumps theory every time. > That's an interesting theory.  Unfortunately, profiling shows it doesn't > work that way. I had a laugh when I read this, but I can see someone being

Re: [PERFORM] GiST index performance

2009-06-05 Thread Robert Haas
On Thu, Jun 4, 2009 at 12:33 PM, Matthew Wakeling wrote: > Do you have a recommendation for how to go about profiling Postgres, what > profiler to use, etc? I'm running on Debian Linux x86_64. I mostly compile with --enable-profiling and use gprof. I know Tom Lane has had success with oprofile fo

Re: [PERFORM] Why is my stats collector so busy?

2009-06-05 Thread Robert Schnabel
On Fri, Jun 5, 2009 at 9:38 AM, Bruce Momjian wrote: Laszlo Nagy wrote: On a 8 processor system, my stats collector is always at 100% CPU. Meanwhile disk I/O is very low. We have many databases, they are accessed frequently. Sometimes there are big table updates, but in most of the tim

Re: [PERFORM] Scalability in postgres

2009-06-05 Thread Greg Smith
On Thu, 4 Jun 2009, Mark Mielke wrote: At it's very simplest, this is the difference between "wake one thread" (which is then responsible for waking the next thread) vs "wake all threads"Any system which actually wakes all threads will probably exhibit scaling limitations. The prototype

Re: [PERFORM] Scalability in postgres

2009-06-05 Thread Greg Smith
On Fri, 5 Jun 2009, Mark Mielke wrote: I disagree that profiling trumps theory every time. That's an interesting theory. Unfortunately, profiling shows it doesn't work that way. Let's see if I can summarize the state of things a bit better here: 1) PostgreSQL stops working as efficiently

Re: [PERFORM] Scalability in postgres

2009-06-05 Thread Craig James
Greg Smith wrote: No amount of theoretical discussion advances that any until you're at least staring at a very specific locking problem you've already characterized extensively via profiling. And even then, profiling trumps theory every time. In theory, there is no difference between theory

Re: [PERFORM] Scalability in postgres

2009-06-05 Thread Robert Haas
On Fri, Jun 5, 2009 at 12:33 AM, wrote: > On Fri, 5 Jun 2009, Greg Smith wrote: > >> On Thu, 4 Jun 2009, Robert Haas wrote: >> >>> That's because this thread has altogether too much theory and >>> altogether too little gprof. >> >> But running benchmarks and profiling is actual work; that's so muc

Re: [PERFORM] Scalability in postgres

2009-06-05 Thread Kevin Grittner
Scott Carey wrote: > If you wake up 10,000 threads, and they all can get significant work > done before yielding no matter what order they run, the system will > scale extremely well. But with roughly twice the average response time you would get throttling active requests to the minimum neede

Re: [PERFORM] Scalability in postgres

2009-06-05 Thread Kevin Grittner
Mark Mielke wrote: > Kevin Grittner wrote: >> James Mansion wrote: >>> Kevin Grittner wrote: >>> Sure, but the architecture of those products is based around all the work being done by "engines" which try to establish affinity to different CPUs, and loop through the various tasks

Re: [PERFORM] Why is my stats collector so busy?

2009-06-05 Thread Merlin Moncure
On Fri, Jun 5, 2009 at 9:38 AM, Bruce Momjian wrote: > Laszlo Nagy wrote: >> On a 8 processor system, my stats collector is always at 100% CPU. >> Meanwhile disk I/O is very low. We have many databases, they are >> accessed frequently. Sometimes there are big table updates, but in most >> of the ti

Re: [PERFORM] Why is my stats collector so busy?

2009-06-05 Thread Tom Lane
Laszlo Nagy writes: > On a 8 processor system, my stats collector is always at 100% CPU. What platform? What Postgres version? regards, tom lane -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://

Re: [PERFORM] Why is my stats collector so busy?

2009-06-05 Thread Bruce Momjian
Laszlo Nagy wrote: > On a 8 processor system, my stats collector is always at 100% CPU. > Meanwhile disk I/O is very low. We have many databases, they are > accessed frequently. Sometimes there are big table updates, but in most > of the time only simple queries are ran against the databases, re

[PERFORM] Why is my stats collector so busy?

2009-06-05 Thread Laszlo Nagy
On a 8 processor system, my stats collector is always at 100% CPU. Meanwhile disk I/O is very low. We have many databases, they are accessed frequently. Sometimes there are big table updates, but in most of the time only simple queries are ran against the databases, returning a few records only