[PERFORM] "iowait" bug?

2009-03-20 Thread M. Edward (Ed) Borasky
I just discovered this on a LinkedIn user group: http://bugzilla.kernel.org/show_bug.cgi?id=12309 Is anyone here seeing evidence of this in PostgreSQL?? -- M. Edward (Ed) Borasky http://www.linkedin.com/in/edborasky I've never met a happy clam. In fact, most of them were pretty steamed. -- Sen

Re: [PERFORM] Proposal of tunable fix for scalability of 8.4

2009-03-20 Thread Robert Haas
On Fri, Mar 20, 2009 at 7:39 PM, Jignesh K. Shah wrote: > Alvaro Herrera wrote: >>> So Simon's correct. >> And perhaps this explains why Jignesh is measuring an improvement on his >> benchmark.  Perhaps an useful experiment would be to turn this behavior >> off and compare performance.  This lack

Re: [PERFORM] Need help with one query

2009-03-20 Thread Robert Haas
On Fri, Mar 20, 2009 at 4:29 PM, Anne Rosset wrote: > Alvaro Herrera wrote: >> Robert Haas escribió: >>> Something is wrong here.  How can setting enable_seqscan to off result >>> in a plan with a far lower estimated cost than the original plan?  If >>> the planner thought the non-seq-scan plan is

Re: [PERFORM] Need help with one query

2009-03-20 Thread Tom Lane
Robert Haas writes: > On Fri, Mar 20, 2009 at 1:16 PM, Anne Rosset wrote: >> The db version is 8.2.4 > Something is wrong here. How can setting enable_seqscan to off result > in a plan with a far lower estimated cost than the original plan? Planner bug no doubt ... given how old the PG release

Re: [PERFORM] Proposal of tunable fix for scalability of 8.4

2009-03-20 Thread Jignesh K. Shah
Alvaro Herrera wrote: Alvaro Herrera escribió: So Simon's correct. And perhaps this explains why Jignesh is measuring an improvement on his benchmark. Perhaps an useful experiment would be to turn this behavior off and compare performance. This lack of measurement is probably the

Re: [PERFORM] Full statement logging problematic on larger machines?

2009-03-20 Thread Frank Joerdens
On Fri, Mar 20, 2009 at 8:21 PM, Andrew Dunstan wrote: > > > Frank Joerdens wrote: >> >> On Thu, Mar 12, 2009 at 1:38 PM, Frank Joerdens wrote: >> >>> >>> On Thu, Mar 12, 2009 at 1:45 AM, Tom Lane wrote: >>> [...] >>> You could try changing _IOLBF to _IOFBF near the head of postma

Re: [PERFORM] Proposal of tunable fix for scalability of 8.4

2009-03-20 Thread Alvaro Herrera
Alvaro Herrera escribió: > So Simon's correct. And perhaps this explains why Jignesh is measuring an improvement on his benchmark. Perhaps an useful experiment would be to turn this behavior off and compare performance. This lack of measurement is probably the cause that the suggested patch to

Re: [PERFORM] Proposal of tunable fix for scalability of 8.4

2009-03-20 Thread Alvaro Herrera
Alvaro Herrera escribió: > Simon's explanation, however, is at odds with the code. > > http://git.postgresql.org/?p=postgresql.git;a=blob;f=src/backend/storage/lmgr/lwlock.c > > There is "queue jumping" in the regular (heavyweight) lock manager, but > that's a pretty different body of code. I'l

Re: [PERFORM] High CPU Utilization

2009-03-20 Thread Joe Uhl
On Mar 20, 2009, at 4:58 PM, Scott Marlowe wrote: On Fri, Mar 20, 2009 at 2:49 PM, Joe Uhl wrote: On Mar 20, 2009, at 4:29 PM, Scott Marlowe wrote: What does the cs entry on vmstat say at this time? If you're cs is skyrocketing then you're getting a context switch storm, which is usually

Re: [PERFORM] High CPU Utilization

2009-03-20 Thread Scott Marlowe
On Fri, Mar 20, 2009 at 2:49 PM, Joe Uhl wrote: > > On Mar 20, 2009, at 4:29 PM, Scott Marlowe wrote: >> What does the cs entry on vmstat say at this time?  If you're cs is >> skyrocketing then you're getting a context switch storm, which is >> usually a sign that there are just too many things g

Re: [PERFORM] High CPU Utilization

2009-03-20 Thread Joe Uhl
On Mar 20, 2009, at 4:29 PM, Scott Marlowe wrote: On Fri, Mar 20, 2009 at 2:26 PM, Joe Uhl wrote: On Mar 17, 2009, at 12:19 AM, Greg Smith wrote: On Tue, 17 Mar 2009, Gregory Stark wrote: Hm, well the tests I ran for posix_fadvise were actually on a Perc5 -- though who knows if it was t

Re: [PERFORM] High CPU Utilization

2009-03-20 Thread Scott Marlowe
On Fri, Mar 20, 2009 at 2:26 PM, Joe Uhl wrote: > On Mar 17, 2009, at 12:19 AM, Greg Smith wrote: > >> On Tue, 17 Mar 2009, Gregory Stark wrote: >> >>> Hm, well the tests I ran for posix_fadvise were actually on a Perc5 -- >>> though >>> who knows if it was the same under the hood -- and I saw bet

Re: [PERFORM] Need help with one query

2009-03-20 Thread Anne Rosset
Alvaro Herrera wrote: Robert Haas escribió: Something is wrong here. How can setting enable_seqscan to off result in a plan with a far lower estimated cost than the original plan? If the planner thought the non-seq-scan plan is cheaper, it would have picked that one to begin with.

Re: [PERFORM] High CPU Utilization

2009-03-20 Thread Joe Uhl
On Mar 17, 2009, at 12:19 AM, Greg Smith wrote: On Tue, 17 Mar 2009, Gregory Stark wrote: Hm, well the tests I ran for posix_fadvise were actually on a Perc5 -- though who knows if it was the same under the hood -- and I saw better performance than this. I saw about 4MB/s for a single drive

Re: [PERFORM] Full statement logging problematic on larger machines?

2009-03-20 Thread Frank Joerdens
On Thu, Mar 12, 2009 at 1:38 PM, Frank Joerdens wrote: > On Thu, Mar 12, 2009 at 1:45 AM, Tom Lane wrote: > [...] >> You could try changing _IOLBF >> to _IOFBF near the head of postmaster/syslogger.c and see if that helps. The patched server is now running on live, and we'll be watching it over

Re: [PERFORM] Need help with one query

2009-03-20 Thread Alvaro Herrera
Robert Haas escribió: > Something is wrong here. How can setting enable_seqscan to off result > in a plan with a far lower estimated cost than the original plan? If > the planner thought the non-seq-scan plan is cheaper, it would have > picked that one to begin with. GEQO? Anne, what's geqo_th

Re: [PERFORM] Need help with one query

2009-03-20 Thread Robert Haas
On Fri, Mar 20, 2009 at 1:16 PM, Anne Rosset wrote: > Richard Huxton wrote: >> Anne Rosset wrote: >>> EXPLAIN ANALYZE >>> SELECT >>>  audit_change.id             AS id, >>>  audit_change.audit_entry_id AS auditEntryId, >>>  audit_entry.object_id       AS objectId, >>>  audit_change.property_name  

Re: [PERFORM] Full statement logging problematic on larger machines?

2009-03-20 Thread Frank Joerdens
On Thu, Mar 12, 2009 at 9:32 PM, Laurent Laborde wrote: > On Wed, Mar 11, 2009 at 11:42 PM, Frank Joerdens wrote: >> >> effective_cache_size            = 4GB > > Only 4GB with 64GB of ram ? I'd been overly cautious lately with config changes as it's been difficult to argue for downtime and assoc

Re: [PERFORM] Proposal of tunable fix for scalability of 8.4

2009-03-20 Thread Scott Carey
On 3/20/09 8:28 AM, "Matthew Wakeling" wrote: > On Thu, 19 Mar 2009, Scott Carey wrote: >> In type B, the ratio of requests that must context switch is always == >> 1. Every request must queue and wait! > > A remarkably good point, although not completely correct. Every request > that arrives

Re: [PERFORM] current transaction in productive database

2009-03-20 Thread Euler Taveira de Oliveira
m...@bortal.de escreveu: > is there a way to find out, how many transactions my currenc productive > database is doing? > If you're looking for number of transactions then you can query the catalogs as: $ export myq="select sum(xact_commit+xact_rollback) from pg_stat_database" $ psql -U postgres

Re: [PERFORM] current transaction in productive database

2009-03-20 Thread Greg Smith
On Fri, 20 Mar 2009, m...@bortal.de wrote: is there a way to find out, how many transactions my currenc productive database is doing? What you probably want here is not a true transaction count, which might include thing that don't matter much for scaling purposes, but instead to count thing

Re: [PERFORM] Need help with one query

2009-03-20 Thread Anne Rosset
Richard Huxton wrote: Anne Rosset wrote: EXPLAIN ANALYZE SELECT audit_change.id AS id, audit_change.audit_entry_id AS auditEntryId, audit_entry.object_id AS objectId, audit_change.property_name AS propertyName, audit_change.property_type AS propertyType, audit_chang

Re: [PERFORM] current transaction in productive database

2009-03-20 Thread Jeff
On Mar 20, 2009, at 5:26 AM, m...@bortal.de wrote: Hello List, is there a way to find out, how many transactions my currenc productive database is doing? I know know how much i an offer with my new database and hardware, but i would also like to know what i actually _need_ on my current

Re: [PERFORM] Proposal of tunable fix for scalability of 8.4

2009-03-20 Thread Tom Lane
Matthew Wakeling writes: > As for ProcArrayLock, it sounds like it is very much a special case. Quite. Read the section "Interlocking Transaction Begin, Transaction End, and Snapshots" in src/backend/access/transam/README before proposing any changes in this area --- it's a lot more delicate tha

Re: [PERFORM] Proposal of tunable fix for scalability of 8.4

2009-03-20 Thread Alvaro Herrera
Scott Carey escribió: > Your description (much of which I cut out) is exactly how I understood > it until Simon Riggs' post which changed my view and understanding. > Under that situation, waking all shared will leave all X at the > front and hence alternate shared/exclusive/shared/exclusive a

Re: [PERFORM] Proposal of tunable fix for scalability of 8.4

2009-03-20 Thread Matthew Wakeling
On Thu, 19 Mar 2009, Scott Carey wrote: In type B, the ratio of requests that must context switch is always == 1. Every request must queue and wait! A remarkably good point, although not completely correct. Every request that arrives when the lock is held in any way already will queue and wai

Re: [PERFORM] Need help with one query

2009-03-20 Thread Tom Lane
Richard Huxton writes: >> Hash Join (cost=8.79..253664.55 rows=4 width=136) (actual >> time=4612.674..6683.158 rows=4 loops=1) >> Hash Cond: ((audit_change.audit_entry_id)::text = (audit_entry.id)::text) >> -> Seq Scan on audit_change (cost=0.00..225212.52 rows=7584852 >> width=123) (actual tim

Re: [PERFORM] Prepared statement does not exist

2009-03-20 Thread Glyn Astill
--- On Fri, 20/3/09, Nimesh Satam wrote: > From: Nimesh Satam > > > We are receving the following error in the > postgres > > > database logs: > > > > > > 2009-03-19 02:14:20 PDT [2547]: [79-1] LOG: > duration: > > > 0.039 ms statement: > > > RESET ALL > > > 2009-03-19 02:14:20 PDT [2547]: [8

[PERFORM] current transaction in productive database

2009-03-20 Thread m...@bortal.de
Hello List, is there a way to find out, how many transactions my currenc productive database is doing? I know know how much i an offer with my new database and hardware, but i would also like to know what i actually _need_ on my current productive system. Is there a way to find this out?

Re: [PERFORM] Need help with one query

2009-03-20 Thread Richard Huxton
Anne Rosset wrote: > EXPLAIN ANALYZE > SELECT > audit_change.id AS id, > audit_change.audit_entry_id AS auditEntryId, > audit_entry.object_id AS objectId, > audit_change.property_name AS propertyName, > audit_change.property_type AS propertyType, > audit_change.old_v

Re: [PERFORM] Prepared statement does not exist

2009-03-20 Thread Nimesh Satam
Glyn Astill, Thank for your reply. But can you confirm on this? As what I see from the logs, its pgpool which is trying to deallocate the prepared statement and not the application. The application just disconnects and not tyring to use the same connection. Regards, Nimesh. On Thu, Mar 19, 2009