Re: [PERFORM] OT: Db2 connection pooling?

2010-01-15 Thread Alan McKay
Ug, sorry! As soon as I hit "enter" I realised this was the wrong list even for OT :-) On Fri, Jan 15, 2010 at 12:16 PM, Alan McKay wrote: > Hey folks, > > Sorry for the OT - we are most of the way through a Db2 --> PG > migration that is some 18 months in the ma

[PERFORM] OT: Db2 connection pooling?

2010-01-15 Thread Alan McKay
Hey folks, Sorry for the OT - we are most of the way through a Db2 --> PG migration that is some 18 months in the making so far.We've got maybe another 3 to 6 months to go before we are complete, and in the meantime have identified the need for connection pooling in Db2, a-la the excellent pgb

[PERFORM] disk I/O problems and Solutions

2009-10-09 Thread Alan McKay
Hey folks, CentOS / PostgreSQL shop over here. I'm hitting 3 of my favorite lists with this, so here's hoping that the BCC trick is the right way to do it :-) We've just discovered thanks to a new Munin plugin http://blogs.amd.co.at/robe/2008/12/graphing-linux-disk-io-statistics-with-munin.html

Re: [PERFORM] statement stats extra load?

2009-09-22 Thread Alan McKay
On Tue, Sep 22, 2009 at 2:42 AM, Magnus Hagander wrote: > That's not true at all. > > If you have many relations in your cluster that have at some point been > touched, the starts collector can create a *significant* load on the I/o > system. I've come across several cases where the only choice wa

Re: [PERFORM] query memory consumption

2009-09-22 Thread Alan McKay
> Best practice to avoid that, is to bump the work_mem temporarily > before the query, and than lower it again, lowers the chance of memory > exhaustion. Interesting - I can do that dynamically? -- “Don't eat anything you've ever seen advertised on TV” - Michael Pollan, author of "In D

Re: [PERFORM] query memory consumption

2009-09-22 Thread Alan McKay
On Mon, Sep 21, 2009 at 4:08 PM, Robert Haas wrote: > Setting work_mem too high is a frequent cause of problems of this sort, I > think. Too high? How high is too high? -- “Don't eat anything you've ever seen advertised on TV” - Michael Pollan, author of "In Defense of Food" -- Se

Re: [PERFORM] statement stats extra load?

2009-09-21 Thread Alan McKay
>> And if so, where does that extra load go?    Disk?  CPU?  RAM? > > As of 8.4.X the load isn't measurable. Thanks Bruce. What about 8.3 since that is our current production DB? -- “Don't eat anything you've ever seen advertised on TV” - Michael Pollan, author of "In Defense of Food

[PERFORM] query memory consumption

2009-09-21 Thread Alan McKay
Hey folks, We are looking to optimize the query I was talking about last week which is killing our system. We have explain and analyze which tell us about the cost of a query time-wise, but what does one use to determine (and trace / predict?) memory consumption? thanks, -Alan -- “Don't eat an

[PERFORM] statement stats extra load?

2009-09-15 Thread Alan McKay
Is there a rule of thumb for the extra load that will be put on a system when statement stats are turned on? And if so, where does that extra load go?Disk? CPU? RAM? -- “Don't eat anything you've ever seen advertised on TV” - Michael Pollan, author of "In Defense of Food" -- Sen

Re: [PERFORM] odd iostat graph

2009-09-11 Thread Alan McKay
> What's the scale on the bottom there?  The label says "by week" but the way > your message is written makes me think it's actually a much smaller time > frame.  If those valleys are around around five minutes apart, those are the > checkpoints finishing; the shape of the graph is right for it to

Re: [PERFORM] odd iostat graph

2009-09-11 Thread Alan McKay
> My guess is this is checkpoint related. I'll assume "checkpoint" is a PG term that I'm not yet familiar with - will query my DBA :-) If this OS buffer cache, wouldn't that be cached an awfully long time? i.e. we're in big trouble if we get a bad crash? -- “Don't eat anything you've ever see

[PERFORM] odd iostat graph

2009-09-11 Thread Alan McKay
Hey folks, Earlier in the week I wrote a Munin plugin that takes the "await" and "average queue length" fields from "iostat -x" and graphs them. This seems rather odd to me : http://picasaweb.google.ca/alan.mckay/Work#5380253477470243954 That is Qlen. And await looks similar http://picasaweb

Re: [PERFORM] what server stats to track / monitor ?

2009-06-28 Thread Alan McKay
Thanks Greg! On Fri, Jun 26, 2009 at 11:27 PM, Greg Smith wrote: -- “Don't eat anything you've ever seen advertised on TV” - Michael Pollan, author of "In Defense of Food" -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscript

[PERFORM] SOLVED: processor running queue - general rule of thumb?

2009-06-23 Thread Alan McKay
Sorry, just in case anyone is filtering on that in the subject line ... On Tue, Jun 23, 2009 at 4:41 PM, Alan McKay wrote: > BTW, our designer got the nytprofile or whatever it is called for Perl > and found out that it was a problem with the POE library that was > being used as a stat

Re: [PERFORM] processor running queue - general rule of thumb?

2009-06-23 Thread Alan McKay
BTW, our designer got the nytprofile or whatever it is called for Perl and found out that it was a problem with the POE library that was being used as a state-machine to drive the whole load suite. It was taking something like 95% of the CPU time! On Fri, Jun 19, 2009 at 11:59 AM, Alan McKay

Re: [PERFORM] processor running queue - general rule of thumb?

2009-06-19 Thread Alan McKay
> Like the other poster said, we likely don't have enough to tell you > what's going on, but from what you've said here it sounds like you're > mostly just CPU bound.  Assuming you're reading the output of vmstat > and top and other tools like that. Thanks. I used 'sadc' from the sysstat RPM (par

[PERFORM] processor running queue - general rule of thumb?

2009-06-19 Thread Alan McKay
Hey folks, I'm new to all this stuff, and am sitting here with kSar looking at some graphed results of some load tests we did, trying to figure things out :-) We got some unsatisfactory results in stressing our system, and now I have to divine where the bottleneck is. We did 4 tests, upping the

Re: [PERFORM] what server stats to track / monitor ?

2009-06-12 Thread Alan McKay
Yes, I'm familiar with Staplr - if anyone from myyearbook.com is listening in, I'm still hoping for that 0.7 update :-) I plan to run both for the immediate term at least. But this only concerns collecting - my biggest concern is how to read/interpret the data! Pointers to good reading material

Re: [PERFORM] what server stats to track / monitor ?

2009-06-12 Thread Alan McKay
> I'm unfamiliar with Munin, but if you can turn off the graphing (so as to > achieve your desired level of un-cluttered-ness) without disabling the capture > of the data that was being graphed, you'll be better off. Others' opinions may > certainly vary, but in my experience, provided you're not c

[PERFORM] what server stats to track / monitor ?

2009-06-12 Thread Alan McKay
Hey folks, I'm new to performance monitoring and tuning of PG/Linux (have a fair bit of experience in Windows, though those skills were last used about 5 years ago) I finally have Munin set up in my production environment, and my goodness it tracks a whole whack of stuff by default! I want to tu

Re: [PERFORM] Best way to load test a postgresql server

2009-06-01 Thread Alan McKay
Disclaimer : I'm very much a newbie here! But I am on the path in my new job to figure this stuff out as well, and went to PG Con here in Ottawa 2 weeks ago and attended quite a few lectures on this topic. Have a look at : http://wiki.postgresql.org/wiki/PgCon_2009 And in particular "Database H

[PERFORM] Continuent (was: Postgres Clustering)

2009-05-28 Thread Alan McKay
Hmmm. Anyone out there have the Continuent solution working with PostgreSQL? If so, what release? We're at 8.3 right now. thanks, -Alan p.s. I'm continuing the cross-post because that is the way I started this thread. Future threads will not be cross-posted. On Thu, May 28, 2009 at 9:34 AM, S

Re: [PERFORM] Postgres Clustering

2009-05-27 Thread Alan McKay
> Depending on your exact needs, which the terminology you're using only allow > to guess about, you might enjoy this reading: >  http://wiki.postgresql.org/wiki/Image:Moskva_DB_Tools.v3.pdf Thanks. To be honest I don't even know myself what my needs are yet. I've only been on the job here for a

[PERFORM] Postgres Clustering

2009-05-27 Thread Alan McKay
Hey folks, I have done some googling and found a few things on the matter. But am looking for some suggestions from the experts out there. Got any good pointers for reading material to help me get up to speed on PostgreSQL clustering? What options are available? What are the issues? Terminol

[PERFORM] running bonnie++

2009-05-27 Thread Alan McKay
Hey folks, During Greg Smith's lecture last week I could have sworn I saw on the screen at some point a really long command line for bonnie++ - with all the switches he uses. But checking his slides I don't see this. Am I mis-remembering? Can someone recommend the best way to run it? What comb