Re: [PERFORM] Caching of Queries (now with pgpool)

2004-09-23 Thread Jason Coene
vmstat 5 output for while ab was running, from start to finish. Any ideas? Jason > -Original Message- > From: [EMAIL PROTECTED] [mailto:pgsql-performance- > [EMAIL PROTECTED] On Behalf Of Josh Berkus > Sent: Thursday, September 23, 2004 8:06 PM > To: Jason Coene; [EMAI

Re: [PERFORM] Caching of Queries

2004-09-23 Thread Jason Coene
stat. I let it run until it had completed 800 requests. Unless I'm missing something, there's more than the "new connection" IO load here. Jason > -----Original Message- > From: Jason Coene [mailto:[EMAIL PROTECTED] > Sent: Thursday, September 23, 2004 3:08 PM &g

Re: [PERFORM] Caching of Queries

2004-09-23 Thread Jason Coene
Hz FSB, Hyperthreading enabled. Unfortunately, I don't have physical access to the machine to turn HT off. Thanks, Jason > -Original Message- > From: Gaetano Mendola [mailto:[EMAIL PROTECTED] > Sent: Thursday, September 23, 2004 1:41 PM > To: Jason Coene > Subject: Re:

Re: [PERFORM] Caching of Queries

2004-09-23 Thread Jason Coene
0 96048K 26232K semwai 2 0:00 0.70% 0.10% postgres 95959 pgsql 40 96048K 34268K sbwait 2 0:00 0.70% 0.10% postgres > -Original Message- > From: Tom Lane [mailto:[EMAIL PROTECTED] > Sent: Thursday, September 23, 2004 1:06 PM > To: Jason Coene > Cc: &#

Re: [PERFORM] Caching of Queries

2004-09-23 Thread Jason Coene
I'm not an expert, but I've been hunting down a killer performance problem for a while now. It seems this may be the cause. At peak load, our database slows to a trickle. The CPU and disk utilization are normal - 20-30% used CPU and disk performance good. All of our "postgres" processes end up

Re: [PERFORM] indexes make other queries slow!

2004-09-16 Thread Jason Coene
> My guess is that all the queries that involves the columns that are > being indexed need to > be rewritten to use the newly created indexes to avoid the performance > issues. The reason > is that REINDEX does not help either. Does it make sense? > Qing, Generally, adding new indexes blindly w

Re: [PERFORM] Hardware upgrade for a high-traffic database

2004-08-11 Thread Jason Coene
> You mean you are doing > SELECT ... WHERE userid = 42 ORDER BY timestamp DESC LIMIT 5; > and hoping that separate indexes on userid and timestamp will get the > job done? They won't. There are only two possible plans for this, > neither very good: select all of user 42's posts and sort them

Re: [PERFORM] Hardware upgrade for a high-traffic database

2004-08-11 Thread Jason Coene
> > Hi Rod, > > > > I was looking at top and vmstat - which always show under 300MB > "Active". > > We may hit 400MB at peak. Everything I see (though this isn't my area > of > > expertise) points to most of the memory simply being unused. Results > below, > > am I missing something? > > This lo

Re: [PERFORM] Hardware upgrade for a high-traffic database

2004-08-11 Thread Jason Coene
> -Original Message- > From: Rod Taylor [mailto:[EMAIL PROTECTED] > Sent: Wednesday, August 11, 2004 5:46 PM > To: Jason Coene > Cc: 'Merlin Moncure'; Postgresql Performance > Subject: Re: [PERFORM] Hardware upgrade for a high-traffic database > > &g

Re: [PERFORM] Hardware upgrade for a high-traffic database

2004-08-11 Thread Jason Coene
> > Right. The point is: is your i/o bottle neck on the read side or the > write side. With 10-30 inserts/sec and fsync off, it's definitely on > the read side. What's interesting is that such a low insert load is > causing i/o storm problems. How does your app run with fsync on? > > With rea

Re: [PERFORM] Hardware upgrade for a high-traffic database

2004-08-11 Thread Jason Coene
Thanks for all the feedback. To clear it up, we are definitely not CPU bound at the moment. Any slowdown seems to be disk dependant, or from to serialization due to a long query (due to disk). We do have a lot of INSERT/UPDATE calls, specifically on tables that track user sessions, then of cours

Re: [PERFORM] Hardware upgrade for a high-traffic database

2004-08-10 Thread Jason Coene
itely not a DBA by any stretch - though I am forced into the role. From reading this list, it seems to me that our settings are reasonable given our usage, and that a disk upgrade is likely in order. I'd love to hear any suggestions. Thanks, Jason -Original Message- From: Rod Taylor

[PERFORM] Hardware upgrade for a high-traffic database

2004-08-10 Thread Jason Coene
Hi All, We're currently running Postgres 7.4.1 on FreeBSD 5.2, a Dual Xeon 2.4, 2GB ECC, 3Ware Serial ATA RAID 5 w/ 4 disks (SLOW!!). Our database is about 20GB on disk, we have some quite large tables - 2M rows with TEXT fields in a sample table, accessed constantly. We average about 4,000 - 5,

Re: [PERFORM] Intermittent slowdowns, connection delays

2004-05-11 Thread Jason Coene
fect connection delays? I think there's still something else at large here... I've attached a vmstat output, while running dd. The RAID array is tw0. It does show the tw0 device getting significantly more work, numbers not seen during normal operation. Thanks, Jason Coen

[PERFORM] Intermittent slowdowns, connection delays

2004-05-11 Thread Jason Coene
d disks usage is virtually nil. I've attached 60 seconds of "vmstat 5". Memory usage looks like this (constantly): Mem: 110M Active, 1470M Inact, 206M Wired, 61M Cache, 112M Buf, 26M Free I've cleaned up and tested query after query, and nothing is a "hog". On an idle serve