RE: [squid-users] Squid Performance (with Polygraph)

2007-11-15 Thread Henrik Nordstrom
On tor, 2007-11-15 at 08:24 +0200, Dave Raven wrote: > I understand that squid would favour only one processor, yet with SMP > on it > lasts 3x longer. My guess would be that's its because the diskd > processes > are able to use more than one processor? Except their cpu usage never > goes > over ~

RE: [squid-users] Squid Performance (with Polygraph)

2007-11-14 Thread Dave Raven
;; squid-users@squid-cache.org Subject: RE: [squid-users] Squid Performance (with Polygraph) On ons, 2007-11-14 at 14:29 +0200, Dave Raven wrote: > Will do - I'll setup polymix-4 tomorrow and try starting on a full > cache. Something interesting though - my processor usage neve

RE: [squid-users] Squid Performance (with Polygraph)

2007-11-14 Thread Henrik Nordstrom
On ons, 2007-11-14 at 14:29 +0200, Dave Raven wrote: > Will do - I'll setup polymix-4 tomorrow and try starting on a full > cache. Something interesting though - my processor usage never really gets > over 50% or so (SMP or single processor) until it crashes; but with SMP > 800RPS lasts 200+

Re: [squid-users] Squid Performance (with Polygraph)

2007-11-14 Thread Adrian Chadd
On Wed, Nov 14, 2007, Tek Bahadur Limbu wrote: > Also your FreeBSD version 4.x might have also made the difference! Its entirely possible - you have to remember that FreeBSD-4.x only allows one process to be in "kernel space" at one time; its entirely possible that avoids various race conditions

RE: [squid-users] Squid Performance (with Polygraph)

2007-11-14 Thread Dave Raven
nutes... Thanks Dave -Original Message- From: Adrian Chadd [mailto:[EMAIL PROTECTED] Sent: Wednesday, November 14, 2007 1:51 PM To: Dave Raven Cc: 'Adrian Chadd'; squid-users@squid-cache.org Subject: Re: [squid-users] Squid Performance (with Polygraph) What you may need to do is r

RE: [squid-users] Squid Performance (with Polygraph)

2007-11-14 Thread Dave Raven
nal Message- From: Tek Bahadur Limbu [mailto:[EMAIL PROTECTED] Sent: Wednesday, November 14, 2007 1:48 PM To: Dave Raven Cc: 'John Moylan'; 'squid-users' Subject: Re: [squid-users] Squid Performance (with Polygraph) Hi Dave, Dave Raven wrote: > I have seen the error messages b

Re: [squid-users] Squid Performance (with Polygraph)

2007-11-14 Thread John Moylan
nesday, November 14, 2007 12:39 PM > To: Dave Raven > Subject: Re: [squid-users] Squid Performance (with Polygraph) > > > Doesn't diskd have a bug whereby it has issues under heavy load. > http://www.squid-cache.org/bugs/show_bug.cgi?id=761 . If so, I am > surprised that i

Re: [squid-users] Squid Performance (with Polygraph)

2007-11-14 Thread Tek Bahadur Limbu
EMAIL PROTECTED] On Behalf Of John Moylan Sent: Wednesday, November 14, 2007 12:39 PM To: Dave Raven Subject: Re: [squid-users] Squid Performance (with Polygraph) Doesn't diskd have a bug whereby it has issues under heavy load. http://www.squid-cache.org/bugs/show_bug.cgi?id=761 . If so, I am surpris

Re: [squid-users] Squid Performance (with Polygraph)

2007-11-14 Thread Adrian Chadd
a slow, steady, predictable pattern? If it was > plateau'ing like originally suggested I'd agree it's obviously hitting a > limit - but my "rawio" tests show each drive is capable of 450 random > writes/reads per second which is far higher than its doing. > >

RE: [squid-users] Squid Performance (with Polygraph)

2007-11-14 Thread Dave Raven
: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of John Moylan Sent: Wednesday, November 14, 2007 12:39 PM To: Dave Raven Subject: Re: [squid-users] Squid Performance (with Polygraph) Doesn't diskd have a bug whereby it has issues under heavy load. http://www.squid-cache.org/bugs/show_bug.c

RE: [squid-users] Squid Performance (with Polygraph)

2007-11-13 Thread Dave Raven
each drive is capable of 450 random writes/reads per second which is far higher than its doing. Thanks Dave -Original Message- From: Adrian Chadd [mailto:[EMAIL PROTECTED] Sent: Saturday, November 10, 2007 12:13 AM To: Dave Raven Cc: 'Adrian Chadd'; squid-users@squid-cache.or

Re: [squid-users] Squid Performance (with Polygraph)

2007-11-09 Thread Adrian Chadd
On Fri, Nov 09, 2007, Dave Raven wrote: > Hi Adrian, > > It works for the full 4 hours with a null cache directory. How would > I see any kind of stats/information on disk IO? From the stats I can see so > far, the disk stats don't change at all when it fails ... That'd be because you're pr

Re: [squid-users] Squid Performance (with Polygraph)

2007-11-09 Thread Marcello Romani
es - but 82 minutes ?? Thanks for all the help Dave -Original Message- From: Adrian Chadd [mailto:[EMAIL PROTECTED] Sent: Friday, November 09, 2007 11:55 AM To: Dave Raven Cc: 'Adrian Chadd'; squid-users@squid-cache.org Subject: Re: [squid-users] Squid Performance (with Polygraph)

RE: [squid-users] Squid Performance (with Polygraph)

2007-11-09 Thread Dave Raven
h ufs and diskd (with the same results, just different times that it fails after). Thanks Dave -Original Message- From: Adrian Chadd [mailto:[EMAIL PROTECTED] Sent: Friday, November 09, 2007 3:35 PM To: Dave Raven Cc: squid-users@squid-cache.org Subject: Re: [squid-users] Squid Performance

Re: [squid-users] Squid Performance (with Polygraph)

2007-11-09 Thread Adrian Chadd
isk load. The > entire time the disks are very underloaded. That said, if I use a null cache > directory this doesn't happen > > I know that sounds like its clearly drives - but 82 minutes ?? > > Thanks for all the help > Dave > > -----Original Message- >

RE: [squid-users] Squid Performance (with Polygraph)

2007-11-09 Thread Dave Raven
ave -Original Message- From: Adrian Chadd [mailto:[EMAIL PROTECTED] Sent: Friday, November 09, 2007 11:55 AM To: Dave Raven Cc: 'Adrian Chadd'; squid-users@squid-cache.org Subject: Re: [squid-users] Squid Performance (with Polygraph) Check netstat -mb and see if you're run

Re: [squid-users] Squid Performance (with Polygraph)

2007-11-09 Thread Adrian Chadd
November 09, 2007 4:08 AM > To: Dave Raven > Cc: squid-users@squid-cache.org > Subject: Re: [squid-users] Squid Performance (with Polygraph) > > On Thu, Nov 08, 2007, Dave Raven wrote: > > Hi Adrian, > > What would cause it to fail after a specific time though - if the >

Re: [squid-users] Squid Performance (with Polygraph)

2007-11-09 Thread Marcello Romani
9, 2007 4:08 AM To: Dave Raven Cc: squid-users@squid-cache.org Subject: Re: [squid-users] Squid Performance (with Polygraph) On Thu, Nov 08, 2007, Dave Raven wrote: Hi Adrian, What would cause it to fail after a specific time though - if the cache_mem is already full and its using the drive

RE: [squid-users] Squid Performance (with Polygraph)

2007-11-08 Thread Dave Raven
Raven Cc: squid-users@squid-cache.org Subject: Re: [squid-users] Squid Performance (with Polygraph) On Thu, Nov 08, 2007, Dave Raven wrote: > Hi Adrian, > What would cause it to fail after a specific time though - if the cache_mem > is already full and its using the drives? I would ha

Re: [squid-users] Squid Performance (with Polygraph)

2007-11-08 Thread Adrian Chadd
On Thu, Nov 08, 2007, Dave Raven wrote: > Hi Adrian, > What would cause it to fail after a specific time though - if the cache_mem > is already full and its using the drives? I would have thought it would fail > immediately ? > > Also there are no log messages about failures or anything... Who

RE: [squid-users] Squid Performance (with Polygraph)

2007-11-08 Thread Dave Raven
Chadd [mailto:[EMAIL PROTECTED] Sent: Thursday, November 08, 2007 8:05 PM To: Dave Raven Cc: 'Adrian Chadd'; squid-users@squid-cache.org Subject: Re: [squid-users] Squid Performance (with Polygraph) On Thu, Nov 08, 2007, Dave Raven wrote: > Hi Adrian, > I've got diskd conf

Re: [squid-users] Squid Performance (with Polygraph)

2007-11-08 Thread Adrian Chadd
On Thu, Nov 08, 2007, Dave Raven wrote: > Hi Adrian, > I've got diskd configured to be used for objects over 500k - the > datacomm run is all 13K objects so essentially it's doing nothing. > Interestingly though I see the same stuff if I use ufs only, or just diskd. Ok. > I am using kqueue

RE: [squid-users] Squid Performance (with Polygraph)

2007-11-08 Thread Dave Raven
add'; squid-users@squid-cache.org Subject: Re: [squid-users] Squid Performance (with Polygraph) On Thu, Nov 08, 2007, Dave Raven wrote: > Hi, > I've been looking for a way to do the profiling, but I'm stuck with > FreeBSD 4 - any ideas? Cache_mem is at 96mb, its almost

Re: [squid-users] Squid Performance (with Polygraph)

2007-11-08 Thread Adrian Chadd
On Thu, Nov 08, 2007, Dave Raven wrote: > Hi, > I've been looking for a way to do the profiling, but I'm stuck with > FreeBSD 4 - any ideas? Cache_mem is at 96mb, its almost definitely getting > filled immediately - I've also tried setting it to 8 just to be sure, no > difference... Hm. Fre

RE: [squid-users] Squid Performance (with Polygraph)

2007-11-08 Thread Dave Raven
-- disk IO I can see with iostat, it seems to stay the same even after my slow down period... Thanks Dave -Original Message- From: Adrian Chadd [mailto:[EMAIL PROTECTED] Sent: Thursday, November 08, 2007 5:17 PM To: Dave Raven Cc: squid-users@squid-cache.org Subject: Re: [squid-users] Squ

Re: [squid-users] Squid Performance (with Polygraph)

2007-11-08 Thread Adrian Chadd
Do some system-level profiling runs (oprofile under Linux, dtrace under Solaris) during the fill phase, the disk intensive phase and the disk overload phase. Are you graphing statistics? Can you graph stuff like CPU, swapping/paging, disk IO? Whats cache_mem set to? Adrian On Thu, Nov 08, 200

[squid-users] Squid Performance (with Polygraph)

2007-11-08 Thread Dave Raven
Hi all, I'm busy testing a squid box with 8xSATA drives, 4gig of DDRII memory and 2x 2.6gig dual core processors. I'm using the basic datacomm test from polygraph. I've configured 6 of the drives to use COSS, and the other two diskd (I've also done basic ufs tests). During all of the tests