[squid-users] Cache performance
Dear Friends, I m going to use squid for cache purpose only.And i heard that for squid cache performance i have to use good RAM and HDD.I have 4gb RAM and 160 GB SATA HDD.And i have 200 users' network.So please suggest me the same.Means can i go with this H/W specification or is there any suggestionPlease friends, suggest me .And in this server i m using only squid for cache gainso also suggest me for RAM also. And one more thing, for better disk performace should i have to go with raid 0 or any other suggestion. And what are the main squid configuration parameter for cache gain? thanks, Benjo j.
Re: [squid-users] Cache performance
Il 17/12/2010 11:09, benjamin fernandis ha scritto: Dear Friends, I m going to use squid for cache purpose only.And i heard that for squid cache performance i have to use good RAM and HDD.I have 4gb RAM and 160 GB SATA HDD.And i have 200 users' network.So please suggest me the same.Means can i go with this H/W specification or is there any suggestionPlease friends, suggest me .And in this server i m using only squid for cache gainso also suggest me for RAM also. And one more thing, for better disk performace should i have to go with raid 0 or any other suggestion. And what are the main squid configuration parameter for cache gain? thanks, Benjo j. I'm no expert but what you have to avoid is having the squid process swap. So following the rough rule of 10MB of RAM for each GB of cache I'd try first with a 100GB cache, which would require 1GB RAM just for managing it. I'd keep cache mem low, like 128M. The rest of RAM would be used by OS for disk cache and buffers. Mount the cache dir with noatime option to avoid unnecessary disk accesses. To increase performance, use multiple cache_dir, each pointing to its own physical disk. Avoid raid if you need performance. That said, I don't want to sound rude, but I think your questions show that you should do some research on your own first. Please search the mailing list archives and the squid site. Many general questions about squid performance and configuration can be solved just by reading those docs and following the links. For a start: http://wiki.squid-cache.org/SquidFaq/ConfiguringSquid HTH -- Marcello Romani
Re: [squid-users] Cache performance
I would also highly recommend using at least a Dual Core CPU, 1.6GHz + for 200 users. CPU performance is also a very important factor for user volume. - Chad E. Naugle Tech Support II, x. 7981 Travel Impressions, Ltd. Marcello Romani mrom...@ottotecnica.com 12/17/2010 10:01 AM Il 17/12/2010 11:09, benjamin fernandis ha scritto: Dear Friends, I m going to use squid for cache purpose only.And i heard that for squid cache performance i have to use good RAM and HDD.I have 4gb RAM and 160 GB SATA HDD.And i have 200 users' network.So please suggest me the same.Means can i go with this H/W specification or is there any suggestionPlease friends, suggest me .And in this server i m using only squid for cache gainso also suggest me for RAM also. And one more thing, for better disk performace should i have to go with raid 0 or any other suggestion. And what are the main squid configuration parameter for cache gain? thanks, Benjo j. I'm no expert but what you have to avoid is having the squid process swap. So following the rough rule of 10MB of RAM for each GB of cache I'd try first with a 100GB cache, which would require 1GB RAM just for managing it. I'd keep cache mem low, like 128M. The rest of RAM would be used by OS for disk cache and buffers. Mount the cache dir with noatime option to avoid unnecessary disk accesses. To increase performance, use multiple cache_dir, each pointing to its own physical disk. Avoid raid if you need performance. That said, I don't want to sound rude, but I think your questions show that you should do some research on your own first. Please search the mailing list archives and the squid site. Many general questions about squid performance and configuration can be solved just by reading those docs and following the links. For a start: http://wiki.squid-cache.org/SquidFaq/ConfiguringSquid HTH -- Marcello Romani Travel Impressions made the following annotations - This message and any attachments are solely for the intended recipient and may contain confidential or privileged information. If you are not the intended recipient, any disclosure, copying, use, or distribution of the information included in this message and any attachments is prohibited. If you have received this communication in error, please notify us by reply e-mail and immediately and permanently delete this message and any attachments. Thank you.
RE: [squid-users] Cache performance
True, but I recommend it, especially for the OS processing the disk I/O, and authenticators, etc, but it's not really a requirement. - Chad E. Naugle Tech Support II, x. 7981 Travel Impressions, Ltd. Bradley, Stephen W. Mr. bradl...@muohio.edu 12/17/2010 11:44 AM I would normally agree but until Squid fully implements SMP what would havin From: Chad Naugle [chad.nau...@travimp.com] Sent: Friday, December 17, 2010 10:12 AM To: Marcello Romani; squid-users@squid-cache.org Subject: Re: [squid-users] Cache performance I would also highly recommend using at least a Dual Core CPU, 1.6GHz + for 200 users. CPU performance is also a very important factor for user volume. - Chad E. Naugle Tech Support II, x. 7981 Travel Impressions, Ltd. Marcello Romani mrom...@ottotecnica.com 12/17/2010 10:01 AM Il 17/12/2010 11:09, benjamin fernandis ha scritto: Dear Friends, I m going to use squid for cache purpose only.And i heard that for squid cache performance i have to use good RAM and HDD.I have 4gb RAM and 160 GB SATA HDD.And i have 200 users' network.So please suggest me the same.Means can i go with this H/W specification or is there any suggestionPlease friends, suggest me .And in this server i m using only squid for cache gainso also suggest me for RAM also. And one more thing, for better disk performace should i have to go with raid 0 or any other suggestion. And what are the main squid configuration parameter for cache gain? thanks, Benjo j. I'm no expert but what you have to avoid is having the squid process swap. So following the rough rule of 10MB of RAM for each GB of cache I'd try first with a 100GB cache, which would require 1GB RAM just for managing it. I'd keep cache mem low, like 128M. The rest of RAM would be used by OS for disk cache and buffers. Mount the cache dir with noatime option to avoid unnecessary disk accesses. To increase performance, use multiple cache_dir, each pointing to its own physical disk. Avoid raid if you need performance. That said, I don't want to sound rude, but I think your questions show that you should do some research on your own first. Please search the mailing list archives and the squid site. Many general questions about squid performance and configuration can be solved just by reading those docs and following the links. For a start: http://wiki.squid-cache.org/SquidFaq/ConfiguringSquid HTH -- Marcello Romani Travel Impressions made the following annotations - This message and any attachments are solely for the intended recipient and may contain confidential or privileged information. If you are not the intended recipient, any disclosure, copying, use, or distribution of the information included in this message and any attachments is prohibited. If you have received this communication in error, please notify us by reply e-mail and immediately and permanently delete this message and any attachments. Thank you. Travel Impressions made the following annotations - This message and any attachments are solely for the intended recipient and may contain confidential or privileged information. If you are not the intended recipient, any disclosure, copying, use, or distribution of the information included in this message and any attachments is prohibited. If you have received this communication in error, please notify us by reply e-mail and immediately and permanently delete this message and any attachments. Thank you.
[squid-users] cache performance: flash drive substitute vs. fast hard drive
Anybody have performance experience (or benchmark results) putting Squid's cache on a Flash Drive? Devices that plug into a disk cable but that contain only what you'd find in a thumb drive are available. They have zero latency and they have much faster transfer speed than a moving disk. On the other hand they don't have any internal cache memory; even small repetetive accesses always go directly to the flash memory. (A regular hard drive typically has 4-32MB cache memory, so although overall access is only as fast as the disk spins, a few repetetive accesses can be very fast.) How do these two opposing tendencies (better average transfer rate but no internal cache memory) net out with Squid's cache access pattern? For a Squid cache, am I better off buying a small but really fast hard drive, or one of these flash drive substitutes? -Chuck Kollars
Re: [squid-users] cache performance: flash drive substitute vs. fast hard drive
Chuck Kollars wrote: Anybody have performance experience (or benchmark results) putting Squid's cache on a Flash Drive? Devices that plug into a disk cable but that contain only what you'd find in a thumb drive are available. They have zero latency and they They have no rotational latency however they're far from zero latency devices... the fastest examples you can get now are in the ~80-100usec range instead of the 8-15ms range. have much faster transfer speed than a moving disk. only in high end parts... many of the ones you see in laptops are actually quite a bit slower than high-end winchester disks. On the other hand they don't have any internal cache memory; Not a generalization that can be made, some enterprise models need battery or capacitor backed write caches to order write erase cycles for wear leveling. In general there's little point in having a read cache however in places where it makes sense, some devices in fact do. . even small repetetive accesses always go directly to the flash memory. high repetive or extremely fragmented writes may be treated differently by the controllers state machine eg by block shadowing so that large regions don't have to be constantly rewritten for small writes. (A regular hard drive typically has 4-32MB cache memory, so although overall access is only as fast as the disk spins, a few repetetive accesses can be very fast.) How do these two opposing tendencies (better average transfer rate but no internal cache memory) net out with Squid's cache access pattern? you're going to have the benchmark a particular variant in order to come to grips with how that nets out... The 16GB sata ssd's I'm using from last year in some security appliances are 1/2 the the speed reading and 1/4th of the speed writing as an analogous 10k rpm 2.5 sas disk in the same box. Compared to a 4200rpm fujitsu ruggedized disk on the same platform they are faster. Looking at the intel x25-m sata disk you can see what a difference a year makes. For a Squid cache, am I better off buying a small but really fast hard drive, or one of these flash drive substitutes? The other part of the equation is the ssd is still around an order of magnitude or more per gigabyte more costly than the sas/sata winchester drive, which is non-trivial when you're talking $700 or so for 80GB of genuinely faster flash. If the alternative were buying 7x300GB 10k rpm sas disks the flash route is a lot spendier for the equivalent capacity. -Chuck Kollars
Re: [squid-users] Cache Performance Tips
Read this: http://www.devshed.com/c/a/Administration/High-Performance-Web-Caching-With-Squid I think this machine is not suitable for a bigger production environment - it wont hold enough disks. If you do a demo, you will probably have only a couple of clients? No need to tune the cache in this case. Otherwise I would suggest reiserfs on the cachedisk/partition. I would increase cache_mem and cache_dir size moderately. How do you define good impression? E.g. using delay pools, redirectors like squidGuard or authentication will probably impress people, but it is not really performance tuning... Regards, Hendrik. Jerry Norton wrote: Hello all, I am new to squid and very impressed so far. I'm feeling a little swamped though as I'm on a timeline to demo this for production. I have the O'Reilly Squid book and have read through the first few chapters and skimmed the rest. With so many config options, I was hoping I could run my server specs by you all to get some suggestions on tuning for performance. I'm just using the defaults currently. Server is a Compaq DL360, 800mhz, 1gb RAM, 9gb HDD. Anyone suggest any variables that I can change from the defaults and why? Again I'm just learning but I really need to make a good impression fast. Thanks everyone for your help, . Jerry Norton broadGap Technologies (801)763-8056 / (877)broadgap 802 East Bamberger Drive - American Fork - UT - 84003 .
RE: [squid-users] Cache Performance Tips
Thank you for the link..that was what I was looking for. Yes the DL360 will just be used for a demo. I am using squidGuard for redirection and blacklists and feel comfortble with that but the sheer volume of different settings in squid.conf has me a little worried. For the demo, the ACL's and blacklists will impress the most but I want to understand the caching better before the server gets thrown into the fire. Is there any way to auto stress test squid? Any apps or the like? If all goes well, we will implement squid in a tradeshow environment with about 500 computers. I have a request to purchase 2 bigger and faster servers that will be run as peers to load balance. Specs for production servers are follows: Debian Woody Stable P4 2.6 GHz / 512k cache 1 gb RAM (2) 80gb ATA HDD - probably be mirrored for failsafe Thanks for the help everyone! -jnorton -Original Message- From: Hendrik Voigtlaender [mailto:[EMAIL PROTECTED] Sent: Thursday, May 20, 2004 6:50 AM To: Jerry Norton Cc: [EMAIL PROTECTED] Subject: Re: [squid-users] Cache Performance Tips Read this: http://www.devshed.com/c/a/Administration/High-Performance-Web-Caching-W ith-Squid I think this machine is not suitable for a bigger production environment - it wont hold enough disks. If you do a demo, you will probably have only a couple of clients? No need to tune the cache in this case. Otherwise I would suggest reiserfs on the cachedisk/partition. I would increase cache_mem and cache_dir size moderately. How do you define good impression? E.g. using delay pools, redirectors like squidGuard or authentication will probably impress people, but it is not really performance tuning... Regards, Hendrik. Jerry Norton wrote: Hello all, I am new to squid and very impressed so far. I'm feeling a little swamped though as I'm on a timeline to demo this for production. I have the O'Reilly Squid book and have read through the first few chapters and skimmed the rest. With so many config options, I was hoping I could run my server specs by you all to get some suggestions on tuning for performance. I'm just using the defaults currently. Server is a Compaq DL360, 800mhz, 1gb RAM, 9gb HDD. Anyone suggest any variables that I can change from the defaults and why? Again I'm just learning but I really need to make a good impression fast. Thanks everyone for your help, . Jerry Norton broadGap Technologies (801)763-8056 / (877)broadgap 802 East Bamberger Drive - American Fork - UT - 84003 .
Re: [squid-users] Cache Performance Tips
http://www.squid-cache.org/Benchmarking/ http://www.web-polygraph.org/ Production spec: I would get more RAM and more disks. Mirror only for system disks, you can afford to lose the cache. At the moment I am using striped cache disks, but striping does not improve perfomance with squid but increases the chance of losing the whole cache completely (thanks to Adam Aube for pointing this out). With ATA I would use at least two or three disks for the cache only. Squid depends heavily on disk performance. If you are going to setup multiple caches you can probably use cheaper but more machines to balance the load. This way you can achieve high availability high performance at moderate costs. We are serving about 1000 Clients / 8...10GByte Traffic a day with a single machine (Compaq ML370, 2GB RAM, 2.something GHz, 2x36GB SCSI for the cache_dir), but I think at a tradeshow you can expect a much higher usage of the clients. Good luck, Hendrik Voigtländer Jerry Norton wrote: Thank you for the link..that was what I was looking for. Yes the DL360 will just be used for a demo. I am using squidGuard for redirection and blacklists and feel comfortble with that but the sheer volume of different settings in squid.conf has me a little worried. For the demo, the ACL's and blacklists will impress the most but I want to understand the caching better before the server gets thrown into the fire. Is there any way to auto stress test squid? Any apps or the like? If all goes well, we will implement squid in a tradeshow environment with about 500 computers. I have a request to purchase 2 bigger and faster servers that will be run as peers to load balance. Specs for production servers are follows: Debian Woody Stable P4 2.6 GHz / 512k cache 1 gb RAM (2) 80gb ATA HDD - probably be mirrored for failsafe Thanks for the help everyone! -jnorton -Original Message- From: Hendrik Voigtlaender [mailto:[EMAIL PROTECTED] Sent: Thursday, May 20, 2004 6:50 AM To: Jerry Norton Cc: [EMAIL PROTECTED] Subject: Re: [squid-users] Cache Performance Tips Read this: http://www.devshed.com/c/a/Administration/High-Performance-Web-Caching-W ith-Squid I think this machine is not suitable for a bigger production environment - it wont hold enough disks. If you do a demo, you will probably have only a couple of clients? No need to tune the cache in this case. Otherwise I would suggest reiserfs on the cachedisk/partition. I would increase cache_mem and cache_dir size moderately. How do you define good impression? E.g. using delay pools, redirectors like squidGuard or authentication will probably impress people, but it is not really performance tuning... Regards, Hendrik. Jerry Norton wrote: Hello all, I am new to squid and very impressed so far. I'm feeling a little swamped though as I'm on a timeline to demo this for production. I have the O'Reilly Squid book and have read through the first few chapters and skimmed the rest. With so many config options, I was hoping I could run my server specs by you all to get some suggestions on tuning for performance. I'm just using the defaults currently. Server is a Compaq DL360, 800mhz, 1gb RAM, 9gb HDD. Anyone suggest any variables that I can change from the defaults and why? Again I'm just learning but I really need to make a good impression fast. Thanks everyone for your help, . Jerry Norton broadGap Technologies (801)763-8056 / (877)broadgap 802 East Bamberger Drive - American Fork - UT - 84003 .
[squid-users] Cache Performance Tips
Hello all, I am new to squid and very impressed so far. I'm feeling a little swamped though as I'm on a timeline to demo this for production. I have the O'Reilly Squid book and have read through the first few chapters and skimmed the rest. With so many config options, I was hoping I could run my server specs by you all to get some suggestions on tuning for performance. I'm just using the defaults currently. Server is a Compaq DL360, 800mhz, 1gb RAM, 9gb HDD. Anyone suggest any variables that I can change from the defaults and why? Again I'm just learning but I really need to make a good impression fast. Thanks everyone for your help, . Jerry Norton broadGap Technologies (801)763-8056 / (877)broadgap 802 East Bamberger Drive - American Fork - UT - 84003 .
Re: [squid-users] cache performance
On Tue, 2 Dec 2003, Nelson Serrao wrote: I spoke to my ISP and found that option b) is the only one thats going to work in my case. I need help on how to use proxy-arp on the proxy server to divide your internal network in two parts without renumbering. See your OS documentation. Each OS does it slightly differently. How to set up proxy-arp is a routing question, not a Squid question. In Linux you assign the same IP on both interfaces and then set up routing so the server knows which IP addresses of the local network segment are on which side and then enable proxy_arp on the affected interfaces. If you like you can cheat by using a 255.255.255.255 netmask on the smallest interface, only requiring the routes for that interface. Regards Henrik
Re: [squid-users] cache performance
I would recommend you to have the clients reconfigured to use the proxy, but there is several ways you can place it as a transparent proxy inbetween the clients and the gateway if you prefer. Any of the following would work: a) Create a new network between the proxy and your gateway, and assign the internal address which was on the gateway to the proxy. b) Use proxy-arp on the proxy server to divide your internal network in two parts without renumbering. c) Run the proxy server as a bridge with interception capabilities. Most likely 'b' is easiest to set up. Regards Henrik On Sat, 29 Nov 2003, Nelson Serrao wrote: hi, my access.log shows a hit rate of 40%. but all this does not matter much because the linux authentication box i use restricts bandwith of my customer for obvious reasons. the cache server is on a live ip with a single nic. it is place in between the router and linux authentication box. i was just imagining the performance it would return if cache server was configured for use on the lan. this would cause cached pages retreival at lan speeds and the results would be wonderful. i am looking out for a way to do it. one of the ways i thought to do this was to place it on the lan but all my customers have the linux authentication box ip as its gateway. the next thing was to use proxy on all client pcs which is a tedious job. any transparent way to do this. thanks in advance
[squid-users] cache performance
hi, my access.log shows a hit rate of 40%. but all this does not matter much because the linux authentication box i use restricts bandwith of my customer for obvious reasons. the cache server is on a live ip with a single nic. it is place in between the router and linux authentication box. i was just imagining the performance it would return if cache server was configured for use on the lan. this would cause cached pages retreival at lan speeds and the results would be wonderful. i am looking out for a way to do it. one of the ways i thought to do this was to place it on the lan but all my customers have the linux authentication box ip as its gateway. the next thing was to use proxy on all client pcs which is a tedious job. any transparent way to do this. thanks in advance
