I agree with Mr. Crockett's suggestion with regards to hyperthreading. You will get more use out of all your CPU cache than half of it. Also, squid would not benefit from the lowered overhead of thread switching provided by hyperthreading. However I don't think that should affect what you are seeing.
I'd be concerned with the "Median Service Times / HTTP Requests (All)" #s that you are seeing. Also, the "Largest file desc currently in use" of 11111 seems excessively high. I'd guess that your connection is slow enough that squid is having to hold and back-log too many requests. You might do well to ssh into your squid box and try doing browsing actually at the proxy itself and compare the experience with an inside browser getting the same page through squid. My guess is it is probably the same--not very fast.
Another thing that might be driving up the # of file desc usage is the "half_closed_clients off" you have. On my squids I run with this as in the default config, "on".
Also, on my systems we are running named instead of using squid's fqdn cache, this may help things out a bit--ymmv..
I was running a squid box with only 1 GB of RAM recently and moved it to 4GB RAM. The only thing I altered was I changed from "cache_mem 128 MB" to "cache_mem 1024 MB" and "maximum_object_size 256 MB" to "maximum_object_size 512 MB". I also increased the amount of "cache_dir" but found that this caused squid to eat up too much RAM for the disk index and pushed it back to the original setting.
I notice you have a "redirect_children 30"--are you running a redirector? This can significantly alter your numbers.
We currently have 3 seperate ICP-ing squids and each box has from 3-5 cache drives. If you do not have multiple squids, you might want to consider more hardware.. Take a look at my CPU usage--these boxes are kept VERY busy, and I have three of them!
I'll give you my settings. Again I am very conservative on the "cache_dir" size as I think this will eat up quite a bit of RAM to keep in-memory-indices.
Most of my squid.conf: # Timeouts: read_timeout 5 minutes request_timeout 30 seconds pconn_timeout 60 seconds half_closed_clients on shutdown_lifetime 0 seconds negative_ttl 30 seconds icp_query_timeout 1000 milliseconds cache_dir aufs /cache1 2048 64 64 cache_dir aufs /cache2 2048 64 64 cache_dir aufs /cache3 2048 64 64 cache_mem 1024 MB maximum_object_size 512 MB minimum_object_size 0 request_body_max_size 0 request_header_max_size 64 KB http_port 3128 cache_store_log none strip_query_terms off fqdncache_size 0 coredump_dir / log_icp_queries off
cachemgr output: Squid Object Cache: Version 2.5.STABLE4 Start Time: Fri, 27 Feb 2004 21:14:43 GMT Current Time: Sat, 06 Mar 2004 00:09:24 GMT Connection information for squid: Number of clients accessing cache: 7835 Number of HTTP requests received: 22331805 Number of ICP messages received: 17983480 Number of ICP messages sent: 17937548 Number of queued ICP replies: 0 Request failure ratio: 0.00 Average HTTP requests per minute since start: 2177.7 Average ICP messages per minute since start: 3502.9 Select loop called: 186487837 times, 3.299 ms avg Cache information for squid: Request Hit Ratios: 5min: 40.7%, 60min: 49.7% Byte Hit Ratios: 5min: 11.8%, 60min: 13.0% Request Memory Hit Ratios: 5min: 42.7%, 60min: 24.6% Request Disk Hit Ratios: 5min: 10.3%, 60min: 9.5% Storage Swap size: 5662232 KB Storage Mem size: 1048544 KB Mean Object Size: 16.77 KB Requests given to unlinkd: 0 Median Service Times (seconds) 5 min 60 min: HTTP Requests (All): 0.02742 0.00919 Cache Misses: 0.14252 0.10857 Cache Hits: 0.00179 0.00179 Near Hits: 0.11465 0.08729 Not-Modified Replies: 0.00091 0.00286 DNS Lookups: 0.00372 0.00278 ICP Queries: 0.00249 0.00298 Resource usage for squid: UP Time: 615280.568 seconds CPU Time: 217225.410 seconds CPU Usage: 35.31% CPU Usage, 5 minute avg: 17.16% CPU Usage, 60 minute avg: 35.59% Process Data Segment Size via sbrk(): 1412292 KB Maximum Resident Size: 0 KB Page faults with physical i/o: 85601 Memory usage for squid via mallinfo(): Total space in arena: 1412292 KB Ordinary blocks: 1368846 KB 34984 blks Small blocks: 0 KB 0 blks Holding blocks: 21624 KB 7 blks Free Small blocks: 0 KB Free Ordinary blocks: 43446 KB Total in use: 1390470 KB 97% Total free: 43446 KB 3% Total size: 1433916 KB Memory accounted for: Total accounted: 1274459 KB memPoolAlloc calls: 2886153831 memPoolFree calls: 2880110606 File descriptor usage for squid: Maximum number of file descriptors: 4096 Largest file desc currently in use: 946 Number of file desc currently in use: 352 Files queued for open: 1 Available number of file descriptors: 3743 Reserved number of file descriptors: 100 Store Disk files open: 1 Internal Data Structures: 343410 StoreEntries 217719 StoreEntries with MemObjects 217703 Hot Object Cache Items 337566 on-disk objects
HTH, Peter Smith
James MacLean wrote:
Hi Folks,
We have tried many suggestions that were found in mail-archives and on different sites, but are having a difficult time getting Squid to handle our workload. The obvious example of this to me is if we set "no_cache deny all", we can start going to sites and notice that they keep getting slower.
The Squid server is a 2xP4 Xeon with hyperthreading 2.4Ghz with 1G of RAM. They have SCSI drives which we have tried various cache sizes on. There is
no load on the server before squid begins, but it of course does drive up
the CPU as it starts to churn.
diskd seems to work the best, but once we start using it, the page load delays increase to a point that users are noticing. This setup is used in a transparent proxy via the typical Linux redirect.
In this Educational setting we have approx. 12,000 active student/staff if I accept the "Largest file desc currently in use:" param, so we realize we need some power to deal with the actual disk cache effectiveness. There are actually more clients in total, but this seems to give a good guess of current activity.
Our Internet feed is a 6Mbs max.
Would like to ask if there are any tuning suggestions we can try to boost Squid in this environment?
2.5-STABLE5 cachemgr output follows for a short run :
Connection information for squid: Number of clients accessing cache: 799 Number of HTTP requests received: 261742 Number of ICP messages received: 0 Number of ICP messages sent: 0 Number of queued ICP replies: 0 Request failure ratio: 0.00 Average HTTP requests per minute since start: 5769.8 Average ICP messages per minute since start: 0.0 Select loop called: 8599 times, 316.530 ms avg Cache information for squid: Request Hit Ratios: 5min: 17.9%, 60min: 13.7% Byte Hit Ratios: 5min: 3.8%, 60min: 1.9% Request Memory Hit Ratios: 5min: 25.1%, 60min: 30.8% Request Disk Hit Ratios: 5min: 32.0%, 60min: 24.9% Storage Swap size: 854948 KB Storage Mem size: 128036 KB Mean Object Size: 10.22 KB Requests given to unlinkd: 0 Median Service Times (seconds) 5 min 60 min: HTTP Requests (All): 2.37608 1.71839 Cache Misses: 2.79397 1.91442 Cache Hits: 0.68577 0.61549 Near Hits: 2.37608 1.91442 Not-Modified Replies: 0.64968 0.49576 DNS Lookups: 0.04048 0.03374 ICP Queries: 0.00000 0.00000 Resource usage for squid: UP Time: 2721.839 seconds CPU Time: 506.162 seconds CPU Usage: 18.60% CPU Usage, 5 minute avg: 78.16% CPU Usage, 60 minute avg: 18.46% Process Data Segment Size via sbrk(): 219948 KB Maximum Resident Size: 0 KB Page faults with physical i/o: 0 Memory usage for squid via mallinfo(): Total space in arena: 219948 KB Ordinary blocks: 219855 KB 375 blks Small blocks: 0 KB 0 blks Holding blocks: 6788 KB 3 blks Free Small blocks: 0 KB Free Ordinary blocks: 92 KB Total in use: 226643 KB 100% Total free: 92 KB 0% Total size: 226736 KB Memory accounted for: Total accounted: 208789 KB memPoolAlloc calls: 38921604 memPoolFree calls: 38222649 File descriptor usage for squid: Maximum number of file descriptors: 32768 Largest file desc currently in use: 11111 Number of file desc currently in use: 3641 Files queued for open: 1 Available number of file descriptors: 29126 Reserved number of file descriptors: 100 Store Disk files open: 46 Internal Data Structures: 88190 StoreEntries 17725 StoreEntries with MemObjects 17534 Hot Object Cache Items 83648 on-disk objects
Lines from sample squid.conf:
request_body_max_size 1000 MB httpd_accel_host virtual httpd_accel_port 80 httpd_accel_with_proxy on httpd_accel_uses_host_header on icp_port 0 cache_mem 128 MB cache_swap_low 90 cache_swap_high 95 cache_dir diskd /var/cache/squid/ 2000 64 64 cache_access_log /var/log/squid/access.log cache_log /var/log/squid/cache.log cache_store_log /var/log/squid/store.log pid_filename /var/run/squid.pid debug_options ALL,1 log_fqdn off redirect_children 30 refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern . 0 20% 4320 quick_abort_min 16 KB quick_abort_max 16 KB quick_abort_pct 95 forwarded_for off minimum_direct_hops 3 store_avg_object_size 10 KB fqdncache_size 16536 maximum_object_size_in_memory 8 KB request_header_max_size 1000 KB half_closed_clients off
