I apologize for not knowing more about memory usage and squid - but, I just started to
really look at memory and squid since cricket chart shows squid memory steadily grows.
I printed the cache manager current memory usage. There is a lot of data and numbers there. Can you give me any hints at what I need to look at and be concerned with? Or, do I need to look at all entries in current memory usage?
Top just reports that squid is using .05% of system memory. Since system has 1gig of memory and squid doesn't need that much memroy, then system has enough memory.
How do I measure memory usage?
Do I need to look at number of clients, avg http requests per minute, etc. from cache mgr?
Listed is cache manager's General Runtime Info and Memory Utilization.
I thank you for replying and for your help.
-Pat
Squid Object Cache: Version 2.5.STABLE1-20030102
Start Time: Wed, 05 Mar 2003 20:10:04 GMT Current Time: Thu, 27 Mar 2003 16:01:06 GMT
Connection information for squid:
Number of clients accessing cache: 342
Number of HTTP requests received: 11201554
Number of ICP messages received: 0
Number of ICP messages sent: 0
Number of queued ICP replies: 0
Request failure ratio: 0.00%
Average HTTP requests per minute since start: 356.4
Average ICP messages per minute since start: 0.0
Select loop called: 180343978 times, 10.457 ms avg
Cache information for squid:
Request Hit Ratios: 5min: 31.8%, 60min: 37.5%
Byte Hit Ratios: 5min: 21.3%, 60min: 15.0%
Request Memory Hit Ratios: 5min: 7.9%, 60min: 2.6%
Request Disk Hit Ratios: 5min: 37.2%, 60min: 37.7%
Storage Swap size: 1887456 KB
Storage Mem size: 8248 KB
Mean Object Size: 10.35 KB
Requests given to unlinkd: 2480416
Median Service Times (seconds) 5 min 60 min:
HTTP Requests (All): 0.09736 0.09219
Cache Misses: 0.15888 0.18699
Cache Hits: 0.01164 0.00919
Near Hits: 0.12783 0.11465
Not-Modified Replies: 0.00379 0.00767
DNS Lookups: 0.01535 0.01686
ICP Queries: 0.00000 0.00000
Resource usage for squid:
UP Time: 1885862.149 seconds
CPU Time: 69315.550 seconds
CPU Usage: 3.68%
CPU Usage, 5 minute avg: 7.76%
CPU Usage, 60 minute avg: 5.93%
Maximum Resident Size: 0 KB
Page faults with physical i/o: 2447
Memory usage for squid via mallinfo():
Total space in arena: 64956 KB
Ordinary blocks: 37133 KB 3958 blks
Small blocks: 0 KB 0 blks
Holding blocks: 192 KB 1 blks
Free Small blocks: 0 KB
Free Ordinary blocks: 27823 KB
Total in use: 37325 KB 57%
Total free: 27823 KB 43%
Total size: 65148 KB
Memory accounted for:
Total accounted: 30327 KB
memPoolAlloc calls: 1588821210
memPoolFree calls: 1588232248
File descriptor usage for squid:
Maximum number of file descriptors: 1024
Largest file desc currently in use: 265
Number of file desc currently in use: 254
Files queued for open: 0
Available number of file descriptors: 770
Reserved number of file descriptors: 100
Store Disk files open: 1
Internal Data Structures:
182373 StoreEntries
1713 StoreEntries with MemObjects
1700 Hot Object Cache Items
182281 on-disk objects------------------------------------------------------------------------
Current memory usage:
Pool Obj Size Allocated In Use Idle Allocations Saved Hit Rate
(bytes) (#) (KB) high (KB) high (hrs) impact (%total) (#) (KB) high (KB) portion (%alloc) (#) (KB) high (KB) (number) (%num) (%vol) (%num) (number)
2K Buffer 2048 47 94 94 211.98 0 3 6 94 6 44 88 88 49347553 3.10 14.24 100.00 49347600
4K Buffer 4096 24 96 96 326.82 0 0 0 96 0 24 96 96 24315562 1.53 14.03 100.00 24315586
8K Buffer 8192 191 1528 1528 326.82 5 52 416 1528 27 139 1112 1528 7558542 0.48 8.72 100.00 7558733
16K Buffer 16384 1 16 16 497.29 0 0 0 16 0 1 16 16 252 0.00 0.00 99.60 253
Client Socket Buffer 4096 329 1316 1316 168.41 4 275 1100 1316 84 54 216 764 49539337 3.12 28.59 100.00 49539666
acl 48 52 3 3 139.82 0 51 3 3 98 1 1 3 872 0.00 0.00 94.37 924
acl_ip_data 16 82 2 2 24.39 0 81 2 2 99 1 1 2 1343 0.00 0.00 94.25 1425
acl_list 12 102 2 2 24.95 0 101 2 2 99 1 1 2 1707 0.00 0.00 94.36 1809
auth_user_t 60 112 7 7 45.98 0 97 6 7 87 15 1 7 11968 0.00 0.00 99.07 12080
auth_user_hash_pointer 24 112 3 3 45.98 0 97 3 3 87 15 1 3 10513 0.00 0.00 98.95 10625
acl_proxy_auth_match_cache 20 220 5 5 212.08 0 96 2 5 44 124 3 5 21125 0.00 0.00 98.97 21345
acl_user_data 8 2 1 1 524.01 0 1 1 1 50 1 1 1 24 0.00 0.00 92.31 26
dread_ctrl 28 2 1 1 524.01 0 0 0 1 0 2 1 1 5394509 0.34 0.02 100.00 5394511
dwrite_q 24 1 1 1 524.01 0 0 0 1 0 1 1 1 14863213 0.93 0.05 100.00 14863214
FwdServer 12 222 3 3 233.56 0 13 1 3 6 209 3 3 7748922 0.49 0.01 100.00 7749144
HttpReply 112 2031 223 223 209.36 1 1762 193 223 87 269 30 68 23753353 1.49 0.37 99.99 23755384
HttpHeaderEntry 20 17777 348 348 214.83 1 15013 294 348 84 2764 54 127 373852313 23.51 1.05 100.00 373870090
HttpHdrCc 16 739 12 12 214.86 0 261 5 12 35 478 8 11 20473359 1.29 0.05 100.00 20474098
HttpHdrRangeSpec 8 800 7 7 238.80 0 0 0 7 0 800 7 7 107435 0.01 0.00 99.26 108235
HttpHdrRange 12 7 1 1 497.76 0 0 0 1 0 7 1 1 17301 0.00 0.00 99.96 17308
HttpHdrContRange 12 13 1 1 210.33 0 0 0 1 0 13 1 1 13293 0.00 0.00 99.90 13306
intlist 8 5 1 1 524.01 0 5 1 1 100 0 0 1 85 0.00 0.00 94.44 90
MemObject 112 2030 223 223 209.36 1 1762 193 223 87 268 30 68 12827428 0.81 0.20 99.98 12829458
mem_node 4104 2219 8894 8894 326.82 29 2037 8164 8894 92 182 730 1620 26129597 1.64 15.11 99.99 26131816
relist 40 4 1 1 524.01 0 4 1 1 100 0 0 1 68 0.00 0.00 94.44 72
request_t 724 223 158 158 233.56 1 14 10 158 6 209 148 158 11210205 0.71 1.14 100.00 11210428
StoreEntry 48 193453 9069 9069 97.10 30 182806 8570 9069 94 10647 500 1166 9096262 0.57 0.06 97.92 9289715
wordlist 8 348 3 3 47.13 0 10 1 3 3 338 3 3 75526 0.00 0.00 99.54 75874
ClientInfo 236 342 79 79 3.65 0 342 79 79 100 0 0 0 0 0.00 0.00 0.00 342
MD5 digest 16 193453 3023 3023 97.10 10 182806 2857 3023 94 10647 167 389 11764020 0.74 0.03 98.38 11957473
helper_request 12 4 1 1 332.00 0 0 0 1 0 4 1 1 18499 0.00 0.00 99.98 18503
storeSwapTLV 16 4 1 1 524.01 0 0 0 1 0 4 1 1 18707183 1.18 0.04 100.00 18707187
clientRequestBuffer 4096 318 1272 1272 233.55 4 58 232 1272 18 260 1040 1272 3755173 0.24 2.17 99.99 3755491
storeSwapLogData 48 1 1 1 524.01 0 0 0 1 0 1 1 1 5139418 0.32 0.03 100.00 5139419
Short Strings 36 19850 698 698 214.83 2 16835 592 698 85 3015 106 256 405392905 25.50 2.06 100.00 405412755
Medium Strings 128 1671 209 209 166.45 1 861 108 209 52 810 102 184 81918713 5.15 1.48 100.00 81920384
Long Strings 512 553 277 277 134.91 1 89 45 277 16 464 232 276 9044474 0.57 0.65 99.99 9045027
cbdata acl_access (1) 32 59 2 2 140.13 0 58 2 2 98 1 1 2 990 0.00 0.00 94.38 1049
cbdata aclCheck_t (2) 248 9 3 3 477.01 0 1 1 3 11 8 2 3 227832111 14.33 7.96 100.00 227832120
cbdata clientHttpRequest (3) 540 230 122 122 233.56 0 28 15 122 12 202 107 118 11210059 0.71 0.85 100.00 11210289
cbdata ConnStateData (4) 180 318 56 56 233.55 0 58 11 56 18 260 46 56 3755173 0.24 0.10 99.99 3755491
cbdata ErrorState (5) 96 103 10 10 233.47 0 2 1 10 2 101 10 10 520207 0.03 0.01 99.98 520310
cbdata FwdState (6) 52 199 11 11 233.50 0 9 1 11 5 190 10 11 7161463 0.45 0.05 100.00 7161662
cbdata generic_cbdata (7) 20 30 1 1 337.91 0 1 1 1 3 29 1 1 3728932 0.23 0.01 100.00 3728962
cbdata helper (8) 72 1 1 1 524.01 0 1 1 1 100 0 0 0 0 0.00 0.00 0.00 1
cbdata helper_server (9) 88 50 5 5 524.01 0 50 5 5 100 0 0 5 1950 0.00 0.00 97.50 2000
cbdata HttpStateData (12) 60 159 10 10 326.83 0 7 1 10 4 152 9 10 7013951 0.44 0.06 100.00 7014110
cbdata ps_state (14) 136 1 1 1 524.01 0 0 0 1 0 1 1 1 7749143 0.49 0.15 100.00 7749144
cbdata RemovalPolicy (15) 56 2 1 1 524.01 0 2 1 1 100 0 0 0 0 0.00 0.00 0.00 2
cbdata RemovalPolicyWalker (16) 32 1 1 1 514.93 0 0 0 1 0 1 1 1 21 0.00 0.00 95.45 22
cbdata RemovalPurgeWalker (17) 44 1 1 1 524.01 0 0 0 1 0 1 1 1 2025112 0.13 0.01 100.00 2025113
cbdata store_client (18) 72 205 15 15 233.50 0 10 1 15 5 195 14 15 13114506 0.82 0.13 100.00 13114711
event 32 23 1 1 405.51 0 8 1 1 35 15 1 1 3645476 0.23 0.02 100.00 3645499
UFS IO State data 8 36 1 1 336.44 0 1 1 1 3 35 1 1 6213443 0.39 0.01 100.00 6213479
LRU policy node 12 195204 2288 2288 97.12 8 184465 2162 2288 94 10739 126 291 4542537 0.29 0.01 95.88 4737741
cbdata body_size (19) 36 5 1 1 140.13 0 4 1 1 80 1 1 1 70 0.00 0.00 93.33 75
cbdata ConnectStateData (20) 72 172 13 13 235.92 0 2 1 13 1 170 12 13 4951373 0.31 0.05 100.00 4951545
CommWriteStateData 24 76 2 2 326.82 0 1 1 2 1 75 2 2 53669301 3.38 0.18 100.00 53669377
close_handler 12 808 10 10 233.55 0 134 2 10 17 674 8 10 24267474 1.53 0.04 100.00 24268282
ipcache_entry 68 957 64 64 187.64 0 921 62 64 96 36 3 4 1238382 0.08 0.01 99.92 1239339
idns_query 564 26 15 15 336.44 0 0 0 15 0 26 15 15 1239313 0.08 0.10 100.00 1239339
Basic Scheme User Data 20 112 3 3 45.98 0 97 2 3 87 15 1 3 11968 0.00 0.00 99.07 12080
cbdata authenticateStateData (21) 28 4 1 1 332.00 0 0 0 1 0 4 1 1 18499 0.00 0.00 99.98 18503
cbdata RebuildState (24) 616 1 1 1 524.01 0 0 0 1 0 1 1 1 0 0.00 0.00 0.00 1
pconn_data 20 86 2 2 187.63 0 42 1 2 49 44 1 2 2163105 0.14 0.01 100.00 2163191
pconn_fds 32 85 3 3 187.63 0 42 2 3 49 43 2 3 2163106 0.14 0.01 100.00 2163191
Authenticate Request Data 20 230 5 5 233.56 0 27 1 5 12 203 4 5 11118766 0.70 0.03 100.00 11118996
Dlink list nodes 12 230 3 3 233.56 0 27 1 3 12 203 3 3 11118766 0.70 0.02 100.00 11118996
cbdata auth_user_ip_t (26) 36 110 4 4 212.08 0 96 4 4 87 14 1 4 15480 0.00 0.00 99.29 15590
cbdata storeIOState (27) 68 36 3 3 336.44 0 1 1 3 3 35 3 3 6213443 0.39 0.06 100.00 6213479
cbdata SslStateData (28) 72 68 5 5 470.84 0 4 1 5 6 64 5 5 587414 0.04 0.01 99.99 587482
cbdata FtpStateData (29) 8384 15 123 123 498.04 0 0 0 123 0 15 123 123 2133 0.00 0.00 99.30 2148
Total - 636996 30328 30328 3.65 100 591631 25149 27713 83 45365 5179 6600 1589405721 99.96 100.00 99.96 1590042717
Cumulative allocated volume: 709.78 GB Current overhead: 528714 bytes (2.053%) Idle pool limit: 2048.00 MB memPoolAlloc calls: 1590042717 memPoolFree calls: 1589451084
String Pool Impact
(%strings) (%volume) Short Strings 95 80 Medium Strings 5 14 Long Strings 1 6 Other Strings 0 0
------------------------------------------------------------------------
Henrik Nordstrom wrote:
Pat Lendon wrote:
We are using Redhat Linux OS. System has 1 gig of memory. Cache
Manager and top command show squid memory at 5%.
I've read squid faq 8.11 "How much memory do I need in my squid server?"
cache_dir ufs /d1/squid_cache 2048 16 256
/d1 resides on separate file system (18gig of disk, with 7 gig
available). Squid cache is configured to use 2 gig.
10 MB of ram per 2 GB of total cach_dirs would be 20MB
cache_mem 8 MB
Plus additional/extra 20MB does calculate to the 5% (50 meg) of the 1
gig of memory system contains. System appears to have enough memory.
The above configuration should be happy with as little as 128MB with a standard bloated full server installation.. or as little as 64MB if the server is stripped down to not run X or any other unneeded memoryhungry services besides Squid..
Looking at chart enclosed, is it normal for squid memory usage to grow
and grow?
No. The normal is that Squid relatively quickly settles on a stdeady value after it has been in use for a while, and grow a little while the cache is being filled up.
If top says Squid is using 5% of the memory, What is it that you measure when you measure the Squid memory usage?
What causes this?
Good question. If it is Squid who is using this memory additional hints may be provided by looking at the "Memory Utilization" cachemgr page.
Furthermore - I'm also seeing slight delays on system. System has two
AMD Athlon(TM) MP 1800+ processors cpu MHz 1533.431. When system seems
delayed we run top and cpu's are not even at 50%, memory looks fine.
Keep in mind that Squid is a single application and can only make use of one CPU..
Regards Henrik
