On 4/08/2013 7:13 p.m., John Joseph wrote:
Thanks Augustus for the email
my information is
-------------------
[root@proxy squid]# squidclient -h 127.0.0.1 mgr:storedir
HTTP/1.0 200 OK
Server: squid/3.1.10
Mime-Version: 1.0
Date: Sun, 04 Aug 2013 07:01:30 GMT
Content-Type: text/plain
Expires: Sun, 04 Aug 2013 07:01:30 GMT
Last-Modified: Sun, 04 Aug 2013 07:01:30 GMT
X-Cache: MISS from proxy
X-Cache-Lookup: MISS from proxy:3128
Via: 1.0 proxy (squid/3.1.10)
Connection: close
Store Directory Statistics:
Store Entries : 13649421
Maximum Swap Size : 583680000 KB
Current Store Swap Size: 250112280 KB
Current Capacity : 43% used, 57% free
Store Directory #0 (aufs): /opt/var/spool/squid
FS Block Size 4096 Bytes
First level subdirectories: 32
Second level subdirectories: 256
Maximum Size: 583680000 KB
Current Size: 250112280 KB
Percent Used: 42.85%
Filemap bits in use: 13649213 of 16777216 (81%)
Filesystem Space in use: 264249784/854534468 KB (31%)
Filesystem Inodes in use: 13657502/54263808 (25%)
Flags: SELECTED
Removal policy: lru
LRU reference age: 44.69 days
You appear to have a good case there for upgrading to squid-3.2 or later
and adding a rock cache_dir.
As you can see 81% of the Filemap is full. That is the file number codes
Squid uses to internally reference stored objects. There is an absolute
limit of 2^24 (or "1677216" in the above report). That will require an
average object size of 35KB to fill your 557 GB storage area. Your
details earlier said the mean object size actually stored so far was 18KB.
If you add a 50GB rock store alongside that UFS directory you should be
able to double the cached object count.
--------------
and my squid.conf is as
----------------------------------------------
always_direct allow all
cache_log /opt/var/log/squid/cache.log
cache_access_log /opt/var/log/squid/access.log
cache_swap_low 90
cache_swap_high 95
acl manager proto cache_object
acl localhost src 127.0.0.1/32 ::1
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1
acl localnet src 172.16.5.0/24 # RFC1918 possible internal network
acl localnet src 172.17.0.0/22 # RFC1918 possible internal network
acl localnet src 192.168.20.0/24 # RFC1918 possible internal network
acl localnet src fc00::/7 # RFC 4193 local private network range
acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged)
machines
always_direct allow local-servers
You are using always_direct allow all above. This line is never even
being checked.
Also, always_direct has no meaning when there are no cache_peer lines to
be overridden (which is the purpose of always_direct). You can remove
both the always_direct lines to make things a bit faster.
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
acl ipgroup src 172.16.5.1-172.16.5.255/32
acl ipgroup src 172.17.0.10-172.17.3.254/32
delay_pools 1
delay_class 1 2
delay_parameters 1 2560000/3860000 140000/180000
delay_access 1 allow ipgroup
delay_access 1 deny all
http_access allow localnet
http_access allow localhost
http_access allow localnet
http_access allow localhost
You have doubled these rules up.
http_access deny all
http_port 3128 transparent
It is a good idea to always have 3128 listing for regular proxy traffic
and redirecting the intercepted traffic to a separate port. The
interception port is a private detail only relevant to teh NAT
infrastructure doing the redirection and Squid. It can be firewalled to
prevent any access directly to the port.
hierarchy_stoplist cgi-bin ?
cache_dir aufs /opt/var/spool/squid 570000 32 256
coredump_dir /opt/var/spool/squid
maximum_object_size 4 GB
Can you try placing this above the cache_dir line please and see if it
makes any difference?
refresh_pattern -i \.flv$ 10080 90% 999999 ignore-no-cache override-expire
ignore-private
refresh_pattern -i \.(gif|png|jpg|jpeg|ico)$ 10080 90% 43200 override-expire
ignore-no-cache ignore-no-store ignore-private
refresh_pattern -i \.(iso|avi|wav|mp3|mp4|mpeg|swf|flv|x-flv)$ 43200 90% 432000
override-expire ignore-no-cache ignore-no-store ignore-private
refresh_pattern -i \.(deb|rpm|exe|zip|tar|tgz|ram|rar|bin|ppt|doc|tiff)$ 10080
90% 43200 override-expire ignore-no-cache ignore-no-store ignore-private
ignore-private and ignore-no-store are actually VERY bad ideas. No
matter that it looks okay for innocent things like images and archives.
Even those types are used in critical systems from time to time (think
security captchas using images, security certificates exchanged in
compressed archive formats, etc, etc).
Please remove them from the above lines. If you need them at all (eg to
fix a specific identifiable problem URL) it is best to target the regex
pattern to the specific domain or URLs.
In general the CMS systems and dynamic page frameworks use no-cache and
Expires to prevent unnecessary caching and force revalidation - 3.1 is
not fully capable of that but an upgrade to recent 3.2 or later releases
Squid can manage no-cache properly.
refresh_pattern -i \.index.(html|htm)$ 0 40% 10080
refresh_pattern -i \.(html|htm|css|js)$ 1440 40% 40320
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 40% 40320
visible_hostname proxy
This should be an FQDN if possible. The error page icons and similar
things will be reference at an HTTP:// URL using the visible hostname as
domain and Squid forward-proxy port as port number. Your configuration
is probably sending traffic to "http://proxy:3128/";, whatever that
resolves to in the client machine(s).
icap_enable on
icap_preview_enable on
icap_preview_size 4096
icap_persistent_connections on
icap_send_client_ip on
icap_send_client_username on
icap_service qlproxy1 reqmod_precache bypass=0 icap://127.0.0.1:1344/reqmod
icap_service qlproxy2 respmod_precache bypass=0 icap://127.0.0.1:1344/respmod
adaptation_access qlproxy1 allow all
adaptation_access qlproxy2 allow all
------------------------------------------------------------------------
Guidance and advice requested
Thanks for the reply
Joseph John
Amos
