[squid-users] too much TCP_MISS

2011-10-26 Thread benjamin fernandis
Hi All,

I setup squid on centos 6 for cache gain purpose and for that i
configured squid.I am not having any problem while installing and
configuring squid on Linux.

But when i try to check caching is happening or not or checking
traffic in access.log i can see too many TCP_MISS tags.

Even for testing , i try to open same url for 2-3 times , but all time
same  , i can see TCP_MISS only for all requests from that site.
---
squid.conf:

#
# Recommended minimum configuration:
#
acl manager proto cache_object
acl localhost src 127.0.0.1/32 ::1
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1

# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
acl localnet src 172.16.0.0/12  # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src fc00::/7   # RFC 4193 local private network range
acl localnet src fe80::/10  # RFC 4291 link-local (directly
plugged) machines

acl SSL_ports port 443
acl Safe_ports port 80  # http
acl Safe_ports port 21  # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70  # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
acl mynetwork src '/etc/squid/mynetwork'
acl youtube dstdomain .youtube.com
cache allow youtube

#
# Recommended minimum Access Permission configuration:
#
# Only allow cachemgr access from localhost
http_access allow manager localhost
http_access deny manager

# Deny requests to certain unsafe ports
http_access deny !Safe_ports

# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports

# We strongly recommend the following be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on localhost is a local user
#http_access deny to_localhost

#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#

# Example rule allowing access from your local networks.
# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
http_access allow mynetwork
http_access allow localhost

# And finally deny all other access to this proxy
http_access deny all

# Squid normally listens to port 3128
http_port 3128
http_port 3129 intercept
# We recommend you to use at least the following line.
hierarchy_stoplist cgi-bin ?

# Uncomment and adjust the following to add a disk cache directory.
cache_dir aufs /CACHE 307200 64 512

# Leave coredumps in the first cache dir
coredump_dir /CACHE



refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i \.(gif|png|jpg|jpeg|ico)$ 10080 90% 43200
override-expire ignore-no-cache ignore-no-store ignore-private
refresh_pattern -i \.(iso|avi|wav|mp3|mp4|mpeg|swf|flv|x-flv)$ 43200
90% 432000 override-expire ignore-no-cache ignore-no-store
ignore-private
refresh_pattern -i
\.(deb|rpm|exe|zip|tar|tgz|ram|rar|bin|ppt|doc|tiff)$ 10080 90% 43200
override-expire ignore-no-cache ignore-no-store ignore-private
refresh_pattern -i \.index.(html|htm)$ 0 40% 10080
refresh_pattern -i \.(html|htm|css|js)$ 1440 40% 40320
refresh_pattern -i \.flv$ 10080 90% 99 ignore-no-cache
override-expire ignore-private
refresh_pattern -i (/cgi-bin/|\?) 0 0%  0
refresh_pattern . 0 40% 40320


cache_mem 2000 MB
httpd_suppress_version_string on
via off

maximum_object_size_in_memory 40 KB
maximum_object_size 4194304 KB

#maximum_object_size 65536 KB
cache_swap_low 96
cache_swap_high 97
memory_replacement_policy lru
cache_replacement_policy heap LFUDA

quick_abort_min 1024 KB
quick_abort_max 2045 KB
quick_abort_pct 90
positive_dns_ttl 8 hours
negative_dns_ttl 0
ipcache_size 2
ipcache_low 95
ipcache_high 97
fqdncache_size 12000
cache_store_log /var/log/squid/store.log
pipeline_prefetch on



I can only see TCP_MISS.

What could be the reason ?

Is there any misconfiguration or ?

Please guide me to resolve this problem.

squid version: 3.1.8
centos 6 64 bit
kernel version: 2.6.32-71.el6.x86_64


Regards,
Benjamin Fernandis


Re: [squid-users] too much TCP_MISS

2011-10-26 Thread Amos Jeffries

On 26/10/11 19:17, benjamin fernandis wrote:

Hi All,

I setup squid on centos 6 for cache gain purpose and for that i
configured squid.I am not having any problem while installing and
configuring squid on Linux.

But when i try to check caching is happening or not or checking
traffic in access.log i can see too many TCP_MISS tags.

Even for testing , i try to open same url for 2-3 times , but all time
same  , i can see TCP_MISS only for all requests from that site.
---
squid.conf:


snip

acl youtube dstdomain .youtube.com
cache allow youtube



Squid default (no cache lines) is to cache everything it can, unless 
you configure cache deny somehow.


You have no other cache lines than the one cache allow youtube.

The way ACL lines in 3.1 and older are processed there is an implicit 
policy at the end opposite to the result of the final access line.
   = Therefore the implicit default after allow youtube is cache 
deny all.


So allow youtube does nothing but create your problem. Remove and you 
will get a better view of what can be cached and what has problems.



snip

# We recommend you to use at least the following line.
hierarchy_stoplist cgi-bin ?

# Uncomment and adjust the following to add a disk cache directory.
cache_dir aufs /CACHE 307200 64 512


Um, 300GB in a single cache area.

What exactly is your avg object size?

 (2^27) * N = maximum useful cache size.




# Leave coredumps in the first cache dir
coredump_dir /CACHE



refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i \.(gif|png|jpg|jpeg|ico)$ 10080 90% 43200
override-expire ignore-no-cache ignore-no-store ignore-private
refresh_pattern -i \.(iso|avi|wav|mp3|mp4|mpeg|swf|flv|x-flv)$ 43200
90% 432000 override-expire ignore-no-cache ignore-no-store
ignore-private
refresh_pattern -i
\.(deb|rpm|exe|zip|tar|tgz|ram|rar|bin|ppt|doc|tiff)$ 10080 90% 43200
override-expire ignore-no-cache ignore-no-store ignore-private
refresh_pattern -i \.index.(html|htm)$ 0 40% 10080
refresh_pattern -i \.(html|htm|css|js)$ 1440 40% 40320
refresh_pattern -i \.flv$ 10080 90% 99 ignore-no-cache
override-expire ignore-private
refresh_pattern -i (/cgi-bin/|\?) 0 0%  0
refresh_pattern . 0 40% 40320



ignore private and no-store. really?  They are very often marked 
private and never to be saved to disk for good reason. Be very, very 
careful with sharing private web objects to others.


.tiff for example are mostly used by online medical systems for sharing 
things like x-rays and scans results. Critical systems requiring the 
extremely accurate (and private) high definition images.



I suggest removing these again and completely re-assessing whether you 
need any of them once you have the above change to cache lines fixed.




cache_mem 2000 MB
httpd_suppress_version_string on
via off

maximum_object_size_in_memory 40 KB
maximum_object_size 4194304 KB


Go on:
  maximum_object_size 4 GB

snip




I can only see TCP_MISS.

What could be the reason ?

Is there any misconfiguration or ?

Please guide me to resolve this problem.

squid version: 3.1.8
centos 6 64 bit
kernel version: 2.6.32-71.el6.x86_64


Regards,
Benjamin Fernandis


Amos
--
Please be using
  Current Stable Squid 2.7.STABLE9 or 3.1.16
  Beta testers wanted for 3.2.0.13