Re: [squid-users] Poor cache
On 21/08/2014 11:56 p.m., Délsio Cabá wrote: > Hi, > > I have just update to the latest version, and the results are clear: > cat /var/log/squid/access.log | awk '{print $4}' | sort | uniq -c | sort -rn > 486561 TCP_MISS/200 > 89612 TCP_MISS/304 > 52123 TCP_MEM_HIT/200 > 40408 TCP_MISS/206 > 36267 TCP_MISS/302 > 20904 TCP_MISS/204 > 12246 TCP_IMS_HIT/304 > 12171 TCP_MISS/404 > 10533 TCP_MISS/301 >9145 TCP_MISS/000 >6004 TCP_OFFLINE_HIT/200 > .. > > It's said that MISS/301, MISS/303 are not cacheable without special > instructions. > > What are those SPECIAL instructions? http://tools.ietf.org/html/rfc7234#section-3 301 is a status code defined as cacheable by default. 303 depends on the other conditions. Amos
Re: [squid-users] Poor cache
Hi, I have just update to the latest version, and the results are clear: cat /var/log/squid/access.log | awk '{print $4}' | sort | uniq -c | sort -rn 486561 TCP_MISS/200 89612 TCP_MISS/304 52123 TCP_MEM_HIT/200 40408 TCP_MISS/206 36267 TCP_MISS/302 20904 TCP_MISS/204 12246 TCP_IMS_HIT/304 12171 TCP_MISS/404 10533 TCP_MISS/301 9145 TCP_MISS/000 6004 TCP_OFFLINE_HIT/200 .. It's said that MISS/301, MISS/303 are not cacheable without special instructions. What are those SPECIAL instructions? On 21 August 2014 00:46, Amos Jeffries wrote: > On 21/08/2014 6:05 a.m., Délsio Cabá wrote: >> Hi, >> Using version: Squid Cache: Version 3.1.10 (Centos RPM) >> > > Ah. The version itself is probably most of the prooblem. > > 3.1 does not cache traffic with Cache-Control:no-cache, which these days > consists of a large percentage (30-40) of all traffic. That is resolved > in 3.2 and later, along with better caching of private and authenticated > traffic. > > You can find details of newer CentOS RPM packages from Eliezer at > http://wiki.squid-cache.org/KnowledgeBase/CentOS > > Amos >
Re: [squid-users] Poor cache
On 21/08/2014 6:05 a.m., Délsio Cabá wrote: > Hi, > Using version: Squid Cache: Version 3.1.10 (Centos RPM) > Ah. The version itself is probably most of the prooblem. 3.1 does not cache traffic with Cache-Control:no-cache, which these days consists of a large percentage (30-40) of all traffic. That is resolved in 3.2 and later, along with better caching of private and authenticated traffic. You can find details of newer CentOS RPM packages from Eliezer at http://wiki.squid-cache.org/KnowledgeBase/CentOS Amos
Re: [squid-users] Poor cache
Hi, Using version: Squid Cache: Version 3.1.10 (Centos RPM) I also have this changes on the OS: /etc/rc.local /sbin/modprobe iptable_nat /sbin/modprobe ip_nat_ftp /sbin/modprobe ip_gre /sbin/modprobe ip_conntrack /sbin/modprobe ip_conntrack_ftp echo 0 > /proc/sys/net/ipv4/tcp_syncookies echo 131072 > /proc/sys/net/ipv4/tcp_max_syn_backlog echo 524288 > /proc/sys/net/netfilter/nf_conntrack_max echo 1 > /proc/sys/net/ipv4/ip_forward echo 0 > /proc/sys/net/ipv4/conf/lo/rp_filter modprobe iptable_nat iptables -t nat -F PREROUTING ip tunnel add gre0 mode gre remote 196.10.148.1 local 196.10.148.6 dev eth0 ip link set gre0 up iptables -t nat -F iptables -t nat -A PREROUTING -i gre0 -p tcp -m tcp --dport 80 -j DNAT --to-destination 196.10.148.6:3401 touch /var/lock/subsys/local ~ /etc/sysctl.conf net.ipv6.conf.all.disable_ipv6 = 1 net.ipv6.conf.default.disable_ipv6 = 1 net.ipv4.ip_local_port_range = 1025 65535 fs.file-max = 372925 net.ipv4.tcp_max_syn_backlog = 2048 net.ipv4.tcp_syncookies = 1 net.ipv4.ip_forward = 1 # Controls source route verification net.ipv4.conf.default.rp_filter = 0 net.ipv4.conf.eth0.rp_filter = 0 net.ipv4.conf.eth0.ip_filter = 0 net.ipv4.conf.gre0.rp_filter = 0 net.ipv4.conf.gre0.ip_filter = 0 net.ipv4.conf.default.accept_source_route = 1 # Controls the System Request debugging functionality of the kernel kernel.sysrq = 0 # Controls whether core dumps will append the PID to the core filename. # Useful for debugging multi-threaded applications. kernel.core_uses_pid = 1 # Controls the use of TCP syncookies net.ipv4.tcp_syncookies = 1 # Disable netfilter on bridges. net.bridge.bridge-nf-call-ip6tables = 0 net.bridge.bridge-nf-call-iptables = 0 net.bridge.bridge-nf-call-arptables = 0 On 20 August 2014 09:50, Amos Jeffries wrote: > On 20/08/2014 9:21 a.m., Délsio Cabá wrote: >> Hi guys, >> Need some help on cache. Basically I do not see many caches. >> >> root@c /]# cat /var/log/squid/access.log | awk '{print $4}' | sort | >> uniq -c | sort -rn >> 17403 TCP_MISS/200 >>3107 TCP_MISS/304 > > - objects in the client browser cache were used. > >>1903 TCP_MISS/000 > > - server was contacted but no response came back. This is bad. Seeing > it in such numbers is very bad. > It is a strong sign that TCP window scaling, ECN or ICMP blocking > (Path-MTUd) issues are occuring on your traffic. > > >>1452 TCP_MISS/204 > > - "204 no content" means there was no object to be cached. > >>1421 TCP_MISS/206 > > - Range request responses. Squid cannot cache these yet, but they > should be cached in the client browser and contribute to those 304 > responses above. > >>1186 TCP_MISS/302 > > - along with the MISS/301, MISS/303 these are not cacheable without > special instructions. > >> 659 TCP_MISS/503 >> 641 NONE/400 >> 548 TCP_MISS/301 >> 231 TCP_OFFLINE_HIT/200 > > - cached object used. > >> 189 TCP_MISS/404 >> 126 TCP_IMS_HIT/304 > > - cache object found, but objects in the client browser cache were used. > >> 112 TCP_MISS/504 >> 68 TCP_MISS/401 >> 56 TCP_MEM_HIT/200 > > - cached object used. > >> 50 TCP_SWAPFAIL_MISS/304 > > - cached object found, but disk error occurred loading it. And the > client request was conditional. So object in client browser cache used > instead. > >> 49 TCP_REFRESH_UNMODIFIED/200 > > - cached objects found, mandatory update check required and resulted in > Squid cached object being delivered to client. > >> 46 TCP_SWAPFAIL_MISS/200 >> 39 TCP_MISS/500 >> 36 TCP_MISS/502 >> 34 TCP_REFRESH_UNMODIFIED/304 > > - cached objects found, mandatory update check required and resulted in > client browser cache object being used. > > >> 31 TCP_MISS/403 >> 25 TCP_MISS/400 >> 19 TCP_CLIENT_REFRESH_MISS/200 > > - cached object found, but client request forced a new fetch. > >> 17 TCP_REFRESH_MODIFIED/200 > > - cached object found, mandatory update check resulted in a new object > being used. > >> 11 NONE/417 >> 9 TCP_MISS/303 >> 6 TCP_HIT/000 > > - cached object used, but client disconnected before it could be delivered. > >> 5 TCP_MISS/501 >> 5 TCP_HIT/200 > > - cached object used. > >> 4 TCP_MISS/202 > > - this is usually only seen on POST or PUT. Which are not cacheable by > Squid. > >> 3 TCP_MISS/412 >> 2 TCP_SWAPFAIL_MISS/000 > > - cache object found, but disk error while loading it and the client > disconnected before a server response was found. > >> 2 TCP_MISS/408 >> 1 TCP_MISS/522 >> 1 TCP_MISS/410 >> 1 TCP_MISS/405 >> 1 TCP_CLIENT_REFRESH_MISS/000 > > - cached object found, but client request mandated an update check. > Then client disconnected before that was completed. > > > > All the 4xx and 5xx status responses are only cacheable short term and > only if the server explicitly provides caching information. It looks > like the servers in your traffic are not providing th
Re: [squid-users] Poor cache
On 20/08/2014 9:21 a.m., Délsio Cabá wrote: > Hi guys, > Need some help on cache. Basically I do not see many caches. > > root@c /]# cat /var/log/squid/access.log | awk '{print $4}' | sort | > uniq -c | sort -rn > 17403 TCP_MISS/200 >3107 TCP_MISS/304 - objects in the client browser cache were used. >1903 TCP_MISS/000 - server was contacted but no response came back. This is bad. Seeing it in such numbers is very bad. It is a strong sign that TCP window scaling, ECN or ICMP blocking (Path-MTUd) issues are occuring on your traffic. >1452 TCP_MISS/204 - "204 no content" means there was no object to be cached. >1421 TCP_MISS/206 - Range request responses. Squid cannot cache these yet, but they should be cached in the client browser and contribute to those 304 responses above. >1186 TCP_MISS/302 - along with the MISS/301, MISS/303 these are not cacheable without special instructions. > 659 TCP_MISS/503 > 641 NONE/400 > 548 TCP_MISS/301 > 231 TCP_OFFLINE_HIT/200 - cached object used. > 189 TCP_MISS/404 > 126 TCP_IMS_HIT/304 - cache object found, but objects in the client browser cache were used. > 112 TCP_MISS/504 > 68 TCP_MISS/401 > 56 TCP_MEM_HIT/200 - cached object used. > 50 TCP_SWAPFAIL_MISS/304 - cached object found, but disk error occurred loading it. And the client request was conditional. So object in client browser cache used instead. > 49 TCP_REFRESH_UNMODIFIED/200 - cached objects found, mandatory update check required and resulted in Squid cached object being delivered to client. > 46 TCP_SWAPFAIL_MISS/200 > 39 TCP_MISS/500 > 36 TCP_MISS/502 > 34 TCP_REFRESH_UNMODIFIED/304 - cached objects found, mandatory update check required and resulted in client browser cache object being used. > 31 TCP_MISS/403 > 25 TCP_MISS/400 > 19 TCP_CLIENT_REFRESH_MISS/200 - cached object found, but client request forced a new fetch. > 17 TCP_REFRESH_MODIFIED/200 - cached object found, mandatory update check resulted in a new object being used. > 11 NONE/417 > 9 TCP_MISS/303 > 6 TCP_HIT/000 - cached object used, but client disconnected before it could be delivered. > 5 TCP_MISS/501 > 5 TCP_HIT/200 - cached object used. > 4 TCP_MISS/202 - this is usually only seen on POST or PUT. Which are not cacheable by Squid. > 3 TCP_MISS/412 > 2 TCP_SWAPFAIL_MISS/000 - cache object found, but disk error while loading it and the client disconnected before a server response was found. > 2 TCP_MISS/408 > 1 TCP_MISS/522 > 1 TCP_MISS/410 > 1 TCP_MISS/405 > 1 TCP_CLIENT_REFRESH_MISS/000 - cached object found, but client request mandated an update check. Then client disconnected before that was completed. All the 4xx and 5xx status responses are only cacheable short term and only if the server explicitly provides caching information. It looks like the servers in your traffic are not providing that info (or not correctly). Also, this grep counting does not account for what method the transaction used. Things like the 204 response and 30x responses cacheability depend on what method is involved. So I see 19k MISS and 4k HIT. About 18% hit rate. What version of Squid are you using? Amos
Re: [squid-users] Poor cache
On 08/20/2014 12:21 AM, Délsio Cabá wrote: 3107 TCP_MISS/304 The above is good... It means that the file was not downloaded from the internet\src and was used the local(machine) copy of the file. Eliezer