Re: [squid-users] Poor cache
Hi,
I have just update to the latest version, and the results are clear:
cat /var/log/squid/access.log | awk '{print $4}' | sort | uniq -c | sort -rn
486561 TCP_MISS/200
89612 TCP_MISS/304
52123 TCP_MEM_HIT/200
40408 TCP_MISS/206
36267 TCP_MISS/302
20904 TCP_MISS/204
12246 TCP_IMS_HIT/304
12171 TCP_MISS/404
10533 TCP_MISS/301
9145 TCP_MISS/000
6004 TCP_OFFLINE_HIT/200
..
It's said that MISS/301, MISS/303 are not cacheable without special
instructions.
What are those SPECIAL instructions?
On 21 August 2014 00:46, Amos Jeffries squ...@treenet.co.nz wrote:
On 21/08/2014 6:05 a.m., Délsio Cabá wrote:
Hi,
Using version: Squid Cache: Version 3.1.10 (Centos RPM)
Ah. The version itself is probably most of the prooblem.
3.1 does not cache traffic with Cache-Control:no-cache, which these days
consists of a large percentage (30-40) of all traffic. That is resolved
in 3.2 and later, along with better caching of private and authenticated
traffic.
You can find details of newer CentOS RPM packages from Eliezer at
http://wiki.squid-cache.org/KnowledgeBase/CentOS
Amos
Re: [squid-users] Poor cache
On 21/08/2014 11:56 p.m., Délsio Cabá wrote:
Hi,
I have just update to the latest version, and the results are clear:
cat /var/log/squid/access.log | awk '{print $4}' | sort | uniq -c | sort -rn
486561 TCP_MISS/200
89612 TCP_MISS/304
52123 TCP_MEM_HIT/200
40408 TCP_MISS/206
36267 TCP_MISS/302
20904 TCP_MISS/204
12246 TCP_IMS_HIT/304
12171 TCP_MISS/404
10533 TCP_MISS/301
9145 TCP_MISS/000
6004 TCP_OFFLINE_HIT/200
..
It's said that MISS/301, MISS/303 are not cacheable without special
instructions.
What are those SPECIAL instructions?
http://tools.ietf.org/html/rfc7234#section-3
301 is a status code defined as cacheable by default.
303 depends on the other conditions.
Amos
Re: [squid-users] Poor cache
On 20/08/2014 9:21 a.m., Délsio Cabá wrote:
Hi guys,
Need some help on cache. Basically I do not see many caches.
root@c /]# cat /var/log/squid/access.log | awk '{print $4}' | sort |
uniq -c | sort -rn
17403 TCP_MISS/200
3107 TCP_MISS/304
- objects in the client browser cache were used.
1903 TCP_MISS/000
- server was contacted but no response came back. This is bad. Seeing
it in such numbers is very bad.
It is a strong sign that TCP window scaling, ECN or ICMP blocking
(Path-MTUd) issues are occuring on your traffic.
1452 TCP_MISS/204
- 204 no content means there was no object to be cached.
1421 TCP_MISS/206
- Range request responses. Squid cannot cache these yet, but they
should be cached in the client browser and contribute to those 304
responses above.
1186 TCP_MISS/302
- along with the MISS/301, MISS/303 these are not cacheable without
special instructions.
659 TCP_MISS/503
641 NONE/400
548 TCP_MISS/301
231 TCP_OFFLINE_HIT/200
- cached object used.
189 TCP_MISS/404
126 TCP_IMS_HIT/304
- cache object found, but objects in the client browser cache were used.
112 TCP_MISS/504
68 TCP_MISS/401
56 TCP_MEM_HIT/200
- cached object used.
50 TCP_SWAPFAIL_MISS/304
- cached object found, but disk error occurred loading it. And the
client request was conditional. So object in client browser cache used
instead.
49 TCP_REFRESH_UNMODIFIED/200
- cached objects found, mandatory update check required and resulted in
Squid cached object being delivered to client.
46 TCP_SWAPFAIL_MISS/200
39 TCP_MISS/500
36 TCP_MISS/502
34 TCP_REFRESH_UNMODIFIED/304
- cached objects found, mandatory update check required and resulted in
client browser cache object being used.
31 TCP_MISS/403
25 TCP_MISS/400
19 TCP_CLIENT_REFRESH_MISS/200
- cached object found, but client request forced a new fetch.
17 TCP_REFRESH_MODIFIED/200
- cached object found, mandatory update check resulted in a new object
being used.
11 NONE/417
9 TCP_MISS/303
6 TCP_HIT/000
- cached object used, but client disconnected before it could be delivered.
5 TCP_MISS/501
5 TCP_HIT/200
- cached object used.
4 TCP_MISS/202
- this is usually only seen on POST or PUT. Which are not cacheable by
Squid.
3 TCP_MISS/412
2 TCP_SWAPFAIL_MISS/000
- cache object found, but disk error while loading it and the client
disconnected before a server response was found.
2 TCP_MISS/408
1 TCP_MISS/522
1 TCP_MISS/410
1 TCP_MISS/405
1 TCP_CLIENT_REFRESH_MISS/000
- cached object found, but client request mandated an update check.
Then client disconnected before that was completed.
All the 4xx and 5xx status responses are only cacheable short term and
only if the server explicitly provides caching information. It looks
like the servers in your traffic are not providing that info (or not
correctly).
Also, this grep counting does not account for what method the
transaction used. Things like the 204 response and 30x responses
cacheability depend on what method is involved.
So I see 19k MISS and 4k HIT. About 18% hit rate.
What version of Squid are you using?
Amos
Re: [squid-users] Poor cache
Hi,
Using version: Squid Cache: Version 3.1.10 (Centos RPM)
I also have this changes on the OS:
/etc/rc.local
/sbin/modprobe iptable_nat
/sbin/modprobe ip_nat_ftp
/sbin/modprobe ip_gre
/sbin/modprobe ip_conntrack
/sbin/modprobe ip_conntrack_ftp
echo 0 /proc/sys/net/ipv4/tcp_syncookies
echo 131072 /proc/sys/net/ipv4/tcp_max_syn_backlog
echo 524288 /proc/sys/net/netfilter/nf_conntrack_max
echo 1 /proc/sys/net/ipv4/ip_forward
echo 0 /proc/sys/net/ipv4/conf/lo/rp_filter
modprobe iptable_nat
iptables -t nat -F PREROUTING
ip tunnel add gre0 mode gre remote 196.10.148.1 local 196.10.148.6 dev eth0
ip link set gre0 up
iptables -t nat -F
iptables -t nat -A PREROUTING -i gre0 -p tcp -m tcp --dport 80 -j DNAT
--to-destination 196.10.148.6:3401
touch /var/lock/subsys/local
~
/etc/sysctl.conf
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv4.ip_local_port_range = 1025 65535
fs.file-max = 372925
net.ipv4.tcp_max_syn_backlog = 2048
net.ipv4.tcp_syncookies = 1
net.ipv4.ip_forward = 1
# Controls source route verification
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.eth0.rp_filter = 0
net.ipv4.conf.eth0.ip_filter = 0
net.ipv4.conf.gre0.rp_filter = 0
net.ipv4.conf.gre0.ip_filter = 0
net.ipv4.conf.default.accept_source_route = 1
# Controls the System Request debugging functionality of the kernel
kernel.sysrq = 0
# Controls whether core dumps will append the PID to the core filename.
# Useful for debugging multi-threaded applications.
kernel.core_uses_pid = 1
# Controls the use of TCP syncookies
net.ipv4.tcp_syncookies = 1
# Disable netfilter on bridges.
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0
On 20 August 2014 09:50, Amos Jeffries squ...@treenet.co.nz wrote:
On 20/08/2014 9:21 a.m., Délsio Cabá wrote:
Hi guys,
Need some help on cache. Basically I do not see many caches.
root@c /]# cat /var/log/squid/access.log | awk '{print $4}' | sort |
uniq -c | sort -rn
17403 TCP_MISS/200
3107 TCP_MISS/304
- objects in the client browser cache were used.
1903 TCP_MISS/000
- server was contacted but no response came back. This is bad. Seeing
it in such numbers is very bad.
It is a strong sign that TCP window scaling, ECN or ICMP blocking
(Path-MTUd) issues are occuring on your traffic.
1452 TCP_MISS/204
- 204 no content means there was no object to be cached.
1421 TCP_MISS/206
- Range request responses. Squid cannot cache these yet, but they
should be cached in the client browser and contribute to those 304
responses above.
1186 TCP_MISS/302
- along with the MISS/301, MISS/303 these are not cacheable without
special instructions.
659 TCP_MISS/503
641 NONE/400
548 TCP_MISS/301
231 TCP_OFFLINE_HIT/200
- cached object used.
189 TCP_MISS/404
126 TCP_IMS_HIT/304
- cache object found, but objects in the client browser cache were used.
112 TCP_MISS/504
68 TCP_MISS/401
56 TCP_MEM_HIT/200
- cached object used.
50 TCP_SWAPFAIL_MISS/304
- cached object found, but disk error occurred loading it. And the
client request was conditional. So object in client browser cache used
instead.
49 TCP_REFRESH_UNMODIFIED/200
- cached objects found, mandatory update check required and resulted in
Squid cached object being delivered to client.
46 TCP_SWAPFAIL_MISS/200
39 TCP_MISS/500
36 TCP_MISS/502
34 TCP_REFRESH_UNMODIFIED/304
- cached objects found, mandatory update check required and resulted in
client browser cache object being used.
31 TCP_MISS/403
25 TCP_MISS/400
19 TCP_CLIENT_REFRESH_MISS/200
- cached object found, but client request forced a new fetch.
17 TCP_REFRESH_MODIFIED/200
- cached object found, mandatory update check resulted in a new object
being used.
11 NONE/417
9 TCP_MISS/303
6 TCP_HIT/000
- cached object used, but client disconnected before it could be delivered.
5 TCP_MISS/501
5 TCP_HIT/200
- cached object used.
4 TCP_MISS/202
- this is usually only seen on POST or PUT. Which are not cacheable by
Squid.
3 TCP_MISS/412
2 TCP_SWAPFAIL_MISS/000
- cache object found, but disk error while loading it and the client
disconnected before a server response was found.
2 TCP_MISS/408
1 TCP_MISS/522
1 TCP_MISS/410
1 TCP_MISS/405
1 TCP_CLIENT_REFRESH_MISS/000
- cached object found, but client request mandated an update check.
Then client disconnected before that was completed.
All the 4xx and 5xx status responses are only cacheable short term and
only if the server explicitly provides caching information. It looks
like the servers in your traffic are not providing that info (or not
correctly).
Also, this grep counting does not account for what method the
transaction used. Things like the 204 response and
Re: [squid-users] Poor cache
On 21/08/2014 6:05 a.m., Délsio Cabá wrote: Hi, Using version: Squid Cache: Version 3.1.10 (Centos RPM) Ah. The version itself is probably most of the prooblem. 3.1 does not cache traffic with Cache-Control:no-cache, which these days consists of a large percentage (30-40) of all traffic. That is resolved in 3.2 and later, along with better caching of private and authenticated traffic. You can find details of newer CentOS RPM packages from Eliezer at http://wiki.squid-cache.org/KnowledgeBase/CentOS Amos
Re: [squid-users] Poor cache
On 08/20/2014 12:21 AM, Délsio Cabá wrote: 3107 TCP_MISS/304 The above is good... It means that the file was not downloaded from the internet\src and was used the local(machine) copy of the file. Eliezer
