[squid-users] cache peering between two squid 2.7 + zph + mikrotik
Hello, We are running two squid servers (squid 2.7 stable 9) and want to have cache peering between each other. Both have static real IP addresses and sits in the same LAN. Bandwidth management for users are done by Mikrotik 3.30 which has a WAN IP of the same subnet and also sits in the same LAN as the squid servers. Users are connected from Mikrotik LAN interface running PPPoE servers. We are re-directing HTTP requests from users to the squid server using DNAT to the squid's real IP + port 3128. Also note that we are also using the ZPH facility of Squid to mark cache hit packets in Mikrotik, and allowing unlimited bandwidth for hit objects. Proxy1 squid.conf :- acl localnet src 10.10.0.0/16 10.15.0.0/16 172.16.0.0/16 192.168.0.0/16 acl gateway02 src XXX.XXX.38.248/32 http_access allow gateway02 icp_access allow gateway02 miss_access deny gateway02 tcp_outgoing_tos 0x30 localnet zph_mode tos zph_local 0x30 zph_sibling 0x31 zph_parent 0 zph_option 136 cache_peer XXX.XXX.38.248 sibling 3128 3130 proxy-only and similarly we have Proxy2 squid.conf :- acl localnet src 10.10.0.0/16 10.15.0.0/16 172.16.0.0/16 192.168.0.0/16 acl gateway01 src XXX.XXX.38.245/32 http_access allow gateway01 icp_access allow gateway01 miss_access deny gateway01 tcp_outgoing_tos 0x30 localnet zph_mode tos zph_local 0x30 zph_sibling 0x31 zph_parent 0 zph_option 136 cache_peer XXX.XXX.38.245 sibling 3128 3130 proxy-only Mikrotik Setting :- /ip firewall mangle add action=mark-packet chain=prerouting disabled=no dscp=12 new-packet-mark=proxy-hit passthrough=no comment="Cache Hit Packets" /queue tree add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=pmark packet-mark=proxy-hit parent=global-out priority=8 queue=default Proxy1 is a new machine, and doesn't have any cache objects in it, because it hasn't been in the service yet. Proxy2 is a old machine, and have a substantial large cache inside it. When I use proxy2 to browse and download, I get the proxy2 objects fast because the zph_local is working and mikrotik can also mark the hit packets properly. Again I clear my browser cache, and tried to browse using proxy1, browsed the same websites are already in cache by proxy2, and I get SIBLING_HIT. Here's the log. 1327761343.689 2663 10.15.180.16 TCP_MISS/200 6299190 GET http://sound21.mp3pk.com/indian/jodibreakers/jodi-breakers08(www.songs.pk).mp3 - SIBLING_HIT/XXX.XXX.38.248 udio/mpeg But, its downloading very slowly, and downloading at the speed where I have set the RX/TX limit for that particular package. That means ZPH is not working. To investigate more, I tried to take tcpdump report from proxy1. Using tcpdump -vni eth1 | grep 'tos 0x31' i got zero results. Its not marking anything with 0x31. AGain I used tcpdump -vni eth1 | grep 'tos 0x30' while browsing those websites which are actually cached by proxy2, but accessed by proxy1. I get results like this:- 20:43:25.324668 IP (tos 0x30, ttl 64, id 27401, offset 0, flags [DF], proto TCP (6), length 40) 20:43:25.324814 IP (tos 0x30, ttl 64, id 7570, offset 0, flags [DF], proto TCP (6), length 40) 20:43:25.324834 IP (tos 0x30, ttl 64, id 51042, offset 0, flags [DF], proto TCP (6), length 40) 20:43:25.325749 IP (tos 0x30, ttl 64, id 17589, offset 0, flags [DF], proto TCP (6), length 290) 20:43:25.325845 IP (tos 0x30, ttl 64, id 47388, offset 0, flags [DF], proto TCP (6), length 290) 20:43:25.325928 IP (tos 0x30, ttl 64, id 46825, offset 0, flags [DF], proto TCP (6), length 290) 20:43:25.326010 IP (tos 0x30, ttl 64, id 27402, offset 0, flags [DF], proto TCP (6), length 290) Proxy1 configuration file says that local hits should be 0x30 and sibling hits should be 0x31, so why is proxy1 marking 0x30 for those packets which are actually SIBLING_HITS and not local hits. Even though its marking 0x30 for SIBLING_HIT .. Mikrotik still can't identify the proxy-hit packets, and therefore the download is slow. I may have confused you with lots of information, but the main fact is, I am unable to get the ZPH working with the SIBLING_HIT. Some Additional information : Few months before I used Squid 3.1 in both proxy servers and ZPH did work SIBLING_HIT. But its not working working in Squid 2.7 Please advise. Thanks & Regards, Saiful
[squid-users] cache peering between two squid 2.7 + zph + mikrotik
Hello, We are running two squid servers (squid 2.7 stable 9) and want to have cache peering between each other. Both have static real IP addresses and sits in the same LAN. Bandwidth management for users are done by Mikrotik 3.30 which has a WAN IP of the same subnet and also sits in the same LAN as the squid servers. Users are connected from Mikrotik LAN interface running PPPoE servers. We are re-directing HTTP requests from users to the squid server using DNAT to the squid's real IP + port 3128. Also note that we are also using the ZPH facility of Squid to mark cache hit packets in Mikrotik, and allowing unlimited bandwidth for hit objects. Proxy1 squid.conf :- acl localnet src 10.10.0.0/16 10.15.0.0/16 172.16.0.0/16 192.168.0.0/16 acl gateway02 src XXX.XXX.38.248/32 http_access allow gateway02 icp_access allow gateway02 miss_access deny gateway02 tcp_outgoing_tos 0x30 localnet zph_mode tos zph_local 0x30 zph_sibling 0x31 zph_parent 0 zph_option 136 cache_peer XXX.XXX.38.248 sibling 3128 3130 proxy-only and similarly we have Proxy2 squid.conf :- acl localnet src 10.10.0.0/16 10.15.0.0/16 172.16.0.0/16 192.168.0.0/16 acl gateway01 src XXX.XXX.38.245/32 http_access allow gateway01 icp_access allow gateway01 miss_access deny gateway01 tcp_outgoing_tos 0x30 localnet zph_mode tos zph_local 0x30 zph_sibling 0x31 zph_parent 0 zph_option 136 cache_peer XXX.XXX.38.245 sibling 3128 3130 proxy-only Mikrotik Setting :- /ip firewall mangle add action=mark-packet chain=prerouting disabled=no dscp=12 new-packet-mark=proxy-hit passthrough=no comment="Cache Hit Packets" /queue tree add burst-limit=0 burst-threshold=0 burst-time=0s disabled=no limit-at=0 max-limit=0 name=pmark packet-mark=proxy-hit parent=global-out priority=8 queue=default Proxy1 is a new machine, and doesn't have any cache objects in it, because it hasn't been in the service yet. Proxy2 is a old machine, and have a substantial large cache inside it. When I use proxy2 to browse and download, I get the proxy2 objects fast because the zph_local is working and mikrotik can also mark the hit packets properly. Again I clear my browser cache, and tried to browse using proxy1, browsed the same websites are already in cache by proxy2, and I get SIBLING_HIT. Here's the log. 1327761343.689 2663 10.15.180.16 TCP_MISS/200 6299190 GET http://sound21.mp3pk.com/indian/jodibreakers/jodi-breakers08(www.songs.pk).mp3 - SIBLING_HIT/XXX.XXX.38.248 audio/mpeg But, its downloading very slowly, and downloading at the speed where I have set the RX/TX limit for that particular package. That means ZPH is not working. To investigate more, I tried to take tcpdump report from proxy1. Using tcpdump -vni eth1 | grep 'tos 0x31' i got zero results. Its not marking anything with 0x31. AGain I used tcpdump -vni eth1 | grep 'tos 0x30' while browsing those websites which are actually cached by proxy2, but accessed by proxy1. I get results like this:- 20:43:25.324668 IP (tos 0x30, ttl 64, id 27401, offset 0, flags [DF], proto TCP (6), length 40) 20:43:25.324814 IP (tos 0x30, ttl 64, id 7570, offset 0, flags [DF], proto TCP (6), length 40) 20:43:25.324834 IP (tos 0x30, ttl 64, id 51042, offset 0, flags [DF], proto TCP (6), length 40) 20:43:25.325749 IP (tos 0x30, ttl 64, id 17589, offset 0, flags [DF], proto TCP (6), length 290) 20:43:25.325845 IP (tos 0x30, ttl 64, id 47388, offset 0, flags [DF], proto TCP (6), length 290) 20:43:25.325928 IP (tos 0x30, ttl 64, id 46825, offset 0, flags [DF], proto TCP (6), length 290) 20:43:25.326010 IP (tos 0x30, ttl 64, id 27402, offset 0, flags [DF], proto TCP (6), length 290) Proxy1 configuration file says that local hits should be 0x30 and sibling hits should be 0x31, so why is proxy1 marking 0x30 for those packets which are actually SIBLING_HITS and not local hits. Even though its marking 0x30 for SIBLING_HIT .. Mikrotik still can't identify the proxy-hit packets, and therefore the download is slow. I may have confused you with lots of information, but the main fact is, I am unable to get the ZPH working with the SIBLING_HIT. Some Additional information : Few months before I used Squid 3.1 in both proxy servers and ZPH did work SIBLING_HIT. But its not working working in Squid 2.7 Please advise. Thanks & Regards, Saiful
[squid-users] Problems with Internet Download Manager and Squid 2.7 Stable 9
Hi, I am using Squid 2.7 Stable on Ubuntu 10.10 x64 Files like mp3 which have a refresh_pattern defined, and downloaded within the browser download manager is cached, but if I download the file with Internet Download Manager 6.05, the file is not cached. Note that IDM by default uses 8 connections to download a single file. Again, if I reduce the default connection number to 1, and try to download again with IDM, then the file is cached instantly. Most users in our network have IDM as their primary download manager, and if we can't cache objects downloaded with IDM, then : Any help or ideas are welcomed. Regards, Saiful
RE: [squid-users] Some pages loading very slow in 3.1.10 Stable
Disabled ipv6 from the OS and also from Squid. Extended RAM to 300MB (Double than before). And browsing speed/response is much faster than before. But www.music.com.bd and www.djmaza.com is still like before. What should I do? Regards, Saiful > From: saifu...@hotmail.com > To: squid-users@squid-cache.org > Date: Tue, 25 Jan 2011 17:17:30 +1100 > Subject: RE: [squid-users] Some pages loading very slow in 3.1.10 Stable > > > As advised earlier, I have disabled ipv6 support while recompiling squid, but > the problem still exist. Do you want me to disable ipv6 in the OS and try > again? > > Regards, > Saiful > > > > Date: Mon, 24 Jan 2011 18:39:42 -0200 > > From: marcus.k...@urlfilterdb.com > > To: squ...@treenet.co.nz > > CC: squid-users@squid-cache.org > > Subject: Re: [squid-users] Some pages loading very slow in 3.1.10 Stable > > > > > > > > Amos Jeffries wrote: > > > On 24/01/11 23:09, Michael Hendrie wrote: > > >> > > >> On 24/01/2011, at 8:17 PM, Saiful Alam wrote: > > >> > > >>> > > >>> OK I have kept your suggestion in my mind, but right now I'm not in > > >>> a position to buy two HDD's. May be I can afford to buy 15 days > > >>> later. For the time being, my prime problem is the loading of two > > >>> major sites from where my users download mp3. Those are > > >>> > > >>> www.music.com.bd and www.djmaza.com > > >>> > > >> > > >> Seems to load fine for me but that doesn't mean your slow = my fine. > > >> > > >> I had issues with some random sites being "slow" with 3.1.10 and > > >> tracked it down to squid trying to get records for the problem > > >> sites (or objects pulled from other sites). Not sure why this was > > >> occurring as IPv6 is not enabled on the OS. I didn't investigate too > > >> much and just recompiled with --disable-ipv6 as it wasn't needed. > > >> Doing so resolved my slow sites issue. > > >> > > > > > > Seems like you actually had IPv6 partially enabled in the OS, and maybe > > > a break in DNS or MTU. > > > > > > When Squid 3.1.10 starts up it probes the OS network capabilities to see > > > if IPv6 connections can be made. When they are possible it enables > > > things like to use those connections. --disable-ipv6 merely sets > > > the result of that test to always be false. > > > > > > With a reasonably fast DNS response time (under a half second) > > > lookups will not be noticeable. > > > > I am one of those who live in Brazil and most lookups are slow > > since the international lines from Brazil to the USA are > > not properly sized. > > > > The log of the DNS server shows that the lookup for the A record is > > always preceded by the lookup of the record. > > > > What is the point of doing record lookups and trying to use the result > > and then reverting to A record lookups and succeeding with the connect() ? > > It is a lot of overhead for systems connected to slowish WANs and > > systems with *many* connections. > > The latter category might get a performance increase if Squid > > automagically detects that there is no IPv6 router or uses > > a new configuration option 'network_has_ipv6_router'. > > > > I did not find options to configure bind/named to ignore lookups either > > so I would love to see Squid have the new option. > > > > > With working MTU there will be almost zero lag from opening and > > > attempting IPv6 connections on an IPv4-only network. > > > > > >> > > >>> Don't know the reason, but music.com.bd loads very slow. And in > > >>> firebug i see that the problem persists while loading 3 ads from > > >>> ads.clicksor.com and some facebook widgets. Can you please check > > > > > > There you have the problem by the looks of it. > > > > > > Ad servers are very bad for being slow. They usually do a lot of > > > processing or slow operations in the background before replying. Due to > > > their tracking desires they do not permit proxies to cache and speed up > > > their results. Some are more noticeable than others. > > > > > > Facebook is designed in a similar way which also suffers from these > > > heavy processing delay problems on the APIs. But they do seem to be > > > emitting useful cache controls on the static bits to avoid that. > > > > > > You have a choice: > > > put up with it > > > or > > > block those URL from being fetched. > > > > > >>> and try to load these two domains if you're running a Squid 3.1.X > > >>> version and see if everything is alright from your end. > > >>> > > > > > > Amos >
RE: [squid-users] Some pages loading very slow in 3.1.10 Stable
As advised earlier, I have disabled ipv6 support while recompiling squid, but the problem still exist. Do you want me to disable ipv6 in the OS and try again? Regards, Saiful > Date: Mon, 24 Jan 2011 18:39:42 -0200 > From: marcus.k...@urlfilterdb.com > To: squ...@treenet.co.nz > CC: squid-users@squid-cache.org > Subject: Re: [squid-users] Some pages loading very slow in 3.1.10 Stable > > > > Amos Jeffries wrote: > > On 24/01/11 23:09, Michael Hendrie wrote: > >> > >> On 24/01/2011, at 8:17 PM, Saiful Alam wrote: > >> > >>> > >>> OK I have kept your suggestion in my mind, but right now I'm not in > >>> a position to buy two HDD's. May be I can afford to buy 15 days > >>> later. For the time being, my prime problem is the loading of two > >>> major sites from where my users download mp3. Those are > >>> > >>> www.music.com.bd and www.djmaza.com > >>> > >> > >> Seems to load fine for me but that doesn't mean your slow = my fine. > >> > >> I had issues with some random sites being "slow" with 3.1.10 and > >> tracked it down to squid trying to get records for the problem > >> sites (or objects pulled from other sites). Not sure why this was > >> occurring as IPv6 is not enabled on the OS. I didn't investigate too > >> much and just recompiled with --disable-ipv6 as it wasn't needed. > >> Doing so resolved my slow sites issue. > >> > > > > Seems like you actually had IPv6 partially enabled in the OS, and maybe > > a break in DNS or MTU. > > > > When Squid 3.1.10 starts up it probes the OS network capabilities to see > > if IPv6 connections can be made. When they are possible it enables > > things like to use those connections. --disable-ipv6 merely sets > > the result of that test to always be false. > > > > With a reasonably fast DNS response time (under a half second) > > lookups will not be noticeable. > > I am one of those who live in Brazil and most lookups are slow > since the international lines from Brazil to the USA are > not properly sized. > > The log of the DNS server shows that the lookup for the A record is > always preceded by the lookup of the record. > > What is the point of doing record lookups and trying to use the result > and then reverting to A record lookups and succeeding with the connect() ? > It is a lot of overhead for systems connected to slowish WANs and > systems with *many* connections. > The latter category might get a performance increase if Squid > automagically detects that there is no IPv6 router or uses > a new configuration option 'network_has_ipv6_router'. > > I did not find options to configure bind/named to ignore lookups either > so I would love to see Squid have the new option. > > > With working MTU there will be almost zero lag from opening and > > attempting IPv6 connections on an IPv4-only network. > > > >> > >>> Don't know the reason, but music.com.bd loads very slow. And in > >>> firebug i see that the problem persists while loading 3 ads from > >>> ads.clicksor.com and some facebook widgets. Can you please check > > > > There you have the problem by the looks of it. > > > > Ad servers are very bad for being slow. They usually do a lot of > > processing or slow operations in the background before replying. Due to > > their tracking desires they do not permit proxies to cache and speed up > > their results. Some are more noticeable than others. > > > > Facebook is designed in a similar way which also suffers from these > > heavy processing delay problems on the APIs. But they do seem to be > > emitting useful cache controls on the static bits to avoid that. > > > > You have a choice: > > put up with it > > or > > block those URL from being fetched. > > > >>> and try to load these two domains if you're running a Squid 3.1.X > >>> version and see if everything is alright from your end. > >>> > > > > Amos
RE: [squid-users] Some pages loading very slow in 3.1.10 Stable
it tries to load, but I am not 100% sure. > > Can anyone try to load the site again and see if the problem persists, or how > should I exempt those URLs from being caching. > > I think the URLS are :- > 1) http://ads.clicksor.com/showAd.php?pid=102683&adtype=1&sid=152482&zone= > 2) http://ads.clicksor.com/showAd.php?pid=102683&adtype=9&sid=152482&zone= > 3) > http://www.facebook.com/ajax/connect/activity_widget.php?__a=1&site=www.music.com.bd&width=350&height=225&header=false&colorscheme=light&border_color=%23006a4e&post_form_id=7c21c99b9bd83930bb871649bf465fcf&user=50560&nb_activities=5&newest=0 > > URL 3 waiting time sometimes goes more than 800ms. Please advise. > > Regards, > Shishir > > > > > Date: Tue, 25 Jan 2011 01:33:10 +1300 > > From: squ...@treenet.co.nz > > To: squid-users@squid-cache.org > > Subject: Re: [squid-users] Some pages loading very slow in 3.1.10 Stable > > > > On 24/01/11 23:09, Michael Hendrie wrote: > > > > > > On 24/01/2011, at 8:17 PM, Saiful Alam wrote: > > > > > >> > > >> OK I have kept your suggestion in my mind, but right now I'm not in > > >> a position to buy two HDD's. May be I can afford to buy 15 days > > >> later. For the time being, my prime problem is the loading of two > > >> major sites from where my users download mp3. Those are > > >> > > >> www.music.com.bd and www.djmaza.com > > >> > > > > > > Seems to load fine for me but that doesn't mean your slow = my fine. > > > > > > I had issues with some random sites being "slow" with 3.1.10 and > > > tracked it down to squid trying to get records for the problem > > > sites (or objects pulled from other sites). Not sure why this was > > > occurring as IPv6 is not enabled on the OS. I didn't investigate too > > > much and just recompiled with --disable-ipv6 as it wasn't needed. > > > Doing so resolved my slow sites issue. > > > > > > > Seems like you actually had IPv6 partially enabled in the OS, and maybe > > a break in DNS or MTU. > > > > When Squid 3.1.10 starts up it probes the OS network capabilities to see > > if IPv6 connections can be made. When they are possible it enables > > things like to use those connections. --disable-ipv6 merely sets > > the result of that test to always be false. > > > > With a reasonably fast DNS response time (under a half second) > > lookups will not be noticeable. > > > > With working MTU there will be almost zero lag from opening and > > attempting IPv6 connections on an IPv4-only network. > > > > > > > >> Don't know the reason, but music.com.bd loads very slow. And in > > >> firebug i see that the problem persists while loading 3 ads from > > >> ads.clicksor.com and some facebook widgets. Can you please check > > > > There you have the problem by the looks of it. > > > > Ad servers are very bad for being slow. They usually do a lot of > > processing or slow operations in the background before replying. Due to > > their tracking desires they do not permit proxies to cache and speed up > > their results. Some are more noticeable than others. > > > > Facebook is designed in a similar way which also suffers from these > > heavy processing delay problems on the APIs. But they do seem to be > > emitting useful cache controls on the static bits to avoid that. > > > > You have a choice: > > put up with it > > or > > block those URL from being fetched. > > > > >> and try to load these two domains if you're running a Squid 3.1.X > > >> version and see if everything is alright from your end. > > >> > > > > Amos > > -- > > Please be using > > Current Stable Squid 2.7.STABLE9 or 3.1.10 > > Beta testers wanted for 3.2.0.4 >
RE: [squid-users] Some pages loading very slow in 3.1.10 Stable
Hi Amor, I have recompiled Squid with the following parameters. ./configure --prefix=/usr/local/squid --enable-async-io=8 --enable-storeio="ufs,aufs,diskd" --enable-removal-policies="lru,heap" --enable-delay-pools --enable-cache-digests --enable-underscores --enable-icap-client --enable-follow-x-forwarded-for --enable-arp-acl --enable-esi --enable-zph-qos --disable-translation --with-large-files --with-filedescriptors=65536 --disable-ipv6 --enable-linux-netfilter DISK I/O is just like before, on average 5% I would say. I am using the default removal policy, not heap. Well after recompiling it, I can see the Median DNS Lookup time reduced by 10 times are the browsing experience is improved than before. Thank you for that. But those two domains i.e www.music.com.bd and www.djmaza.com problem is not solved. I cannot determine exactly what is stopping it to load the site fast, indeed it loads may be 10 minutes later. To my (amateur) understanding, the problem is with the ads.clicksor.com and the facebook widgets it tries to load, but I am not 100% sure. Can anyone try to load the site again and see if the problem persists, or how should I exempt those URLs from being caching. I think the URLS are :- 1) http://ads.clicksor.com/showAd.php?pid=102683&adtype=1&sid=152482&zone= 2) http://ads.clicksor.com/showAd.php?pid=102683&adtype=9&sid=152482&zone= 3) http://www.facebook.com/ajax/connect/activity_widget.php?__a=1&site=www.music.com.bd&width=350&height=225&header=false&colorscheme=light&border_color=%23006a4e&post_form_id=7c21c99b9bd83930bb871649bf465fcf&user=50560&nb_activities=5&newest=0 URL 3 waiting time sometimes goes more than 800ms. Please advise. Regards, Shishir > Date: Tue, 25 Jan 2011 01:33:10 +1300 > From: squ...@treenet.co.nz > To: squid-users@squid-cache.org > Subject: Re: [squid-users] Some pages loading very slow in 3.1.10 Stable > > On 24/01/11 23:09, Michael Hendrie wrote: > > > > On 24/01/2011, at 8:17 PM, Saiful Alam wrote: > > > >> > >> OK I have kept your suggestion in my mind, but right now I'm not in > >> a position to buy two HDD's. May be I can afford to buy 15 days > >> later. For the time being, my prime problem is the loading of two > >> major sites from where my users download mp3. Those are > >> > >> www.music.com.bd and www.djmaza.com > >> > > > > Seems to load fine for me but that doesn't mean your slow = my fine. > > > > I had issues with some random sites being "slow" with 3.1.10 and > > tracked it down to squid trying to get records for the problem > > sites (or objects pulled from other sites). Not sure why this was > > occurring as IPv6 is not enabled on the OS. I didn't investigate too > > much and just recompiled with --disable-ipv6 as it wasn't needed. > > Doing so resolved my slow sites issue. > > > > Seems like you actually had IPv6 partially enabled in the OS, and maybe > a break in DNS or MTU. > > When Squid 3.1.10 starts up it probes the OS network capabilities to see > if IPv6 connections can be made. When they are possible it enables > things like to use those connections. --disable-ipv6 merely sets > the result of that test to always be false. > > With a reasonably fast DNS response time (under a half second) > lookups will not be noticeable. > > With working MTU there will be almost zero lag from opening and > attempting IPv6 connections on an IPv4-only network. > > > > >> Don't know the reason, but music.com.bd loads very slow. And in > >> firebug i see that the problem persists while loading 3 ads from > >> ads.clicksor.com and some facebook widgets. Can you please check > > There you have the problem by the looks of it. > > Ad servers are very bad for being slow. They usually do a lot of > processing or slow operations in the background before replying. Due to > their tracking desires they do not permit proxies to cache and speed up > their results. Some are more noticeable than others. > > Facebook is designed in a similar way which also suffers from these > heavy processing delay problems on the APIs. But they do seem to be > emitting useful cache controls on the static bits to avoid that. > > You have a choice: > put up with it > or > block those URL from being fetched. > > >> and try to load these two domains if you're running a Squid 3.1.X > >> version and see if everything is alright from your end. > >> > > Amos > -- > Please be using > Current Stable Squid 2.7.STABLE9 or 3.1.10 > Beta testers wanted for 3.2.0.4
RE: [squid-users] Some pages loading very slow in 3.1.10 Stable
OK I have kept your suggestion in my mind, but right now I'm not in a position
to buy two HDD's. May be I can afford to buy 15 days later. For the time being,
my prime problem is the loading of two major sites from where my users download
mp3. Those are
www.music.com.bd and www.djmaza.com
Don't know the reason, but music.com.bd loads very slow. And in firebug i see
that the problem persists while loading 3 ads from ads.clicksor.com and some
facebook widgets. Can you please check and try to load these two domains if
you're running a Squid 3.1.X version and see if everything is alright from your
end.
Regards,
Saiful
> Date: Mon, 24 Jan 2011 11:41:04 +0200
> From: elie...@ec.hadorhabaac.com
> To: squid-users@squid-cache.org
> Subject: Re: [squid-users] Some pages loading very slow in 3.1.10 Stable
>
> are you kdding me?
>
>
> 500 clients?
>
> if most of the clients are just doing almost nothing just downloading
> one page of 2 MB..
>
> how much is your HD I\O ?? in this case?(in speed MBps) ?
>
> so first.. change the UFS to AUFS you dont need to do anything to the
> cache it self cause it the same system just with Async options on.
> and just buy two more of these 500GB drives and put all three of them in
> raid 0 or 5.
> or first try 0 on two and then add another one to then if it goes smoothly.
> this will give you a lot more speed.
> i dont now the cause but it looks like or connectivity/dns or I\O problem.
>
> i'v tried to use iptraf but ifstat just gives you the numbers of every
> Interface you have in aginst out traffic meter in a simple way.
>
>
> On 24/01/2011 11:02, Saiful Alam wrote:
>
> >
> >>>> TRIED AUFS, but didn't get better performance, while
> >>>>
> > researching in the web, I read everywhere that AUFS is better than UFS
> > in terms of performance,but I don't know why I get bad performance with
> > this.
> > Processor is Intel(R) Core(TM) i7 CPU 870 @ 2.93GHz
> > Motherboard is Intel® Desktop Board DH55TC
> >
> >
> >
> > RAM is 2x4GB=8GB DDR3
> > DISK 1 = 250GB Hitachi SATA HDP72502 {[USED ONLY FOR UBUNTU SYSTEM& BOOT]}
> > DISK 2 = 500GB Hitachi SATA HDS72105 {[USED FOR CACHE DIRECTORIES ONLY,
> > MOUNTED ON /MEDIA/CACHE FILESYSTEM EXT4]}
> >
> > Normally my Disk I/O never goes more than 15% and I would say the average
> > is about 3-4%.
> > For bandwidth monitoring I usually see iptraf which is also good, but
> > surely I'll try ifstat next time.
> > At peak hours (which is between 10pm - 2am GMT +0600), we have around 500
> > clients connected (approx)
> >
> > I have tried apt-get install squid3 (which is the default 3.1.6 in apt
> > repository) and found the performance of 3.1.10 (my custom configuration)
> > is better than the 3.1.6.
> >
> > Regards,
> > Saiful
> >
> >
> >
> >
> >> Date: Mon, 24 Jan 2011 10:36:14 +0200
> >> From: elie...@ec.hadorhabaac.com
> >> To: squid-users@squid-cache.org
> >> Subject: Re: [squid-users] Some pages loading very slow in 3.1.10 Stable
> >>
> >> It's a small peace and most of the answers are not really suppose to there.
> >>
> >>
> >> the first thing is that your cache is not just cache it's a store house..
> >>
> >> it;'s not bad but you can try to change the ufs to aufs..
> >>
> >> can get better performance.
> >>
> >> what are the specification of the machine?
> >>
> >> core i7 ? ram? what disk? array of disks?
> >>
> >> did you tried to ping from the machine or WGET?
> >>
> >> if it's debian you can install ifstat that can give you real-rime
> >> bandwidth usage and it might be cause of something else that is not
> >> related to squid..
> >>
> >> if you have even like 25 clients downloading obsessively mp3 files for
> >> like 10 or more minutes in this time your I\O of your hard drive will
> >> rise..
> >>
> >> also..
> >>
> >> you can monitor the access to the squid folders onfly while you have the
> >> problem and to understand what is causing it..
> >>
> >> if it's CPU load or DISK I\O load .. or other stuff.
> >>
> >> by the way.. can you try the ubuntu squid3 stable?
> >>
> >> im using squid3 stable on ubuntu 10.04 on an Intel Atom D450 machine
> >> with cache of 40GB and it's taking the load very nicely.
> >>
&
RE: [squid-users] Some pages loading very slow in 3.1.10 Stable
>>>TRIED AUFS, but didn't get better performance, while
researching in the web, I read everywhere that AUFS is better than UFS
in terms of performance,but I don't know why I get bad performance with
this.
Processor is Intel(R) Core(TM) i7 CPU 870 @ 2.93GHz
Motherboard is Intel® Desktop Board DH55TC
RAM is 2x4GB=8GB DDR3
DISK 1 = 250GB Hitachi SATA HDP72502 {[USED ONLY FOR UBUNTU SYSTEM & BOOT]}
DISK 2 = 500GB Hitachi SATA HDS72105 {[USED FOR CACHE DIRECTORIES ONLY, MOUNTED
ON /MEDIA/CACHE FILESYSTEM EXT4]}
Normally my Disk I/O never goes more than 15% and I would say the average is
about 3-4%.
For bandwidth monitoring I usually see iptraf which is also good, but surely
I'll try ifstat next time.
At peak hours (which is between 10pm - 2am GMT +0600), we have around 500
clients connected (approx)
I have tried apt-get install squid3 (which is the default 3.1.6 in apt
repository) and found the performance of 3.1.10 (my custom configuration) is
better than the 3.1.6.
Regards,
Saiful
> Date: Mon, 24 Jan 2011 10:36:14 +0200
> From: elie...@ec.hadorhabaac.com
> To: squid-users@squid-cache.org
> Subject: Re: [squid-users] Some pages loading very slow in 3.1.10 Stable
>
> It's a small peace and most of the answers are not really suppose to there.
>
>
> the first thing is that your cache is not just cache it's a store house..
>
> it;'s not bad but you can try to change the ufs to aufs..
>
> can get better performance.
>
> what are the specification of the machine?
>
> core i7 ? ram? what disk? array of disks?
>
> did you tried to ping from the machine or WGET?
>
> if it's debian you can install ifstat that can give you real-rime
> bandwidth usage and it might be cause of something else that is not
> related to squid..
>
> if you have even like 25 clients downloading obsessively mp3 files for
> like 10 or more minutes in this time your I\O of your hard drive will rise..
>
> also..
>
> you can monitor the access to the squid folders onfly while you have the
> problem and to understand what is causing it..
>
> if it's CPU load or DISK I\O load .. or other stuff.
>
> by the way.. can you try the ubuntu squid3 stable?
>
> im using squid3 stable on ubuntu 10.04 on an Intel Atom D450 machine
> with cache of 40GB and it's taking the load very nicely.
>
>
>
>
> On 24/01/2011 09:12, Saiful Alam wrote:
>
> > Some results of TCPDUMP in -vv mode.
> >
> > 13:12:04.191180 IP (tos 0x0, ttl 127, id 2750, offset 0, flags [DF], proto
> > TCP (6), length 40)
> > 172.16.80.2.1155> 77.67.29.42.www: Flags [.], cksum 0x6de4 (correct), seq
> > 1127903567, ack 4192021369, win 64700, length 0
> > 13:12:04.192822 IP (tos 0x0, ttl 64, id 4692, offset 0, flags [DF], proto
> > TCP (6), length 823)
> > www-12-02-snc5.facebook.com.www> 10.16.63.123.3714: Flags [P.], cksum
> > 0xc789 (correct), seq 4235393899:4235394682, ack 3661185770, win 14600,
> > length 783
> > 13:12:04.831628 IP (tos 0x0, ttl 127, id 29487, offset 0, flags [DF], proto
> > TCP (6), length 40)
> > 172.16.80.6.rmiregistry> cds174.lax.llnw.net.www: Flags [.], cksum 0xfa45
> > (correct), seq 0, ack 460801, win 65535, length 0
> > 13:12:05.355300 IP (tos 0x0, ttl 127, id 2265, offset 0, flags [DF], proto
> > TCP (6), length 40)
> > 192.168.96.165.57362> 195.122.152.64.www: Flags [.], cksum 0xfc6c
> > (correct), seq 1875916651, ack 4162436021, win 64240, length 0
> > 13:12:05.355322 IP (tos 0x0, ttl 64, id 18785, offset 0, flags [DF], proto
> > TCP (6), length 1500)
> > 195.122.152.64.www> 192.168.96.165.57362: Flags [.], cksum 0x0f8f
> > (correct), seq 4381:5841, ack 0, win 6432, length 1460
> > 13:12:05.355328 IP (tos 0x0, ttl 127, id 8325, offset 0, flags [DF], proto
> > TCP (6), length 40)
> > 172.16.112.124.cvspserver> 8.27.4.126.www: Flags [.], cksum 0xfe6f
> > (correct), seq 0, ack 662461, win 65535, length 0
> > 13:12:05.655764 IP (tos 0x0, ttl 127, id 5393, offset 0, flags [DF], proto
> > TCP (6), length 40)
> > 172.16.64.10.2308> co63.reliablehosting.com.www: Flags [.], cksum 0x1b43
> > (correct), seq 63164814, ack 3610294926, win 65535, length 0
> > 13:12:05.655789 IP (tos 0x0, ttl 64, id 65532, offset 0, flags [DF], proto
> > TCP (6), length 1500)
> > co63.reliablehosting.com.www> 172.16.64.10.2308: Flags [P.], cksum 0x4e5f
> > (correct), seq 1461:2921, ack 0, win 17184, length 1460
> > 13:12:05.655798 IP (tos 0x0, ttl 64, id 65533, offset 0, flags [DF], proto
> > TCP (6), length 1215)
RE: [squid-users] Some pages loading very slow in 3.1.10 Stable
Jan 2011 18:06:02 +1100 > Subject: RE: [squid-users] Some pages loading very slow in 3.1.10 Stable > > > Thanks for your prompt reply. Well I am not much experienced admin in linux > or in terms of squid, and therefore I havent installed > wireshark/tshark/tcpdump in squid yet, but I will install it now to go in > deep. > > My previous version of squid was 2.7 Stable downloaded from aptitude which > was installed in a different machine, and not in this new machine. My current > configuration in the current PC (Core i7) is a squid 3.1.10 custom compiled > by me with only the ZPH and the Linux Net Filter enabled. I haven't used any > other parameters while configuring it. > > Default Cache_Dir is splitted into three directories: > UFS /media/cache/storage501 51400 16 256 > UFS /media/cache/storage502 51400 16 256 > UFS /media/cache/storage503 51400 16 256 > > and cache_mem is 150MB > > Sometimes during peak hours while the network load is high, we do experience > very slow browsing (in all websites) which is usually resolved after a quick > restart of the squid process/service. > > Thanks & Best Regards, > Shishir > > >> Date: Mon, 24 Jan 2011 07:48:12 +0100 >> From: rm...@free.fr >> To: saifu...@hotmail.com >> CC: rm...@free.fr >> Subject: Re: [squid-users] Some pages loading very slow in 3.1.10 Stable >> >> Hi Saiful, >> On same plateform, do you have a old squid version working better? what is >> your previous squid version please? >> can you start network capture like wireshark/tshark/tcpdump on your squid >> please? and run firebug for example and compare last and your previous squid >> version please? >> can you compare squid cpu usage last and previous version please? >> Regards >> Rmkml >> >> >> On Mon, 24 Jan 2011, Saiful Alam wrote: >> >>> >>> Hello Everyone, >>> My Squid Configuration is pretty much default, except >>> the fact, that I have added some refresh_patterns myself and collected >>> from internet in order to get more hits. The server is a Squid3.1.10 (in >>> 3128 intercept transparent) running on Ubuntu 10.10 and comprises of >>> Intel(R) Core(TM) i7 CPU 870 @ 2.93GHz. I would say that >>> everything is fine, except some pages are loading very very slow. For >>> example, www.music.com.bd / www.djmaza.com ... I will try to give more >>> examples as I find more, but for the time being these two sites are very >>> important for my clients. Don't understand actually whats wrong. If >>> this has anything to do with the squid.conf file, I will attach it with >>> the next reply. For the time being here's my refresh_patterns. >>> >>> refresh_pattern >>> ^http://.*\.facebook.com.*\.(jpg|gif|png|swf|wav|3gp|flv|swf|wmv|zip|rar) >>> 10080 300% 40320 reload-into-ims >>> refresh_pattern >>> ^http://.*\.fbcdn.net.*\.(jpg|gif|png|swf|wav|mp3|3gp|flv|swf|wmv|zip|rar) >>> 10080 300% 40320 reload-into-ims >>> refresh_pattern >>> ^http://.*\.yimg.com.*\.(jpg|gif|png|swf|wav|3gp|flv|swf|wmv|zip|rar) 10080 >>> 300% 40320 reload-into-ims >>> refresh_pattern >>> ^http://.*\.ytimg.com.*\.(jpg|gif|png|swf|wav|mp3|3gp|flv|swf|wmv|zip|rar) >>> 10080 300% 40320 reload-into-ims >>> refresh_pattern ^http://.*\.gstatic.com.*\.(jpg|gif|png) 10080 300% 40320 >>> reload-into-ims >>> refresh_pattern ^http://.*\.google.com.*\.(jpg|gif|png) 10080 300% 40320 >>> reload-into-ims >>> refresh_pattern dsebd\.org.*\.(jpg|gif|png) 600 50% 3000 >>> refresh_pattern windowsupdate.com/.*\.(cab|exe) 4320 100% 43200 >>> reload-into-ims >>> refresh_pattern download.microsoft.com/.*\.(cab|exe) 4320 100% 43200 >>> reload-into-ims >>> refresh_pattern ^http://.*\.cnn\.com 1000 50% 4320 override-lastmod >>> refresh_pattern ^http://news\.bbc\.co\.uk 1000 50% 4320 override-lastmod >>> refresh_pattern microsoft 1080 150% 10080 override-lastmod >>> refresh_pattern msn\.com 4320 150% 10080 override-lastmod >>> refresh_pattern ^http://.*FIDO 360 1000% 480 >>> refresh_pattern \.r[0-9][0-0]$ 10080 150% 40320 >>> refresh_pattern ^http://.*\.asis$ 1440 50% 20160 >>> refresh_pattern -i \.art$ 10080 150% 43200 >>> refresh_pattern -i \.qtm$ 10080 150% 40320 reload-into-ims >>> refresh_pattern -i \.viv$ 10080 150% 40320 reload-into-ims >>> refresh_pattern -i \.pdf$ 10080 300% 40320 reload-into-ims >>> refresh_pattern -i \.jpg$ 10080 300% 40320 reload-into-ims >
RE: [squid-users] Some pages loading very slow in 3.1.10 Stable
Thanks for your prompt reply. Well I am not much experienced admin in linux or in terms of squid, and therefore I havent installed wireshark/tshark/tcpdump in squid yet, but I will install it now to go in deep. My previous version of squid was 2.7 Stable downloaded from aptitude which was installed in a different machine, and not in this new machine. My current configuration in the current PC (Core i7) is a squid 3.1.10 custom compiled by me with only the ZPH and the Linux Net Filter enabled. I haven't used any other parameters while configuring it. Default Cache_Dir is splitted into three directories: UFS /media/cache/storage501 51400 16 256 UFS /media/cache/storage502 51400 16 256 UFS /media/cache/storage503 51400 16 256 and cache_mem is 150MB Sometimes during peak hours while the network load is high, we do experience very slow browsing (in all websites) which is usually resolved after a quick restart of the squid process/service. Thanks & Best Regards, Shishir > Date: Mon, 24 Jan 2011 07:48:12 +0100 > From: rm...@free.fr > To: saifu...@hotmail.com > CC: rm...@free.fr > Subject: Re: [squid-users] Some pages loading very slow in 3.1.10 Stable > > Hi Saiful, > On same plateform, do you have a old squid version working better? what is > your previous squid version please? > can you start network capture like wireshark/tshark/tcpdump on your squid > please? and run firebug for example and compare last and your previous squid > version please? > can you compare squid cpu usage last and previous version please? > Regards > Rmkml > > > On Mon, 24 Jan 2011, Saiful Alam wrote: > > > > > Hello Everyone, > > My Squid Configuration is pretty much default, except > > the fact, that I have added some refresh_patterns myself and collected > > from internet in order to get more hits. The server is a Squid3.1.10 (in > > 3128 intercept transparent) running on Ubuntu 10.10 and comprises of > > Intel(R) Core(TM) i7 CPU 870 @ 2.93GHz. I would say that > > everything is fine, except some pages are loading very very slow. For > > example, www.music.com.bd / www.djmaza.com ... I will try to give more > > examples as I find more, but for the time being these two sites are very > > important for my clients. Don't understand actually whats wrong. If > > this has anything to do with the squid.conf file, I will attach it with > > the next reply. For the time being here's my refresh_patterns. > > > > refresh_pattern > > ^http://.*\.facebook.com.*\.(jpg|gif|png|swf|wav|3gp|flv|swf|wmv|zip|rar) > > 10080 300% 40320 reload-into-ims > > refresh_pattern > > ^http://.*\.fbcdn.net.*\.(jpg|gif|png|swf|wav|mp3|3gp|flv|swf|wmv|zip|rar) > > 10080 300% 40320 reload-into-ims > > refresh_pattern > > ^http://.*\.yimg.com.*\.(jpg|gif|png|swf|wav|3gp|flv|swf|wmv|zip|rar) 10080 > > 300% 40320 reload-into-ims > > refresh_pattern > > ^http://.*\.ytimg.com.*\.(jpg|gif|png|swf|wav|mp3|3gp|flv|swf|wmv|zip|rar) > > 10080 300% 40320 reload-into-ims > > refresh_pattern ^http://.*\.gstatic.com.*\.(jpg|gif|png) 10080 300% 40320 > > reload-into-ims > > refresh_pattern ^http://.*\.google.com.*\.(jpg|gif|png) 10080 300% 40320 > > reload-into-ims > > refresh_pattern dsebd\.org.*\.(jpg|gif|png) 600 50% 3000 > > refresh_pattern windowsupdate.com/.*\.(cab|exe) 4320 100% 43200 > > reload-into-ims > > refresh_pattern download.microsoft.com/.*\.(cab|exe) 4320 100% 43200 > > reload-into-ims > > refresh_pattern ^http://.*\.cnn\.com 1000 50% 4320 override-lastmod > > refresh_pattern ^http://news\.bbc\.co\.uk 1000 50% 4320 override-lastmod > > refresh_pattern microsoft 1080 150% 10080 override-lastmod > > refresh_pattern msn\.com 4320 150% 10080 override-lastmod > > refresh_pattern ^http://.*FIDO 360 1000% 480 > > refresh_pattern \.r[0-9][0-0]$ 10080 150% 40320 > > refresh_pattern ^http://.*\.asis$ 1440 50% 20160 > > refresh_pattern -i \.art$ 10080 150% 43200 > > refresh_pattern -i \.qtm$ 10080 150% 40320 reload-into-ims > > refresh_pattern -i \.viv$ 10080 150% 40320 reload-into-ims > > refresh_pattern -i \.pdf$ 10080 300% 40320 reload-into-ims > > refresh_pattern -i \.jpg$ 10080 300% 40320 reload-into-ims > > refresh_pattern -i \.png$ 10080 300% 40320 reload-into-ims > > refresh_pattern -i \.gif$ 10080 300% 40320 reload-into-ims > > refresh_pattern -i \.flv$ 10080 300% 40320 reload-into-ims > > refresh_pattern -i \.ico$ 10080 300% 40320 reload-into-ims > > refresh_pattern -i \.jpeg$ 10080 300% 40320 reload-into-ims > > refresh_pattern -i \.x-flv$ 10080 300% 40320 reload-into-ims > > refresh_patter
[squid-users] Some pages loading very slow in 3.1.10 Stable
Hello Everyone, My Squid Configuration is pretty much default, except the fact, that I have added some refresh_patterns myself and collected from internet in order to get more hits. The server is a Squid3.1.10 (in 3128 intercept transparent) running on Ubuntu 10.10 and comprises of Intel(R) Core(TM) i7 CPU 870 @ 2.93GHz. I would say that everything is fine, except some pages are loading very very slow. For example, www.music.com.bd / www.djmaza.com ... I will try to give more examples as I find more, but for the time being these two sites are very important for my clients. Don't understand actually whats wrong. If this has anything to do with the squid.conf file, I will attach it with the next reply. For the time being here's my refresh_patterns. refresh_pattern ^http://.*\.facebook.com.*\.(jpg|gif|png|swf|wav|3gp|flv|swf|wmv|zip|rar) 10080 300% 40320 reload-into-ims refresh_pattern ^http://.*\.fbcdn.net.*\.(jpg|gif|png|swf|wav|mp3|3gp|flv|swf|wmv|zip|rar) 10080 300% 40320 reload-into-ims refresh_pattern ^http://.*\.yimg.com.*\.(jpg|gif|png|swf|wav|3gp|flv|swf|wmv|zip|rar) 10080 300% 40320 reload-into-ims refresh_pattern ^http://.*\.ytimg.com.*\.(jpg|gif|png|swf|wav|mp3|3gp|flv|swf|wmv|zip|rar) 10080 300% 40320 reload-into-ims refresh_pattern ^http://.*\.gstatic.com.*\.(jpg|gif|png) 10080 300% 40320 reload-into-ims refresh_pattern ^http://.*\.google.com.*\.(jpg|gif|png) 10080 300% 40320 reload-into-ims refresh_pattern dsebd\.org.*\.(jpg|gif|png) 600 50% 3000 refresh_pattern windowsupdate.com/.*\.(cab|exe) 4320 100% 43200 reload-into-ims refresh_pattern download.microsoft.com/.*\.(cab|exe) 4320 100% 43200 reload-into-ims refresh_pattern ^http://.*\.cnn\.com 1000 50% 4320 override-lastmod refresh_pattern ^http://news\.bbc\.co\.uk 1000 50% 4320 override-lastmod refresh_pattern microsoft 1080 150% 10080 override-lastmod refresh_pattern msn\.com 4320 150% 10080 override-lastmod refresh_pattern ^http://.*FIDO 360 1000% 480 refresh_pattern \.r[0-9][0-0]$ 10080 150% 40320 refresh_pattern ^http://.*\.asis$ 1440 50% 20160 refresh_pattern -i \.art$ 10080 150% 43200 refresh_pattern -i \.qtm$ 10080 150% 40320 reload-into-ims refresh_pattern -i \.viv$ 10080 150% 40320 reload-into-ims refresh_pattern -i \.pdf$ 10080 300% 40320 reload-into-ims refresh_pattern -i \.jpg$ 10080 300% 40320 reload-into-ims refresh_pattern -i \.png$ 10080 300% 40320 reload-into-ims refresh_pattern -i \.gif$ 10080 300% 40320 reload-into-ims refresh_pattern -i \.flv$ 10080 300% 40320 reload-into-ims refresh_pattern -i \.ico$ 10080 300% 40320 reload-into-ims refresh_pattern -i \.jpeg$ 10080 300% 40320 reload-into-ims refresh_pattern -i \.x-flv$ 10080 300% 40320 reload-into-ims refresh_pattern -i \.swf$ 10080 300% 40320 reload-into-ims refresh_pattern -i \.wav$ 10080 300% 40320 reload-into-ims refresh_pattern -i \.mp3$ 10080 300% 40320 override-expire override-lastmod reload-into-ims ignore-reload refresh_pattern -i \.mp4$ 10080 300% 40320 override-expire override-lastmod reload-into-ims ignore-reload refresh_pattern -i \.zip$ 10080 300% 40320 override-expire override-lastmod reload-into-ims ignore-reload refresh_pattern -i \.rar$ 10080 300% 40320 override-expire override-lastmod reload-into-ims ignore-reload refresh_pattern -i \.exe$ 10080 300% 40320 override-expire override-lastmod reload-into-ims ignore-reload refresh_pattern -i \.zip$ 10080 300% 40320 override-expire override-lastmod reload-into-ims ignore-reload refresh_pattern -i \.doc$ 10080 300% 40320 override-expire override-lastmod reload-into-ims ignore-reload refresh_pattern -i \.ppt$ 10080 300% 40320 override-expire override-lastmod reload-into-ims ignore-reload refresh_pattern -i \.tiff$ 10080 300% 40320 override-expire override-lastmod reload-into-ims ignore-reload refresh_pattern -i \.ram$ 10080 300% 40320 override-expire override-lastmod reload-into-ims ignore-reload refresh_pattern -i \.rm$ 10080 300% 40320 override-expire override-lastmod reload-into-ims ignore-reload refresh_pattern -i \.rpm$ 10080 300% 40320 override-expire override-lastmod reload-into-ims ignore-reload refresh_pattern -i \.htm$ 1440 50% 40320 override-expire override-lastmod reload-into-ims ignore-reload refresh_pattern -i \.html$ 1440 50% 40320 override-expire override-lastmod reload-into-ims ignore-reload refresh_pattern -i \.txt$ 1440 100% 20160 reload-into-ims override-lastmod refresh_pattern -i \.arj$ 2880 200% 40320 refresh_pattern -i \.Z$ 10080 200% 40320 refresh_pattern ^ftp: 1440 50% 10080 refresh_pattern ^gopher: 1440 10% 1440 refresh_pattern . 0 40% 4320 negative_ttl 1 minute positive_dns_ttl 15 hour negative_dns_ttl 20 second Thanks & Best Regards, Saiful
[squid-users] Always getting negative hit
Hi, I have this strange problem started from tonight in my squid box. Actually everything was working ok, but I was trying to do some port forwarding things with iptables. But suddendly i realized that my cache hits has drop after i restarted the networking service quite a few times. At first i wasn't worried, but after an hour I got worried. Although calamaris is showing more byte hit % , unfortunately cachemgr.cgi is showing wrong value i guess. My operating system is a Ubuntu 10.10 server. Squid Object Cache: Version 3.1.10 Start Time:Sat, 15 Jan 2011 22:17:57 GMT Current Time:Sat, 15 Jan 2011 23:09:17 GMT Connection information for squid: Number of clients accessing cache: 100 Number of HTTP requests received: 60132 Number of ICP messages received:0 Number of ICP messages sent:0 Number of queued ICP replies: 0 Number of HTCP messages received: 0 Number of HTCP messages sent: 0 Request failure ratio: 0.00 Average HTTP requests per minute since start: 1171.5 Average ICP messages per minute since start:0.0 Select loop called: 5143660 times, 0.599 ms avg Cache information for squid: Hits as % of all requests: 5min: 12.3%, 60min: 18.8% Hits as % of bytes sent:5min: -12.5%, 60min: -1.9% Memory hits as % of hit requests: 5min: 12.4%, 60min: 4.3% Disk hits as % of hit requests: 5min: 50.6%, 60min: 64.5% Storage Swap size: 33978424 KB Storage Swap capacity: 64.6% used, 35.4% free Storage Mem size: 101084 KB Storage Mem capacity: 99.7% used, 0.3% free Mean Object Size: 16.65 KB Requests given to unlinkd: 2605 Median Service Times (seconds) 5 min60 min: HTTP Requests (All): 7.79421 0.61549 Cache Misses: 26.97993 0.80651 Cache Hits:0.01387 0.00919 Near Hits: 0.58309 0.46965 Not-Modified Replies: 0.0 0.0 DNS Lookups: 0.51867 0.16304 ICP Queries: 0.0 0.0 Resource usage for squid: UP Time:3079.731 seconds CPU Time: 115.800 seconds CPU Usage: 3.76% CPU Usage, 5 minute avg:3.81% CPU Usage, 60 minute avg: 3.76% Process Data Segment Size via sbrk(): 497056 KB Maximum Resident Size: 2051920 KB Page faults with physical i/o: 0 Memory usage for squid via mallinfo(): Total space in arena: 497188 KB Ordinary blocks: 495769 KB 9420 blks Small blocks: 0 KB 0 blks Holding blocks: 36380 KB 9 blks Free Small blocks: 0 KB Free Ordinary blocks:1418 KB Total in use: 532149 KB 100% Total free: 1418 KB 0% Total size:533568 KB Memory accounted for: Total accounted: 404284 KB 76% memPool accounted: 404284 KB 76% memPool unaccounted: 129283 KB 24% memPoolAlloc calls: 17894845 memPoolFree calls: 18064591 File descriptor usage for squid: Maximum number of file descriptors: 65535 Largest file desc currently in use: 1409 Number of file desc currently in use: 1255 Files queued for open: 0 Available number of file descriptors: 64280 Reserved number of file descriptors: 100 Store Disk files open: 16 Internal Data Structures: 2041007 StoreEntries 8633 StoreEntries with MemObjects 8141 Hot Object Cache Items 2040138 on-disk objects Please help to understand why I am getting negative hits all of a sudden? And even if it caused by something else, why isn't it be solved after an hour when so many clients are online. Regards, Shishir
