Re: [squid-users] Some pages loading very slow in 3.1.10 Stable

Eliezer Mon, 24 Jan 2011 01:41:24 -0800

are you kdding me?


500 clients?

if most of the clients are just doing almost nothing just downloadingone page of 2 MB..


how much is your HD I\O ?? in this case?(in speed MBps) ?

so first.. change the UFS to AUFS you dont need to do anything to thecache it self cause it the same system just with Async options on.and just buy two more of these 500GB drives and put all three of them inraid 0 or 5.

or first try 0 on two and then add another one to then if it goes smoothly.
this will give you a lot more speed.
i dont now the cause but it looks like or connectivity/dns or I\O problem.

i'v tried to use iptraf but ifstat just gives you the numbers of everyInterface you have in aginst out traffic meter in a simple way.



On 24/01/2011 11:02, Saiful Alam wrote:

TRIED AUFS, but didn't get better performance, while

researching in the web, I read everywhere that AUFS is better than UFS
in terms of performance,but I don't know why I get bad performance with
this.
Processor is Intel(R) Core(TM) i7 CPU         870  @ 2.93GHz
Motherboard is Intel® Desktop Board DH55TC



                 RAM is 2x4GB=8GB DDR3
DISK 1 = 250GB      Hitachi SATA HDP72502 {[USED ONLY FOR UBUNTU SYSTEM&  BOOT]}
DISK 2 = 500GB Hitachi SATA HDS72105 {[USED FOR CACHE DIRECTORIES ONLY, MOUNTED 
ON /MEDIA/CACHE FILESYSTEM EXT4]}

Normally my Disk I/O never goes more than 15% and I would say the average is 
about 3-4%.
For bandwidth monitoring I usually see iptraf which is also good, but surely 
I'll try ifstat next time.
At peak hours (which is between 10pm - 2am GMT +0600), we have around 500 
clients connected (approx)

I have tried apt-get install squid3 (which is the default 3.1.6 in apt 
repository) and found the performance of 3.1.10 (my custom configuration) is 
better than the 3.1.6.

Regards,
Saiful


----------------------------------------

Date: Mon, 24 Jan 2011 10:36:14 +0200
From: elie...@ec.hadorhabaac.com
To: squid-users@squid-cache.org
Subject: Re: [squid-users] Some pages loading very slow in 3.1.10 Stable

It's a small peace and most of the answers are not really suppose to there.

the first thing is that your cache is not just cache it's a store house..

it;'s not bad but you can try to change the ufs to aufs..

can get better performance.

what are the specification of the machine?

core i7 ?.... ram? what disk? array of disks?

did you tried to ping from the machine or WGET?

if it's debian you can install ifstat that can give you real-rime
bandwidth usage and it might be cause of something else that is not
related to squid..

if you have even like 25 clients downloading obsessively mp3 files for
like 10 or more minutes in this time your I\O of your hard drive will rise..

also..

you can monitor the access to the squid folders onfly while you have the
problem and to understand what is causing it..

if it's CPU load or DISK I\O load .. or other stuff.

by the way.. can you try the ubuntu squid3 stable?

im using squid3 stable on ubuntu 10.04 on an Intel Atom D450 machine
with cache of 40GB and it's taking the load very nicely.

On 24/01/2011 09:12, Saiful Alam wrote:

Some results of TCPDUMP in -vv mode.

13:12:04.191180 IP (tos 0x0, ttl 127, id 2750, offset 0, flags [DF], proto TCP 
(6), length 40)
172.16.80.2.1155>  77.67.29.42.www: Flags [.], cksum 0x6de4 (correct), seq 
1127903567, ack 4192021369, win 64700, length 0
13:12:04.192822 IP (tos 0x0, ttl 64, id 4692, offset 0, flags [DF], proto TCP 
(6), length 823)
www-12-02-snc5.facebook.com.www>  10.16.63.123.3714: Flags [P.], cksum 0xc789 
(correct), seq 4235393899:4235394682, ack 3661185770, win 14600, length 783
13:12:04.831628 IP (tos 0x0, ttl 127, id 29487, offset 0, flags [DF], proto TCP 
(6), length 40)
172.16.80.6.rmiregistry>  cds174.lax.llnw.net.www: Flags [.], cksum 0xfa45 
(correct), seq 0, ack 460801, win 65535, length 0
13:12:05.355300 IP (tos 0x0, ttl 127, id 2265, offset 0, flags [DF], proto TCP 
(6), length 40)
192.168.96.165.57362>  195.122.152.64.www: Flags [.], cksum 0xfc6c (correct), 
seq 1875916651, ack 4162436021, win 64240, length 0
13:12:05.355322 IP (tos 0x0, ttl 64, id 18785, offset 0, flags [DF], proto TCP 
(6), length 1500)
195.122.152.64.www>  192.168.96.165.57362: Flags [.], cksum 0x0f8f (correct), 
seq 4381:5841, ack 0, win 6432, length 1460
13:12:05.355328 IP (tos 0x0, ttl 127, id 8325, offset 0, flags [DF], proto TCP 
(6), length 40)
172.16.112.124.cvspserver>  8.27.4.126.www: Flags [.], cksum 0xfe6f (correct), 
seq 0, ack 662461, win 65535, length 0
13:12:05.655764 IP (tos 0x0, ttl 127, id 5393, offset 0, flags [DF], proto TCP 
(6), length 40)
172.16.64.10.2308>  co63.reliablehosting.com.www: Flags [.], cksum 0x1b43 
(correct), seq 63164814, ack 3610294926, win 65535, length 0
13:12:05.655789 IP (tos 0x0, ttl 64, id 65532, offset 0, flags [DF], proto TCP 
(6), length 1500)
co63.reliablehosting.com.www>  172.16.64.10.2308: Flags [P.], cksum 0x4e5f 
(correct), seq 1461:2921, ack 0, win 17184, length 1460
13:12:05.655798 IP (tos 0x0, ttl 64, id 65533, offset 0, flags [DF], proto TCP 
(6), length 1215)
co63.reliablehosting.com.www>  172.16.64.10.2308: Flags [P.], cksum 0xa23e 
(correct), seq 2921:4096, ack 0, win 17184, length 1175
13:12:05.656066 IP (tos 0x0, ttl 126, id 42247, offset 0, flags [DF], proto TCP 
(6), length 73)
172.16.30.6.1575>  79-110-88-28.gfsrv.net.15103: Flags [P.], cksum 0x207b 
(correct), seq 22088:22121, ack 55870, win 64475, length 33
13:12:05.656081 IP (tos 0x0, ttl 126, id 42248, offset 0, flags [DF], proto TCP 
(6), length 51)
172.16.30.6.1575>  79-110-88-28.gfsrv.net.15103: Flags [P.], cksum 0xd9a7 
(correct), seq 22121:22132, ack 55870, win 64475, length 11
13:12:05.949960 IP (tos 0x0, ttl 126, id 5909, offset 0, flags [DF], proto TCP 
(6), length 40)
172.16.10.9.1408>  GGCpmo1.pal.seabone.net.www: Flags [.], cksum 0xfe1a 
(correct), seq 3933402590, ack 133923280, win 65535, length 0
13:12:05.949983 IP (tos 0x0, ttl 126, id 31989, offset 0, flags [DF], proto TCP 
(6), length 52)
172.16.52.6.62360>  c-n180-u0282-208.webazilla.com.www: Flags [.], cksum 0x0bbb 
(correct), seq 0, ack 1119485, win 64240, options [nop,nop,sack 1 
{1120945:1129705}], length 0
13:12:05.949997 IP (tos 0x0, ttl 127, id 1202, offset 0, flags [DF], proto TCP 
(6), length 40)
172.16.200.42.1454>  146.82.204.207.www: Flags [.], cksum 0x5a1a (correct), seq 
0, ack 1965427, win 65375, length 0
13:12:05.950985 IP (tos 0x0, ttl 126, id 36156, offset 0, flags [DF], proto TCP 
(6), length 40)
172.16.30.7.1795>  cds17.sin.llnw.net.www: Flags [.], cksum 0x779d (correct), 
seq 4079975438, ack 3825116412, win 65535, length 0
13:12:05.951132 IP (tos 0x0, ttl 126, id 5910, offset 0, flags [DF], proto TCP 
(6), length 996)
172.16.10.9.1408>  GGCpmo1.pal.seabone.net.www: Flags [P.], cksum 0xf7ba 
(correct), seq 0:956, ack 1, win 65535, length 956
13:12:05.951152 IP (tos 0x0, ttl 64, id 20092, offset 0, flags [DF], proto TCP 
(6), length 40)
GGCpmo1.pal.seabone.net.www>  172.16.10.9.1408: Flags [.], cksum 0xdc7e 
(correct), seq 1, ack 956, win 7648, length 0
13:12:05.955120 IP (tos 0x0, ttl 127, id 2284, offset 0, flags [DF], proto TCP 
(6), length 40)
192.168.96.165.57362>  195.122.152.64.www: Flags [.], cksum 0xda34 (correct), 
seq 0, ack 8761, win 64240, length 0
13:12:05.955148 IP (tos 0x0, ttl 64, id 18790, offset 0, flags [DF], proto TCP 
(6), length 972)
195.122.152.64.www>  192.168.96.165.57362: Flags [P.], cksum 0x02b4 (correct), 
seq 8761:9693, ack 0, win 6432, length 932
13:12:05.955605 IP (tos 0x0, ttl 126, id 20667, offset 0, flags [DF], proto TCP 
(6), length 40)
^C 172.16.106.3.49956>  213.254.248.146.www: Flags [.], cksum 0x696e (correct), 
seq 348423304, ack 3320774375, win 16425, length 0

226 packets captured
107740 packets received by filter
107433 packets dropped by kernel


Regards,
Saiful

----------------------------------------

From: saifu...@hotmail.com
To: rm...@free.fr
CC: squid-users@squid-cache.org
Date: Mon, 24 Jan 2011 18:06:02 +1100
Subject: RE: [squid-users] Some pages loading very slow in 3.1.10 Stable

Thanks for your prompt reply. Well I am not much experienced admin in linux or 
in terms of squid, and therefore I havent installed wireshark/tshark/tcpdump in 
squid yet, but I will install it now to go in deep.

My previous version of squid was 2.7 Stable downloaded from aptitude which was 
installed in a different machine, and not in this new machine. My current 
configuration in the current PC (Core i7) is a squid 3.1.10 custom compiled by 
me with only the ZPH and the Linux Net Filter enabled. I haven't used any other 
parameters while configuring it.

Default Cache_Dir is splitted into three directories:
UFS /media/cache/storage501 51400 16 256
UFS /media/cache/storage502 51400 16 256
UFS /media/cache/storage503 51400 16 256

and cache_mem is 150MB

Sometimes during peak hours while the network load is high, we do experience 
very slow browsing (in all websites) which is usually resolved after a quick 
restart of the squid process/service.

Thanks&  Best Regards,
Shishir

----------------------------------------

Date: Mon, 24 Jan 2011 07:48:12 +0100
From: rm...@free.fr
To: saifu...@hotmail.com
CC: rm...@free.fr
Subject: Re: [squid-users] Some pages loading very slow in 3.1.10 Stable

Hi Saiful,
On same plateform, do you have a old squid version working better? what is your 
previous squid version please?
can you start network capture like wireshark/tshark/tcpdump on your squid 
please? and run firebug for example and compare last and your previous squid 
version please?
can you compare squid cpu usage last and previous version please?
Regards
Rmkml

On Mon, 24 Jan 2011, Saiful Alam wrote:

Hello Everyone,
My Squid Configuration is pretty much default, except
the fact, that I have added some refresh_patterns myself and collected
from internet in order to get more hits. The server is a Squid3.1.10 (in
3128 intercept transparent) running on Ubuntu 10.10 and comprises of
Intel(R) Core(TM) i7 CPU 870 @ 2.93GHz. I would say that
everything is fine, except some pages are loading very very slow. For
example, www.music.com.bd / www.djmaza.com ... I will try to give more
examples as I find more, but for the time being these two sites are very
important for my clients. Don't understand actually whats wrong. If
this has anything to do with the squid.conf file, I will attach it with
the next reply. For the time being here's my refresh_patterns.

refresh_pattern 
^http://.*\.facebook.com.*\.(jpg|gif|png|swf|wav|3gp|flv|swf|wmv|zip|rar) 10080 
300% 40320 reload-into-ims
refresh_pattern 
^http://.*\.fbcdn.net.*\.(jpg|gif|png|swf|wav|mp3|3gp|flv|swf|wmv|zip|rar) 
10080 300% 40320 reload-into-ims
refresh_pattern 
^http://.*\.yimg.com.*\.(jpg|gif|png|swf|wav|3gp|flv|swf|wmv|zip|rar) 10080 
300% 40320 reload-into-ims
refresh_pattern 
^http://.*\.ytimg.com.*\.(jpg|gif|png|swf|wav|mp3|3gp|flv|swf|wmv|zip|rar) 
10080 300% 40320 reload-into-ims
refresh_pattern ^http://.*\.gstatic.com.*\.(jpg|gif|png) 10080 300% 40320 
reload-into-ims
refresh_pattern ^http://.*\.google.com.*\.(jpg|gif|png) 10080 300% 40320 
reload-into-ims
refresh_pattern dsebd\.org.*\.(jpg|gif|png) 600 50% 3000
refresh_pattern windowsupdate.com/.*\.(cab|exe) 4320 100% 43200 reload-into-ims
refresh_pattern download.microsoft.com/.*\.(cab|exe) 4320 100% 43200 
reload-into-ims
refresh_pattern ^http://.*\.cnn\.com 1000 50% 4320 override-lastmod
refresh_pattern ^http://news\.bbc\.co\.uk 1000 50% 4320 override-lastmod
refresh_pattern microsoft 1080 150% 10080 override-lastmod
refresh_pattern msn\.com 4320 150% 10080 override-lastmod
refresh_pattern ^http://.*FIDO 360 1000% 480
refresh_pattern \.r[0-9][0-0]$ 10080 150% 40320
refresh_pattern ^http://.*\.asis$ 1440 50% 20160
refresh_pattern -i \.art$ 10080 150% 43200
refresh_pattern -i \.qtm$ 10080 150% 40320 reload-into-ims
refresh_pattern -i \.viv$ 10080 150% 40320 reload-into-ims
refresh_pattern -i \.pdf$ 10080 300% 40320 reload-into-ims
refresh_pattern -i \.jpg$ 10080 300% 40320 reload-into-ims
refresh_pattern -i \.png$ 10080 300% 40320 reload-into-ims
refresh_pattern -i \.gif$ 10080 300% 40320 reload-into-ims
refresh_pattern -i \.flv$ 10080 300% 40320 reload-into-ims
refresh_pattern -i \.ico$ 10080 300% 40320 reload-into-ims
refresh_pattern -i \.jpeg$ 10080 300% 40320 reload-into-ims
refresh_pattern -i \.x-flv$ 10080 300% 40320 reload-into-ims
refresh_pattern -i \.swf$ 10080 300% 40320 reload-into-ims
refresh_pattern -i \.wav$ 10080 300% 40320 reload-into-ims
refresh_pattern -i \.mp3$ 10080 300% 40320 override-expire override-lastmod 
reload-into-ims ignore-reload
refresh_pattern -i \.mp4$ 10080 300% 40320 override-expire override-lastmod 
reload-into-ims ignore-reload
refresh_pattern -i \.zip$ 10080 300% 40320 override-expire override-lastmod 
reload-into-ims ignore-reload
refresh_pattern -i \.rar$ 10080 300% 40320 override-expire override-lastmod 
reload-into-ims ignore-reload
refresh_pattern -i \.exe$ 10080 300% 40320 override-expire override-lastmod 
reload-into-ims ignore-reload
refresh_pattern -i \.zip$ 10080 300% 40320 override-expire override-lastmod 
reload-into-ims ignore-reload
refresh_pattern -i \.doc$ 10080 300% 40320 override-expire override-lastmod 
reload-into-ims ignore-reload
refresh_pattern -i \.ppt$ 10080 300% 40320 override-expire override-lastmod 
reload-into-ims ignore-reload
refresh_pattern -i \.tiff$ 10080 300% 40320 override-expire override-lastmod 
reload-into-ims ignore-reload
refresh_pattern -i \.ram$ 10080 300% 40320 override-expire override-lastmod 
reload-into-ims ignore-reload
refresh_pattern -i \.rm$ 10080 300% 40320 override-expire override-lastmod 
reload-into-ims ignore-reload
refresh_pattern -i \.rpm$ 10080 300% 40320 override-expire override-lastmod 
reload-into-ims ignore-reload
refresh_pattern -i \.htm$ 1440 50% 40320 override-expire override-lastmod 
reload-into-ims ignore-reload
refresh_pattern -i \.html$ 1440 50% 40320 override-expire override-lastmod 
reload-into-ims ignore-reload
refresh_pattern -i \.txt$ 1440 100% 20160 reload-into-ims override-lastmod
refresh_pattern -i \.arj$ 2880 200% 40320
refresh_pattern -i \.Z$ 10080 200% 40320
refresh_pattern ^ftp: 1440 50% 10080
refresh_pattern ^gopher: 1440 10% 1440
refresh_pattern . 0 40% 4320

negative_ttl 1 minute
positive_dns_ttl 15 hour
negative_dns_ttl 20 second

Thanks&  Best Regards,
Saiful

Re: [squid-users] Some pages loading very slow in 3.1.10 Stable

Reply via email to