Re: [squid-users] Squid 3.x very slow loading on ireport.cnn.com
* Max Feil mf...@qnx.com: Already did use Wireshark. Here is some more info: If you look through the traces you'll notice that at some point Squid sends a TCP [FIN, ACK] right in the middle of a connection for seemingly no reason. (Attempting to close the connection) The server ignores this and sends the rest of the data, which Squid responds to with TCP RST (request to reset) since it now believes the connection to be closed. That sounds like a Checkpoint FW-1 with smart defense (aka bloody stupid crap) somewhere in the path -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | http://www.charite.de
RE: [squid-users] Squid 3.x very slow loading on ireport.cnn.com
sön 2011-01-23 klockan 23:35 -0500 skrev Max Feil: If you look through the traces you'll notice that at some point Squid sends a TCP [FIN, ACK] right in the middle of a connection for seemingly no reason. From the browser side it seems to be given no notification that the connection was closed (and indeed I can see no reason why it should be closed) so it seems to sit around doing nothing as it may have reached the max connections limit. Odd. Can you reproduce the problem? If so then it would be very helpful if you could run Squid with full debug output enabled (squid -k debug) and also capture the data with wireshark. Then send the collected data to ftp://ftp.squid-cache.se/incoming/ and notify me. Regards Henrik
Re: [squid-users] Squid 3.x very slow loading on ireport.cnn.com
i will try now On 24/01/2011 22:01, Henrik Nordström wrote: squid -k debug
Re: [squid-users] Squid 3.x very slow loading on ireport.cnn.com
On Mon, 24 Jan 2011 18:56:48 +0100, Ralf Hildebrandt wrote: * Max Feil: Already did use Wireshark. Here is some more info: If you look through the traces you'll notice that at some point Squid sends a TCP [FIN, ACK] right in the middle of a connection for seemingly no reason. (Attempting to close the connection) The server ignores this and sends the rest of the data, which Squid responds to with TCP RST (request to reset) since it now believes the connection to be closed. That sounds like a Checkpoint FW-1 with smart defense (aka bloody stupid crap) somewhere in the path Ooh, thanks. So that was the Checkpoint problem. Yes Squid will not send FIN or RST to just one end of the connection mid way. Either both will get the FIN/RST or the server will be re-tried and the client connection will get the latter response. FWIW; the Linux guys have added demo config for this type of TCP link aborting to their public recommendations. Note that it is really only useful for *DDoS* situations. Not for normal traffic. Amos
RE: [squid-users] Squid 3.x very slow loading on ireport.cnn.com
tor 2011-01-20 klockan 02:50 -0500 skrev Max Feil: Thanks. I am looking at the squid access.log and the delay is caused by a GET which for some reason does not result in a response from the server. Either there is no response or Squid is missing the response. After a 120 second time-out the page continues loading, but the end result may be malformed due to the object which did not load. I would take a peek at the traffic using wireshark to get some insight in what is going on there. REgards Henrik
RE: [squid-users] Squid 3.x very slow loading on ireport.cnn.com
Already did use Wireshark. Here is some more info: If you look through the traces you'll notice that at some point Squid sends a TCP [FIN, ACK] right in the middle of a connection for seemingly no reason. (Attempting to close the connection) The server ignores this and sends the rest of the data, which Squid responds to with TCP RST (request to reset) since it now believes the connection to be closed. From the browser side it seems to be given no notification that the connection was closed (and indeed I can see no reason why it should be closed) so it seems to sit around doing nothing as it may have reached the max connections limit. After about 2 minutes (possibly related to a persistent connection timeout limit in squid) Squid seems to terminate all the connections with FIN,ACKs. The browser then seems to realize its connections are gone and it requests the remaining resources resulting in a bunch of TCP SYNs followed by the rest of the resources. Why it does this in the middle of connections we still have no clue, however turning off server_persistent_connections seems to make it load fast. However this is probably a bad idea in general... Max -Original Message- From: Henrik Nordström [mailto:hen...@henriknordstrom.net] Sent: Sunday, January 23, 2011 7:16 PM To: Max Feil Cc: squid-users@squid-cache.org Subject: RE: [squid-users] Squid 3.x very slow loading on ireport.cnn.com tor 2011-01-20 klockan 02:50 -0500 skrev Max Feil: Thanks. I am looking at the squid access.log and the delay is caused by a GET which for some reason does not result in a response from the server. Either there is no response or Squid is missing the response. After a 120 second time-out the page continues loading, but the end result may be malformed due to the object which did not load. I would take a peek at the traffic using wireshark to get some insight in what is going on there. REgards Henrik
Re: [squid-users] Squid 3.x very slow loading on ireport.cnn.com
On 24/01/2011 06:35, Max Feil wrote: Already did use Wireshark. Here is some more info: If you look through the traces you'll notice that at some point Squid sends a TCP [FIN, ACK] right in the middle of a connection for seemingly no reason. (Attempting to close the connection) The server ignores this and sends the rest of the data, which Squid responds to with TCP RST (request to reset) since it now believes the connection to be closed. From the browser side it seems to be given no notification that the connection was closed (and indeed I can see no reason why it should be closed) so it seems to sit around doing nothing as it may have reached the max connections limit. After about 2 minutes (possibly related to a persistent connection timeout limit in squid) Squid seems to terminate all the connections with FIN,ACKs. The browser then seems to realize its connections are gone and it requests the remaining resources resulting in a bunch of TCP SYNs followed by the rest of the resources. Why it does this in the middle of connections we still have no clue, however turning off server_persistent_connections seems to make it load fast. However this is probably a bad idea in general... Max -Original Message- From: Henrik Nordström [mailto:hen...@henriknordstrom.net] Sent: Sunday, January 23, 2011 7:16 PM To: Max Feil Cc: squid-users@squid-cache.org Subject: RE: [squid-users] Squid 3.x very slow loading on ireport.cnn.com tor 2011-01-20 klockan 02:50 -0500 skrev Max Feil: Thanks. I am looking at the squid access.log and the delay is caused by a GET which for some reason does not result in a response from the server. Either there is no response or Squid is missing the response. After a 120 second time-out the page continues loading, but the end result may be malformed due to the object which did not load. I would take a peek at the traffic using wireshark to get some insight in what is going on there. REgards Henrik just noticed your relply. and also the mail daemon didnt like my log from a reason so i will send it to you seperetly: try to make an icl for this sites\domains in the list below to not use cache at all. send a log with much detail on the requests (headers\debug mode) the last message below: there was another guy with cnn problem no? (named max) did you made basic test like not with ping and dns stuff? cnn like many others are using CDN.. what makes it a little problem sometimes. did you compiled it yourself? this is the second time so try these: i will give you domain names and IP. and also do you use a local dns server? or ISP ? or eles? try to set the name server for the proxy as 8.8.8.8 (google dns) ping it first.. the page has like 8-10 domains\names it is trying to get ireport.cnn.com i.cdn.turner.com i2.cdn.turner.com audience.cnn.com b.scorecardresearch.com metrics.cnn.com metrics.ireport.com to to ping and dig... each one of them and send it in the email. then try to put in the hosts file of the squid OS these lines 157.166.255.213 ireport.cnn.com 207.123.56.126 i.cdn.turner.com 192.12.94.30 i2.cdn.turner.com 157.166.255.80audience.cnn.com 92.123.69.155b.scorecardresearch.com 66.235.143.121 metrics.cnn.com 192.33.14.30metrics.ireport.com also try to just get to the ip http://192.12.94.30/ send the results for these. another thing.. send us your settings file. if squid is running in transparent mode specify the ipv4 address . if it's not transparent even so set it to be able to... next thing is to make sure that Failed DNS cache time is set on 5 seconds dns_v4_fallback on and of cvourse a log will be nice. i will show you some of mine.
Re: [squid-users] Squid 3.x very slow loading on ireport.cnn.com
well i have found the problem.. it's not your proxy... your proxy is doing fine cause it's identifying files mimes and stuff=20 like that. have you ever heard of ZIP BOMB? well it's not it but it's something like it. the site itself working fine and the page is getting to your computer in=20 like the 5 first seconds... but... they are using such amount of Java script that i dont know how=20 even a PIII computer will handle it. well it's not the point. the point is that you wont see the page until liek the 50+ element in=20 the page.. so if one of the elements in the site is stuck cause of a bug=20 in the server or what so.. you wont get it. to make sure of it i uses paros to interogate it and i noticed this stuff= . so now the thing i do i allow only html mime... i want you to try suff the first page is 100KB wget will get you the page and you can try to look at the source and=20 stuff like that. the thing is the after the object 13 in the page... you will get the object: http://ireport.cnn.com/themes/custom/resources/username-overlay.js and then the action begins.. so after the 84 object it takes forever ok so last line in here. first use wget to get the index.html file it will take about 1-2 seconds. then open oit using any browser you want and tell me what happend with=20 squid on... for me it took a second to show up.. the same page just from http://ireport.cnn.com/ that loads every thing.= .. takes to *render* a long time. *so the guys who asked.. that is the case.* what i did was to get the page ( i see on the top of the page the RSS=20 feed is here, in firefox) i stop the page from loading got into the source copy the source paste it in new html file... load the file in firefox and get it without all the css ... the pictures=20 and every thing but not the look they wanted. On 24/01/2011 06:35, Max Feil wrote: Already did use Wireshark. Here is some more info: If you look through the traces you'll notice that at some point Squid sends a TCP [FIN, ACK] right in the middle of a connection for seemingly no reason. (Attempting to close the connection) The server ignores this and sends the rest of the data, which Squid responds to with TCP RST (request to reset) since it now believes the connection to be closed. From the browser side it seems to be given no notification that the connection was closed (and indeed I can see no reason why it should be closed) so it seems to sit around doing nothing as it may have reached the max connections limit. After about 2 minutes (possibly related to a persistent connection timeout limit in squid) Squid seems to terminate all the connections with FIN,ACKs. The browser then seems to realize its connections are gone and it requests the remaining resources resulting in a bunch of TCP SYNs followed by the rest of the resources. Why it does this in the middle of connections we still have no clue, however turning off server_persistent_connections seems to make it load fast. However this is probably a bad idea in general... Max -Original Message- From: Henrik Nordström [mailto:hen...@henriknordstrom.net] Sent: Sunday, January 23, 2011 7:16 PM To: Max Feil Cc: squid-users@squid-cache.org Subject: RE: [squid-users] Squid 3.x very slow loading on ireport.cnn.com tor 2011-01-20 klockan 02:50 -0500 skrev Max Feil: Thanks. I am looking at the squid access.log and the delay is caused by a GET which for some reason does not result in a response from the server. Either there is no response or Squid is missing the response. After a 120 second time-out the page continues loading, but the end result may be malformed due to the object which did not load. I would take a peek at the traffic using wireshark to get some insight in what is going on there. REgards Henrik
Re: [squid-users] Squid 3.x very slow loading on ireport.cnn.com
On 20/01/11 13:31, Max Feil wrote: I'm wondering if anybody knows what might be causing this. I've confirmed this problem in linux builds of Squid 3.0, 3.1.1, 3.1.10 and 3.2.0.4. Using firefox (or probably any browser - it also happens in a webkit based browser under development) clear the browser's disk cache and try to load or reload http://ireport.cnn.com (with proxy address/port set to Squid of course). Loading the page takes a very long time (several minutes) even on a fast network connection. Take Squid out of the mix and everything loads in seconds. This is using the default squid.conf file. The problem does not happen in Squid 2.7! Thanks, Max There are 101 different objects assembled into that one page coming from 10 different domains. Browsers set a very low limit on the amount of connections and objects fetched in parallel when using a proxy as compared to going direct. Large pages like this make the speed difference more noticeable. That will account for some of the extra time. But should not be taking that much longer. You will need to find out which objects are taking too long (firebug or the webkit dev tools should help) and then figure out why them. Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.10 Beta testers wanted for 3.2.0.4
RE: [squid-users] Squid 3.x very slow loading on ireport.cnn.com
Thanks. I am looking at the squid access.log and the delay is caused by a GET which for some reason does not result in a response from the server. Either there is no response or Squid is missing the response. After a 120 second time-out the page continues loading, but the end result may be malformed due to the object which did not load. The error object is different every time and seems random! So the page never loads properly with Squid 3.x and takes about 125 seconds to load. It always loads properly without Squid and takes about 5 seconds to load. It always loads properly using Squid 2.7 and takes about 5 seconds to load. For consistency in tracking the problem down, I have Squid's disk and memory caches disabled so every client request is a cache miss. Strange eh? Max P.S. I am debugging natively on my Ubuntu 10.10 64 bit laptop using Firefox, but the original problem comes from an embedded device running the QNX RTOS using a libcurl based WebKit browser (both the browser and Squid are running on 127.0.0.1 in each case, but this problem happens across the network as well). -Original Message- From: Amos Jeffries [mailto:squ...@treenet.co.nz] Sent: Wednesday, January 19, 2011 9:18 PM To: squid-users@squid-cache.org Subject: Re: [squid-users] Squid 3.x very slow loading on ireport.cnn.com On 20/01/11 13:31, Max Feil wrote: I'm wondering if anybody knows what might be causing this. I've confirmed this problem in linux builds of Squid 3.0, 3.1.1, 3.1.10 and 3.2.0.4. Using firefox (or probably any browser - it also happens in a webkit based browser under development) clear the browser's disk cache and try to load or reload http://ireport.cnn.com (with proxy address/port set to Squid of course). Loading the page takes a very long time (several minutes) even on a fast network connection. Take Squid out of the mix and everything loads in seconds. This is using the default squid.conf file. The problem does not happen in Squid 2.7! Thanks, Max There are 101 different objects assembled into that one page coming from 10 different domains. Browsers set a very low limit on the amount of connections and objects fetched in parallel when using a proxy as compared to going direct. Large pages like this make the speed difference more noticeable. That will account for some of the extra time. But should not be taking that much longer. You will need to find out which objects are taking too long (firebug or the webkit dev tools should help) and then figure out why them. Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.10 Beta testers wanted for 3.2.0.4
