Re: [squid-users] Cache firefox ok, chrome... not!
On Sat, 2010-12-11 at 02:59 +1300, Amos Jeffries wrote: ... Second thing I notice is the absence of ETag features for variant identification. Which Squid could use to identify that Chrome wants the cached object, or that the 304 response Chrome got is allowed to update the firefox variant's object details. Chrome is sending an If-Modified-Since header and the 304 applies to that. So if Squid is behaving properly and not sharing the variants with different possible encodings then it has no stored object to replace the firefox variant with and each IMS request will be a MISS. To test this out there are some permutations that will show if it's working: - check two Chrome requests in a row. When the browser cache is erased before testing and between requests. (good behaviour: the second is a HIT). Cool! Clearing cache between requests makes the second a HIT with chrome! First request after clearing browser's cache: 1292054351.774 37 127.0.0.1 TCP_MISS/200 617 GET http://www.elpais.com/css/i_portadillas.css? - DIRECT/80.156.250.8 text/css ... Second request after clearing browser's cache: 1292054375.498 1 127.0.0.1 TCP_MEM_HIT/200 624 GET http://www.elpais.com/css/i_portadillas.css? - NONE/- text/css ... For a fix: Adding ETag support by the web server may fix this or reduce the MISS a lot. Grats, you solved the issue! But I dont want to ask hundreds of sites to apply the etag to their servers. Isn't there a way of using squid and chrome and cache gifs, jpgs and css as firefox does? Maybe some obscure squid ~please-ignore-evil-etag~ header? Or should we migrate to firefox to use caching? Thanks! -- Rodolfo Alcazar Portillo - nosp...@gmail.com otbits.blogspot.com / counter.li.org: #367962 -- Ich will Microsoft wirklich nicht zerstören. Das wird nur ein gänzlich unbeabsichtigter Nebeneffekt sein. - Linus Torvalds
Re: [squid-users] Cache firefox ok, chrome... not!
On 11/12/10 21:37, Rodolfo Alcazar Portillo wrote: On Sat, 2010-12-11 at 02:59 +1300, Amos Jeffries wrote: ... Second thing I notice is the absence of ETag features for variant identification. Which Squid could use to identify that Chrome wants the cached object, or that the 304 response Chrome got is allowed to update the firefox variant's object details. Chrome is sending an If-Modified-Since header and the 304 applies to that. So if Squid is behaving properly and not sharing the variants with different possible encodings then it has no stored object to replace the firefox variant with and each IMS request will be a MISS. To test this out there are some permutations that will show if it's working: - check two Chrome requests in a row. When the browser cache is erased before testing and between requests. (good behaviour: the second is a HIT). Cool! Clearing cache between requests makes the second a HIT with chrome! First request after clearing browser's cache: 1292054351.774 37 127.0.0.1 TCP_MISS/200 617 GET http://www.elpais.com/css/i_portadillas.css? - DIRECT/80.156.250.8 text/css ... Second request after clearing browser's cache: 1292054375.498 1 127.0.0.1 TCP_MEM_HIT/200 624 GET http://www.elpais.com/css/i_portadillas.css? - NONE/- text/css ... For a fix: Adding ETag support by the web server may fix this or reduce the MISS a lot. Grats, you solved the issue! But I dont want to ask hundreds of sites to apply the etag to their servers. Isn't there a way of using squid and chrome and cache gifs, jpgs and css as firefox does? Maybe some obscure squid ~please-ignore-evil-etag~ header? Or should we migrate to firefox to use caching? So this confirms the underlying problem is Squids lack of support for caching multiple variants objects simultaneously. Combined with Squid not converting IMS requests to regular fetches on MISS. ETag avoids the problem, as will using a uniform set of browsers due to their uniform feature support (regardless of which particular is picked). It's just more visible with Chrome vs Firefox due to sdch encoding support. Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.9 Beta testers wanted for 3.2.0.3
Re: [squid-users] Cache firefox ok, chrome... not!
[sorry, if this message repeats, I've got a rejection, and not sure if it arrived.] On Fri, 2010-12-10 at 17:03 +1300, Amos Jeffries wrote: Its clear from your message that something other than the usual timing headers are affecting things. Yes, but I suppose it has no relation with this ffox vs chrome issue. Sadly the other headers I was going to check are not there, can you supply the full headers please? use a.b.c.d or example.invalid to obscure any IPs or domains you don't want to publish. No problem, find them attached. Remember: ignore-cc is already set -squid restarted and reloaded several times-! The example is not a first petition, I mean, I reloaded the page a couple of times so squid store it before piping the logs. Firefox keeps doing fine HITs: 1291985100.121 0 127.0.0.1 TCP_MEM_HIT/200 625 GET http://www.elpais.com/css/i_portadillas.css? - NONE/- text/css meanwhile chrome -same petition- keeps failing: 1291985161.424 29 127.0.0.1 TCP_MISS/304 340 GET http://www.elpais.com/css/i_portadillas.css? - DIRECT/62.73.184.40 text/css Thanks for your time. -- Rodolfo Alcazar Portillo - nosp...@gmail.com otbits.blogspot.com / counter.li.org: #367962 -- Es gibt kein unmoralisches Buch. Bücher sind gut geschrieben oder schlecht geschrieben. Das ist alles. - Oscar Wilde Taken at 13:26:23 local (Spain/Madrid), 12:26:23 GMT MAIN PAGE REQUEST: == GET http://www.elpais.com/ HTTP/1.1 Host: www.elpais.com Proxy-Connection: keep-alive Cache-Control: max-age=0 Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.215 Safari/534.10 Accept-Encoding: gzip,deflate,sdch Accept-Language: en,en-GB;q=0.8,de-DE;q=0.6,de;q=0.4 Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3 Cookie: elPaisGlobal=0; uid=W9g/8E0AewO8wDcTAxViAg==#62f0a5cf7c3eca72da28981e5fdbce85; cUser=nouser; s_nr=1291884415836; s_cc=true; s_sq=%5B%5BB%5D%5D If-Modified-Since: Fri, 10 Dec 2010 12:25:56 GMT [syncs, acks, etc.] RESPONSE HTTP/1.0 200 OK Content-Length: 40013 Content-Type: text/html; charset=iso-8859-1 Cache-Control: max-age=10 Content-Encoding: gzip Server: Apache Last-Modified: Fri, 10 Dec 2010 12:25:59 GMT Date: Fri, 10 Dec 2010 12:26:23 GMT Vary: Accept-Encoding X-Cache: MISS from rodolfoap.rap.es X-Cache-Lookup: HIT from rodolfoap.rap.es:3128 Via: 1.0 rodolfoap.rap.es (squid/3.1.9) Connection: keep-alive [data] CSS REQUEST === GET http://www.elpais.com/css/i_portadillas.css?update=1320101204 HTTP/1.1 Host: www.elpais.com Proxy-Connection: keep-alive Referer: http://www.elpais.com/ Cache-Control: max-age=0 If-Modified-Since: Sat, 04 Dec 2010 21:22:54 GMT Accept: text/css,*/*;q=0.1 User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.215 Safari/534.10 Accept-Encoding: gzip,deflate,sdch Accept-Language: en,en-GB;q=0.8,de-DE;q=0.6,de;q=0.4 Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3 Cookie: elPaisGlobal=0; uid=W9g/8E0AewO8wDcTAxViAg==#62f0a5cf7c3eca72da28981e5fdbce85; cUser=nouser; s_nr=1291884415836; s_cc=true; s_sq=%5B%5BB%5D%5D [data] RESPONSE HTTP/1.0 304 Not Modified Content-Type: text/css Last-Modified: Sat, 04 Dec 2010 21:22:54 GMT Cache-Control: max-age=3 Date: Fri, 10 Dec 2010 12:26:23 GMT Vary: Accept-Encoding X-Cache: MISS from rodolfoap.rap.es X-Cache-Lookup: MISS from rodolfoap.rap.es:3128 Via: 1.0 rodolfoap.rap.es (squid/3.1.9) Connection: keep-alive [data] [rodolfoap] /root # tail -f /var/log/squid/access.log 1291983983.400 38 127.0.0.1 TCP_MISS/304 340 GET http://www.elpais.com/css/i_portadillas.css? - DIRECT/80.156.250.17 text/css [NOTE: the previous line corresponds to the headers example] 1291983983.405 40 127.0.0.1 TCP_MISS/304 340 GET http://www.elpais.com/css/promos_marketplace.css? - DIRECT/80.156.250.17 text/css 1291983983.410 44 127.0.0.1 TCP_MISS/304 340 GET http://www.elpais.com/css/ep-portada.css? - DIRECT/80.156.250.17 text/css 1291983983.415 47 127.0.0.1 TCP_MISS/304 340 GET http://www.elpais.com/css/pie.css? - DIRECT/80.156.250.17 text/css 1291983983.420 50 127.0.0.1 TCP_MISS/304 340 GET http://www.elpais.com/css/eskup/eskup_ventana_inferior.css? - DIRECT/80.156.250.17 text/css 1291983983.429 32 127.0.0.1 TCP_MISS/304 340 GET http://www.elpais.com/css/marcadores.css? - DIRECT/80.156.250.17 text/css 1291983983.440 37 127.0.0.1 TCP_MISS/304 340 GET http://www.elpais.com/css/especiales/pgc/modulos.css? - DIRECT/80.156.250.17 text/css 1291983983.530 52 127.0.0.1 TCP_MISS/304 340 GET http://www.elpais.com/css/comunes.css? - DIRECT/80.156.250.17 text/css 1291983983.540 53 127.0.0.1 TCP_MISS/304 340 GET http://www.elpais.com/css/portadillas.css? - DIRECT/80.156.250.17 text/css 1291983983.550 56
Re: [squid-users] Cache firefox ok, chrome... not!
On 11/12/10 02:04, Rodolfo Alcazar Portillo wrote: [sorry, if this message repeats, I've got a rejection, and not sure if it arrived.] On Fri, 2010-12-10 at 17:03 +1300, Amos Jeffries wrote: Its clear from your message that something other than the usual timing headers are affecting things. Yes, but I suppose it has no relation with this ffox vs chrome issue. Sadly the other headers I was going to check are not there, can you supply the full headers please? use a.b.c.d or example.invalid to obscure any IPs or domains you don't want to publish. No problem, find them attached. Remember: ignore-cc is already set -squid restarted and reloaded several times-! The example is not a first petition, I mean, I reloaded the page a couple of times so squid store it before piping the logs. Firefox keeps doing fine HITs: Okay, there are two things I notice about these requests. First thing is that the content varies on Accept-Encoding header, plus Chrome advertises support for sdch encoding. AFAIK firefox does not support that encoding format so will not advertise it. This means Chrome and firefox are requesting completely different objects in the cache. 3.x does not yet support caching multiple variants at once. Second thing I notice is the absence of ETag features for variant identification. Which Squid could use to identify that Chrome wants the cached object, or that the 304 response Chrome got is allowed to update the firefox variant's object details. Chrome is sending an If-Modified-Since header and the 304 applies to that. So if Squid is behaving properly and not sharing the variants with different possible encodings then it has no stored object to replace the firefox variant with and each IMS request will be a MISS. To test this out there are some permutations that will show if it's working: - check two Chrome requests in a row. When the browser cache is erased before testing and between requests. (good behaviour: the second is a HIT). -check two Chrome requests in a row. when the browser cache is cleared before each request AND the squid cache is empty before the first request. (good behaviour: first is a MISS/200, second is a HIT). - check a firefox request after a Chrome request. when both browser caches are cleared before requesting and squid cache is empty to start with. (good behaviour: both are MISS/200). For a fix: Adding ETag support by the web server may fix this or reduce the MISS a lot. Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.9 Beta testers wanted for 3.2.0.3
Re: [squid-users] Cache firefox ok, chrome... not!
On Wed, 2010-12-08 at 18:19 +1300, Amos Jeffries wrote: On 08/12/10 16:51, Rodolfo Alcazar Portillo wrote: With Firefox, TCP_MEM_HIT: 1291778947.206 0 127.0.0.1 TCP_MEM_HIT/200 626 GET http://www.elpais.com/css/i_portadillas.css? - NONE/- text/css ... The VERY SAME PAGE with Google Chrome, several reloads, always TCP_MISS: 1291778941.929108 127.0.0.1 TCP_MISS/304 340 GET http://www.elpais.com/css/i_portadillas.css? - DIRECT/80.157.169.17 text/css ... You need to look at the headers they are sending Squid. See what is different between the two. The message turns to TCP_REFRESH_UNMODIFIED/304 after clearing chrome's cache, but it still grabbing the css, jpg, etc. from the server for each request. Answering to your questions: Squid 3.1.9, I'm john doe browsing an unknown server. Don't want to cache all by ignore-cc. Just want to cache things like css or jpg. Please check the chrome headers (I deleted the irrelevant, time of this shot: 09:13): BROWSER REQUEST: GET http://www.elpais.com/css/i_portadillas.css?update=1320101204 HTTP/1.1 Cache-Control: max-age=0 If-Modified-Since: Sat, 04 Dec 2010 21:22:54 GMT ... SQUID RESPONSE: === HTTP/1.0 304 Not Modified Last-Modified: Sat, 04 Dec 2010 21:22:54 GMT Cache-Control: max-age=3 Date: Thu, 09 Dec 2010 08:12:52 GMT Warning: 110 squid/3.1.9 Response is stale Warning: 111 squid/3.1.9 Revalidation failed X-Cache: HIT from rodolfoap.rap.es X-Cache-Lookup: HIT from rodolfoap.rap.es:3128 ... Squid log for this request: 1291882372.347125 127.0.0.1 TCP_REFRESH_UNMODIFIED/304 432 GET http://www.elpais.com/css/i_portadillas.css? - DIRECT/80.156.250.8 text/css (the same produces a HIT with firefox! chrome grabs the css every time). What are those warnings? Thanks. -- Rodolfo Alcazar Portillo - nosp...@gmail.com otbits.blogspot.com / counter.li.org: #367962 -- Any sufficiently advanced technology is indistinguishable from a rigged demo -- Andy Finkel
Re: [squid-users] Cache firefox ok, chrome... not!
On 09/12/10 21:28, Rodolfo Alcazar Portillo wrote: On Wed, 2010-12-08 at 18:19 +1300, Amos Jeffries wrote: On 08/12/10 16:51, Rodolfo Alcazar Portillo wrote: With Firefox, TCP_MEM_HIT: 1291778947.206 0 127.0.0.1 TCP_MEM_HIT/200 626 GET http://www.elpais.com/css/i_portadillas.css? - NONE/- text/css ... The VERY SAME PAGE with Google Chrome, several reloads, always TCP_MISS: 1291778941.929108 127.0.0.1 TCP_MISS/304 340 GET http://www.elpais.com/css/i_portadillas.css? - DIRECT/80.157.169.17 text/css ... You need to look at the headers they are sending Squid. See what is different between the two. The message turns to TCP_REFRESH_UNMODIFIED/304 after clearing chrome's cache, but it still grabbing the css, jpg, etc. from the server for each request. Answering to your questions: Squid 3.1.9, I'm john doe browsing an unknown server. Don't want to cache all by ignore-cc. Just want to cache things like css or jpg. Please check the chrome headers (I deleted the irrelevant, time of this shot: 09:13): Its clear from your message that something other than the usual timing headers are affecting things. Sadly the other headers I was going to check are not there, can you supply the full headers please? use a.b.c.d or example.invalid to obscure any IPs or domains you don't want to publish. BROWSER REQUEST: GET http://www.elpais.com/css/i_portadillas.css?update=1320101204 HTTP/1.1 Cache-Control: max-age=0 If-Modified-Since: Sat, 04 Dec 2010 21:22:54 GMT ... SQUID RESPONSE: === HTTP/1.0 304 Not Modified Last-Modified: Sat, 04 Dec 2010 21:22:54 GMT Cache-Control: max-age=3 Date: Thu, 09 Dec 2010 08:12:52 GMT Warning: 110 squid/3.1.9 Response is stale Warning: 111 squid/3.1.9 Revalidation failed X-Cache: HIT from rodolfoap.rap.es X-Cache-Lookup: HIT from rodolfoap.rap.es:3128 ... Squid log for this request: 1291882372.347125 127.0.0.1 TCP_REFRESH_UNMODIFIED/304 432 GET http://www.elpais.com/css/i_portadillas.css? - DIRECT/80.156.250.8 text/css (the same produces a HIT with firefox! chrome grabs the css every time). What are those warnings? warning 110 indicates the object is known to be past its expiry time. warning 111 indicates that squid did try to check for a newer version but the server check failed. Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.9 Beta testers wanted for 3.2.0.3
Re: [squid-users] Cache firefox ok, chrome... not!
On 08/12/10 16:51, Rodolfo Alcazar Portillo wrote: Hi. Fedora 14/x86_64 With Firefox, TCP_MEM_HIT: # tail -f /var/log/squid/access.log |grep css 1291778947.206 0 127.0.0.1 TCP_MEM_HIT/200 626 GET http://www.elpais.com/css/i_portadillas.css? - NONE/- text/css 1291778947.210 0 127.0.0.1 TCP_MEM_HIT/200 693 GET http://www.elpais.com/css/promos_marketplace.css? - NONE/- text/css 1291778947.212 0 127.0.0.1 TCP_MEM_HIT/200 4501 GET http://www.elpais.com/css/ep-portada.css? - NONE/- text/css 1291778947.214 0 127.0.0.1 TCP_MEM_HIT/200 3403 GET http://www.elpais.com/css/pie.css? - NONE/- text/css 1291778947.217 0 127.0.0.1 TCP_MEM_HIT/200 1926 GET http://www.elpais.com/css/eskup/eskup_ventana_inferior.css? - NONE/- text/css 1291778947.222 0 127.0.0.1 TCP_MEM_HIT/200 3641 GET http://www.elpais.com/css/marcadores.css? - NONE/- text/css 1291778947.225 0 127.0.0.1 TCP_MEM_HIT/200 3524 GET http://www.elpais.com/css/especiales/pgc/modulos.css? - NONE/- text/css 1291778947.557 0 127.0.0.1 TCP_MEM_HIT/200 18220 GET http://www.elpais.com/css/comunes.css? - NONE/- text/css 1291778947.558 0 127.0.0.1 TCP_MEM_HIT/200 13822 GET http://www.elpais.com/css/portadillas.css? - NONE/- text/css 1291778947.559 0 127.0.0.1 TCP_MEM_HIT/200 2297 GET http://www.elpais.com/css/n_fotos.css? - NONE/- text/css 1291778947.562 0 127.0.0.1 TCP_MEM_HIT/200 842 GET http://www.elpais.com/css/estilos_antiguos.css? - NONE/- text/css 1291778947.692 0 127.0.0.1 TCP_MEM_HIT/200 1041 GET http://www.elpais.com/css/interstitial/interstitial.css - NONE/- text/css 1291778949.238 0 127.0.0.1 TCP_MEM_HIT/200 2599 GET http://eskup.elpais.com/Estilos/v1.x/v1.0/eskup_portada_elpais_superior_escribir.css - NONE/- text/css The VERY SAME PAGE with Google Chrome, several reloads, always TCP_MISS: 1291778941.929108 127.0.0.1 TCP_MISS/304 340 GET http://www.elpais.com/css/i_portadillas.css? - DIRECT/80.157.169.17 text/css 1291778941.947125 127.0.0.1 TCP_MISS/304 340 GET http://www.elpais.com/css/promos_marketplace.css? - DIRECT/80.157.169.17 text/css 1291778941.959136 127.0.0.1 TCP_MISS/304 340 GET http://www.elpais.com/css/ep-portada.css? - DIRECT/80.157.169.17 text/css 1291778941.977132 127.0.0.1 TCP_MISS/304 340 GET http://www.elpais.com/css/pie.css? - DIRECT/80.157.169.17 text/css 1291778941.996 77 127.0.0.1 TCP_MISS/304 340 GET http://www.elpais.com/css/marcadores.css? - DIRECT/80.157.169.17 text/css 1291778942.008 84 127.0.0.1 TCP_MISS/304 340 GET http://www.elpais.com/css/especiales/pgc/modulos.css? - DIRECT/80.157.169.17 text/css 1291778942.253 93 127.0.0.1 TCP_MISS/304 340 GET http://www.elpais.com/css/comunes.css? - DIRECT/80.157.169.17 text/css 1291778942.268 89 127.0.0.1 TCP_MISS/304 340 GET http://www.elpais.com/css/portadillas.css? - DIRECT/80.157.169.17 text/css 1291778942.286 93 127.0.0.1 TCP_MISS/304 340 GET http://www.elpais.com/css/n_fotos.css? - DIRECT/80.157.169.17 text/css 1291778942.301 88 127.0.0.1 TCP_MISS/304 340 GET http://www.elpais.com/css/estilos_antiguos.css? - DIRECT/80.157.169.17 text/css 1291778942.323 59 127.0.0.1 TCP_MISS/304 340 GET http://www.elpais.com/css/interstitial/interstitial.css - DIRECT/80.157.169.17 text/css 1291778944.734378 127.0.0.1 TCP_REFRESH_UNMODIFIED/304 460 GET http://eskup.elpais.com/Estilos/v1.x/v1.0/eskup_portada_elpais_superior_escribir.css - DIRECT/91.216.63.240 - How can I change this Chrome behaviour? Am I doing something wrong? You need to look at the headers they are sending Squid. See what is different between the two. Both of these works the same (top of my refresh-pattern list): refresh_pattern ^http: 0 100% 1440 refresh_pattern ^http: 0 100% 1440 override-expire override-lastmod reload-into-ims ignore-reload ignore-no-cache ignore-no-store ignore-must-revalidate ignore-private ignore-auth refresh-ims Which release version of Squid? And are you doing this on a reverse proxy for elpais.com? If you have squid 3.1+ use the ignore-cc option on your accel http_port instead of all that horrid algorithm overriding. If you are a separate admin do the whole world a favor and contact them about fixing their website headers so their content caches appropriately. That the server produces non-compressed objects with the wrong Vary: header. http://redbot.org/?uri=http%3A%2F%2Fwww.elpais.com%2Fcss%2Finterstitial%2Finterstitial.css Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.9 Beta testers wanted for 3.2.0.3
