Re: Cacheability - changed in Varnish 2?
Poul-Henning Kamp wrote: As far as I can tell, a zero TTL (number after RFC) can only happen here if your default_ttl parameter is set to zero, OR if there is clock-skew between the varnish machine and the backend machine. Right on! Backend is running on the same box as Varnish for this test. But: you are right about default TTL being 0. After setting default TTL to 60, everything runs smooth. So this issue was not about Varnish versions, but different default TTL configs. Thanks alot. /Anton ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Cacheability - changed in Varnish 2?
Poul-Henning Kamp wrote: I hope we don't ship varnish that way on any platforms, the default ttl should be 120 seconds... No, the 0 default ttl comes from this one: http://pypi.python.org/pypi/plone.recipe.varnish From the changelog: Add a default_ttl of zero seconds to the Varnish runner to avoid a Varnish bug with the handling of an Expires header with a date in the past. /Anton ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Cacheability - changed in Varnish 2?
Poul-Henning Kamp wrote: Can you get me varnishlog of such a request ? I'm not able to tell from your description which path we're talking about. Sure, just thought this was intended behavior so I wouldn't bore with details. Here we go. Running Varnish 2.0.2 on Ubuntu, has tested on Debian Etch 64bit as well. Failure: 9 SessionOpen c 127.0.0.1 41768 127.0.0.1:8000 9 ReqStart c 127.0.0.1 41768 1998529221 9 RxRequestc GET 9 RxURLc /test_varnish 9 RxProtocol c HTTP/1.1 9 RxHeader c Host: localhost:8000 9 RxHeader c User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.5) Gecko/2008121622 Ubuntu/8.10 (intrepid) Firefox/3.0.5 9 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 9 RxHeader c Accept-Language: en-us,en;q=0.5 9 RxHeader c Accept-Encoding: gzip,deflate 9 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 9 RxHeader c Keep-Alive: 300 9 RxHeader c Connection: keep-alive 9 VCL_call c recv 9 VCL_return c lookup 9 VCL_call c hash 9 VCL_return c hash 9 HitPass c 1998529216 9 VCL_call c pass 9 VCL_return c pass 9 Backend c 10 backend_0 backend_0 10 TxRequest- GET 10 TxURL- /test_varnish 10 TxProtocol - HTTP/1.1 10 TxHeader - Host: localhost:8000 10 TxHeader - User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.5) Gecko/2008121622 Ubuntu/8.10 (intrepid) Firefox/3.0.5 10 TxHeader - Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 10 TxHeader - Accept-Language: en-us,en;q=0.5 10 TxHeader - Accept-Encoding: gzip,deflate 10 TxHeader - Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 10 TxHeader - X-Varnish: 1998529221 10 TxHeader - X-Forwarded-For: 127.0.0.1 10 RxProtocol - HTTP/1.1 10 RxStatus - 200 10 RxResponse - OK 10 RxHeader - Server: Zope/(Zope 2.10.6-final, python 2.4.5, linux2) ZServer/1.1 Plone/3.1.5.1 10 RxHeader - Date: Wed, 28 Jan 2009 10:13:23 GMT 10 RxHeader - Content-Length: 4 10 RxHeader - Content-Type: text/plain; charset=utf-8 9 ObjProtocol c HTTP/1.1 9 ObjStatusc 200 9 ObjResponse c OK 9 ObjHeaderc Server: Zope/(Zope 2.10.6-final, python 2.4.5, linux2) ZServer/1.1 Plone/3.1.5.1 9 ObjHeaderc Date: Wed, 28 Jan 2009 10:13:23 GMT 9 ObjHeaderc Content-Type: text/plain; charset=utf-8 10 BackendReuse - backend_0 9 TTL c 1998529221 RFC 0 1233137603 0 0 0 0 9 VCL_call c fetch 9 TTL c 1998529221 VCL 86400 1233137604 9 VCL_return c pass 9 Length c 4 9 VCL_call c deliver 9 VCL_return c deliver 9 TxProtocol c HTTP/1.1 9 TxStatus c 200 9 TxResponse c OK 9 TxHeader c Server: Zope/(Zope 2.10.6-final, python 2.4.5, linux2) ZServer/1.1 Plone/3.1.5.1 9 TxHeader c Content-Type: text/plain; charset=utf-8 9 TxHeader c Content-Length: 4 9 TxHeader c X-Varnish-Action: FETCH (pass - not cacheable) 9 TxHeader c Date: Wed, 28 Jan 2009 10:13:23 GMT 9 TxHeader c X-Varnish: 1998529221 9 TxHeader c Age: 0 9 TxHeader c Via: 1.1 varnish 9 TxHeader c Connection: keep-alive 9 ReqEnd c 1998529221 1233137603.757895708 1233137603.761220932 0.2 0.003257036 0.68188 0 StatAddr - 127.0.0.1 0 760 6 6 0 5 6 1970 24 Same backend header, just added a cache-control request key. Succes: 9 SessionOpen c 127.0.0.1 41774 127.0.0.1:8000 9 ReqStart c 127.0.0.1 41774 1998529224 9 RxRequestc GET 9 RxURLc /test_varnish 9 RxProtocol c HTTP/1.1 9 RxHeader c Host: localhost:8000 9 RxHeader c User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.5) Gecko/2008121622 Ubuntu/8.10 (intrepid) Firefox/3.0.5 9 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 9 RxHeader c Accept-Language: en-us,en;q=0.5 9 RxHeader c Accept-Encoding: gzip,deflate 9 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 9 RxHeader c Keep-Alive: 300 9 RxHeader c Connection: keep-alive 9 VCL_call c recv 9 VCL_return c lookup 9 VCL_call c hash 9 VCL_return c hash 9 HitPass c 1998529216 9 VCL_call c pass 9 VCL_return c pass 9 Backend c 10 backend_0 backend_0 10 TxRequest- GET 10 TxURL- /test_varnish 10 TxProtocol - HTTP/1.1 10 TxHeader - Host: localhost:8000 10 TxHeader - User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.5) Gecko/2008121622 Ubuntu/8.10 (intrepid) Firefox/3.0.5 10 TxHeader - Accept:
Re: Cacheability - changed in Varnish 2?
Hi Poul-Henning, 9 VCL_call c hash 9 VCL_return c hash 9 HitPass c 1998529216 9 VCL_call c pass 9 VCL_return c pass This is the important bit, you have a cached object that says this object cannot be cached, but should be passed. Thanks for digging into this and pointing out the relevant part. I see that my log-snippets wasn't that useful to nail the issue. New try. First, a request with no expire or cache-control header. 9 RxURLc /test_varnish_noheader 9 RxProtocol c HTTP/1.1 9 RxHeader c Host: localhost:8000 9 RxHeader c User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.5) Gecko/2008121622 Ubuntu/8.10 (intrepid) Firefox/3.0.5 9 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 9 RxHeader c Accept-Language: en-us,en;q=0.5 9 RxHeader c Accept-Encoding: gzip,deflate 9 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 9 RxHeader c Keep-Alive: 300 9 RxHeader c Connection: keep-alive 9 VCL_call c recv 9 VCL_return c lookup 9 VCL_call c hash 9 VCL_return c hash 9 VCL_call c miss 9 VCL_return c fetch 10 BackendOpen b backend_0 127.0.0.1 36975 127.0.0.1 8080 9 Backend c 10 backend_0 backend_0 10 TxRequestb GET 10 TxURLb /test_varnish_noheader 10 TxProtocol b HTTP/1.1 10 TxHeader b Host: localhost:8000 10 TxHeader b User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.5) Gecko/2008121622 Ubuntu/8.10 (intrepid) Firefox/3.0.5 10 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 10 TxHeader b Accept-Language: en-us,en;q=0.5 10 TxHeader b Accept-Encoding: gzip,deflate 10 TxHeader b Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 10 TxHeader b X-Varnish: 1495399095 10 TxHeader b X-Forwarded-For: 127.0.0.1 10 RxProtocol b HTTP/1.1 10 RxStatus b 200 10 RxResponse b OK 10 RxHeader b Server: Zope/(Zope 2.10.6-final, python 2.4.5, linux2) ZServer/1.1 Plone/3.1.5.1 10 RxHeader b Date: Thu, 29 Jan 2009 00:10:40 GMT 10 RxHeader b Content-Length: 4 10 RxHeader b Content-Type: text/plain; charset=utf-8 9 ObjProtocol c HTTP/1.1 9 ObjStatusc 200 9 ObjResponse c OK 9 ObjHeaderc Server: Zope/(Zope 2.10.6-final, python 2.4.5, linux2) ZServer/1.1 Plone/3.1.5.1 9 ObjHeaderc Date: Thu, 29 Jan 2009 00:10:40 GMT 9 ObjHeaderc Content-Type: text/plain; charset=utf-8 10 BackendReuse b backend_0 9 TTL c 1495399095 RFC 0 1233187840 0 0 0 0 9 VCL_call c fetch 9 TTL c 1495399095 VCL 86400 1233187841 9 VCL_return c pass 9 Length c 4 9 VCL_call c deliver 9 VCL_return c deliver pass is called inside vcl_fetch because the object is classified as !cachable. Subsequent requests gives same result as in my first mail: 9 VCL_call c recv 9 VCL_return c lookup 9 VCL_call c hash 9 VCL_return c hash 9 HitPass c 1495399095 9 VCL_call c pass 9 VCL_return c pass 9 Backend Then, restart Varnish, same vcl file, same backend reponse except that we are now adding a cache-control header: 9 RxRequestc GET 9 RxURLc /test_varnish 9 RxProtocol c HTTP/1.1 9 RxHeader c Host: localhost:8000 9 RxHeader c User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.5) Gecko/2008121622 Ubuntu/8.10 (intrepid) Firefox/3.0.5 9 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 9 RxHeader c Accept-Language: en-us,en;q=0.5 9 RxHeader c Accept-Encoding: gzip,deflate 9 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 9 RxHeader c Keep-Alive: 300 9 RxHeader c Connection: keep-alive 9 VCL_call c recv 9 VCL_return c lookup 9 VCL_call c hash 9 VCL_return c hash 9 VCL_call c miss 9 VCL_return c fetch 10 BackendOpen b backend_0 127.0.0.1 38871 127.0.0.1 8080 9 Backend c 10 backend_0 backend_0 10 TxRequestb GET 10 TxURLb /test_varnish 10 TxProtocol b HTTP/1.1 10 TxHeader b Host: localhost:8000 10 TxHeader b User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.5) Gecko/2008121622 Ubuntu/8.10 (intrepid) Firefox/3.0.5 10 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 10 TxHeader b Accept-Language: en-us,en;q=0.5 10 TxHeader b Accept-Encoding: gzip,deflate 10 TxHeader b Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 10 TxHeader b X-Varnish: 1198489031 10 TxHeader b X-Forwarded-For: 127.0.0.1 10 RxProtocol b HTTP/1.1 10
Re: [varnish] Re: Cacheability - changed in Varnish 2?
Ricardo Newbery skrev:
sub vcl_recv {
set req.grace = 120s;
set req.backend = backend_0;
}
Is this truly all you have in vcl_recv? This will mean that any cookied
requests will get passed. Is this intentional?
No, this is not a production setup. My problem is not that I cache too
much, but the opposite.
And yep, I know about the cookie issue:
http://markmail.org/message/pfpx7lanicpumsdg
Thanks for noticing.
/Anton
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Cacheability - changed in Varnish 2?
Hi there,
Is it just me, my setup or has cacheable classification changed
between Varnish 1 and Varnish 2?
In the old days you could just say e.g.
sub vcl_fetch {
set obj.ttl = 1d;
}
and Varnish would cache everything for one day.
I Varnish 2 it looks like the objects must have either Cache-Control
or Expires to be cached at all -- and then you can tamper with obj.ttl
afterwards. If the backend doesn't set those headers, set obj.ttl
doesn't seem have any effect.
It feels a bit limiting compared to the old days. If I want to cache
more aggressive its much easier just to change a few lines in the
Varnish conf than adding cache headers to the backend.
So is there a way to configure Varnish 2 to cache objects with no cache
headers?
/Anton
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Varnish Dirty Caching
Poul-Henning Kamp wrote: The critical question is how we define backend is down and how fast and efficient we can detect it. Ideas for how to express it in VCL are very welcome. Maybe naive: # First, we setup decide how to sniff that a backend is down # # options_ping: Send a HTTP OPTIONS (Perlbal does that) # timeout: If the backend does not answer within x seconds, it is # probably down # icp: Abuse the protocol. (Squid + Zope does that) backend.down_protocol = options_ping | timeout | icp # What is the timeout limit? backend.timeout = 30 # How long time should the backend be marked as down before we try # again? backend.retry_after = 300 # And then just use it if(backend.down) /Anton Stonor ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Varnish Dirty Caching
Poul-Henning Kamp wrote: The critical question is how we define backend is down and how fast and efficient we can detect it. Right. I tend to like the Perlbal approach: Issue a http OPTIONS and check if we get anything back from the backend. It is quite lightweight. /Anton ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Checking for purge prevents caching
Dag-Erling Smørgrav wrote: Anton Stonor [EMAIL PROTECTED] writes: At first glance that kind of works. When somebody hits ctrl-F5 the page gets purged and I get my custom error message. However suddenly the page is not served from the cache _at all_. That is unless I just hit F5, _then_ I get a cached page. Hitting F5 sends this request header in Firefox: Cache-Control: max-age=0 I don't send any caching headers on the backend if that matters, just relying on the default Varnish fallback to 120 sec. Any ideas? Hard to tell without logs... OK, here it is: FIRST NORMAL REQUEST - it should be served from the cache --- 0 StatAddr 127.0.0.1 0 264 2 6 0 0 2 1672 145 0 CLIRd ping 0 CLIWr 0 200 PONG 1175105738 0 CLIRd vcl.load boot /tmp/vcl.XXu1YKyr 0 CLIWr 0 200 Loaded /tmp/vcl.XXu1YKyr as boot 0 CLIRd vcl.use boot 0 CLIWr 0 200 0 CLIRd start 0 CLIWr 0 200 0 WorkThread 0xae04f1b4 start 13 SessionOpen c 127.0.0.1 42443 13 ReqStart c 127.0.0.1 42443 1440795409 13 RxRequestc GET 13 RxURLc /headerstuff 13 RxProtocol c HTTP/1.1 13 RxHeader c Host: localhost:9008 13 RxHeader c User-Agent: Mozilla/5.0 (X11; U; Linux i686; da-DK; rv:1.8.0.5) Gecko/20060731 Ubuntu/dapper-security Firefox/1.5.0.5 13 RxHeader c Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 13 RxHeader c Accept-Language: da,en-us;q=0.7,en;q=0.3 13 RxHeader c Accept-Encoding: gzip,deflate 13 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 13 RxHeader c Keep-Alive: 300 13 RxHeader c Connection: keep-alive 13 VCL_call c recv 13 VCL_return c lookup 13 VCL_call c miss 13 VCL_return c fetch 16 BackendOpen b default 127.0.0.1 54436 127.0.0.1 8080 16 BackendXID b 1440795409 13 Backend c 16 default 16 TxRequestb GET 16 TxURLb /headerstuff 16 TxProtocol b HTTP/1.1 16 TxHeader b Host: localhost:9008 16 TxHeader b User-Agent: Mozilla/5.0 (X11; U; Linux i686; da-DK; rv:1.8.0.5) Gecko/20060731 Ubuntu/dapper-security Firefox/1.5.0.5 16 TxHeader b Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 16 TxHeader b Accept-Language: da,en-us;q=0.7,en;q=0.3 16 TxHeader b Accept-Encoding: gzip,deflate 16 TxHeader b Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 16 TxHeader b X-Varnish: 1440795409 16 TxHeader b X-Forwarded-for: 127.0.0.1 16 RxProtocol b HTTP/1.1 16 RxStatus b 200 16 RxResponse b OK 16 RxHeader b Server: Zope/(Zope 2.10.2-final, python 2.4.3, linux2) ZServer/1.1 16 RxHeader b Date: Wed, 28 Mar 2007 18:15:40 GMT 16 RxHeader b Content-Length: 29 16 RxHeader b Content-Type: text/plain; charset=iso-8859-15 13 TTL c 1440795409 RFC 120 1175105740 1175105740 0 0 0 13 VCL_call c fetch 13 VCL_return c insert 13 ObjProtocol c HTTP/1.1 13 ObjStatusc 200 13 ObjResponse c OK 13 ObjHeaderc Server: Zope/(Zope 2.10.2-final, python 2.4.3, linux2) ZServer/1.1 13 ObjHeaderc Date: Wed, 28 Mar 2007 18:15:40 GMT 13 ObjHeaderc Content-Type: text/plain; charset=iso-8859-15 13 ObjHeaderc Content-Length: 29 16 BackendReuse b default 13 Length c 29 13 TxProtocol c HTTP/1.1 13 TxStatus c 200 13 TxResponse c OK 13 TxHeader c Server: Zope/(Zope 2.10.2-final, python 2.4.3, linux2) ZServer/1.1 13 TxHeader c Date: Wed, 28 Mar 2007 18:15:40 GMT 13 TxHeader c Content-Type: text/plain; charset=iso-8859-15 13 TxHeader c Content-Length: 29 13 TxHeader c X-Varnish: 1440795409 13 TxHeader c Age: 0 13 TxHeader c Via: 1.1 varnish 13 ReqEnd c 1440795409 1175105740.472784000 1175105740.642282000 0.000369000 0.009144000 0.160354000 SECOND NORMAL REQUEST - now, it should really be in the cache --- 0 StatAddr 127.0.0.1 0 0 1 1 0 0 1 240 29 0 CLIRd vcl.load boot /tmp/vcl.XXu1YKyr 0 CLIWr 0 200 Loaded /tmp/vcl.XXu1YKyr as boot 0 CLIRd vcl.use boot 0 CLIWr 0 200 0 CLIRd start 0 CLIWr 0 200 0 WorkThread 0xae04f1b4 start 13 SessionOpen c 127.0.0.1 42445 13 ReqStart c 127.0.0.1 42445 555485910 13 RxRequestc GET 13 RxURLc /headerstuff 13 RxProtocol c HTTP/1.1 13 RxHeader c Host: localhost:9008 13 RxHeader c User-Agent: Mozilla/5.0 (X11; U; Linux i686; da-DK; rv:1.8.0.5) Gecko/20060731 Ubuntu/dapper-security Firefox/1.5.0.5 13 RxHeader c
Re: Urchin
Arne wrote:
So if I understand you correctly (the list and the man-page) , then
practically nothing on my website would be cached using the default
configuration, due to (req.request == GET req.http.cookie) ?
Is there any way to ignore *only* the Google-cookies? They're
consistently named __utma, __utmb, __utmc and __utmz
You might want to place this default vcl statement
if (req.http.Authenticate || req.http.Cookie) {
... with something in the neighborhood of this
if (req.http.Authenticate || req.http.Cookie ~ __ut=) {
/Anton
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Urchin
I wrote: You might want to place this default vcl statement Please ignore that - it will have the opposite effect as intended. Sorry. /Anton ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
