Re: Handling graced objects between varnish servers.
Is there a way to tell or flag set if the fetched object is a graced object? On Apr 30, 2009, at 12:16 AM, Poul-Henning Kamp wrote: In message 6d6e4f5a-b9d6-41eb-9b08-27afec5d4...@funnyordie.com, Jeff Anderson writes: How long can I cache objects and keep them available as graced objects? As long as you want. If a front end varnish cache receives an object from a downstream varnish cache that was served as a graced object will the front end varnish cache declare it invalid? If so is there a way to bypass that invalidation? There is no indication passed along that an object is graced, so the downstream varnish sees it just like any other object and will interpret its validity based on the headers. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. --Jeff j...@funnyordie.com ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Handling graced objects between varnish servers.
Sorry. You answered that question below. I was hoping to capture graced object statistics and potentially manipulate headers of an object if it is graced. On Apr 30, 2009, at 12:16 AM, Poul-Henning Kamp wrote: In message 6d6e4f5a-b9d6-41eb-9b08-27afec5d4...@funnyordie.com, Jeff Anderson writes: How long can I cache objects and keep them available as graced objects? As long as you want. If a front end varnish cache receives an object from a downstream varnish cache that was served as a graced object will the front end varnish cache declare it invalid? If so is there a way to bypass that invalidation? There is no indication passed along that an object is graced, so the downstream varnish sees it just like any other object and will interpret its validity based on the headers. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. --Jeff j...@funnyordie.com ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Restart fetches the page but still displays a 503 to client
This client received a 503 even though it looks like the cache restart successfully fetched the page from the backend. Would the 5 discards in a row cause the 503 in this thread? What's happening here in the logs? Thanks, 111 SessionOpen c 64.12.95.228 5530 :80 111 ReqStart c 64.12.95.228 5530 442458545 111 RxRequestc GET 111 RxURLc /videos/e8afc40df0/happy-halloween-from-kevin- masterson-from-kevin-masterson 111 RxProtocol c HTTP/1.1 111 RxHeader c Connection: close 111 RxHeader c Accept: */* 111 RxHeader c User-Agent: NSPlayer/9.0.0.3250 111 RxHeader c Host: www.funnyordie.com 111 VCL_call c recv lookup 111 VCL_call c hash hash 111 VCL_call c miss fetch 111 Backend c 112 cachecluster cachemaster 111 ObjProtocol c HTTP/1.1 111 ObjStatusc 503 111 ObjResponse c Service Unavailable 111 ObjHeaderc Server: Varnish 111 ObjHeaderc Retry-After: 0 111 ObjHeaderc Content-Type: text/html; charset=utf-8 111 ObjHeaderc Date: Mon, 30 Mar 2009 18:51:50 GMT 111 ObjHeaderc X-Varnish: 1765034392 111 ObjHeaderc Via: 1.1 varnish 111 ObjHeaderc Served-by: web9/(null) 111 TTL c 442458545 RFC 120 1238439110 0 0 0 0 111 VCL_call c fetch restart 111 VCL_call c recv lookup 111 VCL_call c hash hash 111 VCL_call c miss fetch 111 Backend c 112 cachecluster cachemaster 111 ObjProtocol c HTTP/1.1 111 ObjStatusc 200 111 ObjResponse c OK 111 ObjHeaderc Content-Type: text/html; charset=utf-8 111 ObjHeaderc X-Runtime: 1941ms 111 ObjHeaderc ETag: 543372ea478715e48c5a3123b0a05d76 111 ObjHeaderc Cache-Control: no-cache, public, max-age=300 111 ObjHeaderc Server: LiteSpeed 111 ObjHeaderc Date: Mon, 30 Mar 2009 18:51:52 GMT 111 ObjHeaderc X-Varnish: 1765034393 111 ObjHeaderc Via: 1.1 varnish 111 ObjHeaderc Served-by: web9/app11 111 VCL_call c discard discard 111 VCL_call c discard discard 111 VCL_call c discard discard 111 VCL_call c discard discard 111 VCL_call c discard discard 111 TTL c 442458545 RFC 300 1238439112 0 0 300 0 111 VCL_call c fetch deliver 111 Length c 136482 111 VCL_call c deliver deliver 111 TxProtocol c HTTP/1.1 111 TxStatus c 200 111 TxResponse c OK 111 TxHeader c Content-Type: text/html; charset=utf-8 111 TxHeader c X-Runtime: 1941ms 111 TxHeader c ETag: 543372ea478715e48c5a3123b0a05d76 111 TxHeader c Cache-Control: no-cache, public, max-age=300 111 TxHeader c Server: LiteSpeed 111 TxHeader c X-Varnish: 1765034393 111 TxHeader c Via: 1.1 varnish 111 TxHeader c Content-Length: 136482 111 TxHeader c Date: Mon, 30 Mar 2009 18:51:52 GMT 111 TxHeader c X-Varnish: 442458545 111 TxHeader c Age: 0 111 TxHeader c Via: 1.1 varnish 111 TxHeader c Served-by: web10/web9/app11 111 TxHeader c Connection: close 111 ReqEnd c 442458545 1238439110.499521971 1238439112.896286964 0.36001 2.142404079 0.254360914 --Jeff j...@funnyordie.com ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Caching details of objects in varnish...
Actually yes query stripping is what I want to do on the inbound
request not on the fetch. My mistake for not being clear.
On Feb 10, 2009, at 1:39 AM, Paras Fadte wrote:
query stripping ?
On Thu, Feb 5, 2009 at 10:05 PM, Jeff Anderson j...@funnyordie.com
wrote:
Is there a way in VCL to cache just the base html of a page without
its parameters? For example:
/advertproviderformat.html?provider=2342342foo=3434bar=34213142
Is the entire url cached or just the advertprovider.html portion?
It seems
the whole url is cached because I observe so many misses for these
calls.
The parameters are nearly always random so caching the entire url is
very inefficient. I've tried in vcl_fetch:
if obj.url ~ 'advertproviderformat.html {
set obj.ttl = 24h;
}
But I think it is just caching the entire url which extremely
inefficient.
How can I just instruct varnish to serve the
advertproviderformat.html when
it receives one of the full url requests? How can I handle this in
VCL?
Thanks,
--Jeff
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
--Jeff
j...@funnyordie.com
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Caching details of objects in varnish...
Can you please give me a brief code example regarding how to do this in vcl? Thanks, On Feb 5, 2009, at 3:11 PM, Dag-Erling Smørgrav wrote: Jeff Anderson j...@funnyordie.com writes: Is there a way in VCL to cache just the base html of a page without its parameters? For example: /advertproviderformat.html?provider=2342342foo=3434bar=34213142 Yes, hook into vcl_hash and modify the hash string as appropriate. DES -- Dag-Erling Smørgrav - d...@des.no --Jeff j...@funnyordie.com ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Caching details of objects in varnish...
Thanks. On Feb 6, 2009, at 1:22 PM, Dag-Erling Smørgrav wrote: Jeff Anderson j...@funnyordie.com writes: Dag-Erling Smørgrav d...@des.no writes: Jeff Anderson j...@funnyordie.com writes: Is there a way in VCL to cache just the base html of a page without its parameters? For example: /advertproviderformat.html?provider=2342342foo=3434bar=34213142 Yes, hook into vcl_hash and modify the hash string as appropriate. Can you please give me a brief code example regarding how to do this in vcl? Read vcl(7); basically, you want to replace the stock vcl_hash (a copy of which is included in the man page) with a version that uses regsub() to extract the relevant portion of the URL. DES -- Dag-Erling Smørgrav - d...@des.no --Jeff j...@funnyordie.com ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Caching details of objects in varnish...
Is there a way in VCL to cache just the base html of a page without
its parameters? For example:
/advertproviderformat.html?provider=2342342foo=3434bar=34213142
Is the entire url cached or just the advertprovider.html portion? It seems
the whole url is cached because I observe so many misses for these calls.
The parameters are nearly always random so caching the entire url is
very inefficient. I've tried in vcl_fetch:
if obj.url ~ 'advertproviderformat.html {
set obj.ttl = 24h;
}
But I think it is just caching the entire url which extremely inefficient.
How can I just instruct varnish to serve the advertproviderformat.html when
it receives one of the full url requests? How can I handle this in VCL?
Thanks,
--Jeff
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Re: Varnish Serves only uncompressed objects if they are requested first
Our app servers are sending the Vary on the Accept-Encoding when
compression is requested. If compression is not requested they do not
perform the Vary. Does that mean we should find a way to send a Vary:
Accept-Encoding: null,gzip,deflate or something? Is there a 'no
compression' accept-encoding header?
On Dec 2, 2008, at 10:36 PM, Per Buer wrote:
Jeff Anderson skrev:
It looks like if the first requested page is for an uncompressed page
varnish will only deliver the uncompressed page from cache even if a
compressed page is requested.
As long as you don't Vary: on the Accept-Encoding I guess that is
expected. Varnish doesn't not understand the Accept-Encoding header.
(..)
What could be causing this? The only way to fix appears to be to add
the lines below:
sub vcl_hash {
if (req.http.Accept-Encoding ~ gzip || req.http.Accept-Encoding ~
deflate) {
set req.hash += req.http.Accept-Encoding;
}
Either use the fix you suggested or add a Vary: Accept-Encoding on the
backend.
--
Per Buer - Leder Infrastruktur og Drift - Redpill Linpro
Telefon: 21 54 41 21 - Mobil: 958 39 117
http://linpro.no/ | http://redpill.se/
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Vary Header Strange Behavior
The latest version of Varnish seems to behave unexpectedly with regard
to the Vary on Accept-Encoding headers.
Our production servers report back different headers than our
development and test varnish servers even though they are identical
VCL. In production it serves up uncompressed versions no matter what
version is requested. The vary header doesn't even show up like it
does on the development severs.
$ curl http://prodweb9 --head --compressed
HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
X-Runtime: 0.58210
ETag: 359535e914cddcc7ce0c879f1b3d4668
Cache-Control: no-cache, public, max-age=300
Server: LiteSpeed
Content-Length: 60711
Date: Mon, 01 Dec 2008 23:10:47 GMT
X-Varnish: 1013927736 1013926899
Age: 59
Via: 1.1 varnish
Served-by: prodweb9/prodapp7
Connection: close
$ curl http://prodweb9 --head
HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
X-Runtime: 0.58210
ETag: 359535e914cddcc7ce0c879f1b3d4668
Cache-Control: no-cache, public, max-age=300
Server: LiteSpeed
Content-Length: 60711
Date: Mon, 01 Dec 2008 23:10:56 GMT
X-Varnish: 1013927873 1013926899
Age: 69
Via: 1.1 varnish
Served-by: prodweb9/prodapp7
Connection: close
However on our development servers:
$ curl http://devweb1 --head
HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
ETag: 4ebc8c72bde3a68dc965f52d392ac630
X-Runtime: 0.12171
Cache-Control: no-cache, public, max-age=300
Vary: Accept-Encoding
X-Varnish: 191681248
Age: 0
Via: 1.1 varnish
Served-by: devweb1/devapp1
Date: Mon, 01 Dec 2008 23:04:15 GMT
Server: LiteSpeed
Connection: Keep-Alive
Keep-Alive: timeout=5, max=100
WWW-Authenticate: Basic realm=Protected Area
$ curl http://devweb1 --head --compressed
HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
ETag: 4ebc8c72bde3a68dc965f52d392ac630
X-Runtime: 0.12171
Cache-Control: no-cache, public, max-age=300
Content-Encoding: gzip
Vary: Accept-Encoding
X-Varnish: 191681255 191681248
Age: 30
Via: 1.1 varnish
Served-by: devweb1/devapp1
Date: Mon, 01 Dec 2008 23:04:45 GMT
Server: LiteSpeed
Connection: Keep-Alive
Keep-Alive: timeout=5, max=100
WWW-Authenticate: Basic realm=Protected Area
The lines below were added to the VCL to resolve the problem. And
while they did resolve the issue it seems to have made varnish double
the look up of incoming requests according to the varnishncsa log.
sub vcl_hash {
if (req.http.Accept-Encoding ~ gzip || req.http.Accept-Encoding ~
deflate) {
set req.hash += req.http.Accept-Encoding;
}
Is this normal or is something overriding the proper functioning of
varnish?
Thanks,
--Jeff
___
varnish-misc mailing list
varnish-misc@projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc
Using req/obj.grace to serve stale objects when backend fails
I've experimented with req/obj.grace set to a few hours so a site can be served even if the backend fails. For example, with the backend down and when req/obj.grace is set to several hours I can open a new browser and get a 503 if I try to open a known cached (and graced) page. However if I refresh the same browser several times very rapidly I finally receive the graced page. It seems to be working as expected from what I read in the documentation regarding the graced object being served while the same object is being fetched by another thread. The rapid refreshing is generating a second thread request which then satisifies the requirements to have the graced object served. Is there a way to configure varnish to serve the cached graced object if the backend fails without the browser ever seeing the 503? Can this also be tied into the backend probing/polling? Thanks, --J ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Version of Varnish reporting in the response header
Using firebug i get: Via 1.1 varnish In the response headers. Should that be 2.0.1 instead or is it referring to something other than the version of Varnish? Thanks, --Jeff ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
Using req/obj.grace to serve stale objects when backend fails
Sorry for the second post. I've experimented with req/obj.grace set to a few hours so a site can be served even if the backend fails. For example, with the backend down and when req/obj.grace is set to several hours I can open a new browser and get a 503 if I try to open a known cached (and graced) page. However if I refresh the same browser several times very rapidly I finally receive the graced page. It seems to be working as expected from what I read in the documentation regarding the graced object being served while the same object is being fetched by another thread. The rapid refreshing is generating a second thread request which then satisifies the requirements to have the graced object served. Is there a way to configure varnish to serve the cached graced object if the backend fails without the browser ever seeing the 503? Can this also be tied into the backend probing/polling to serve the graced page if all the backends fail? Thanks, --J ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
