Compressed and uncompressed cached object handling
Hi guys, I'm having a problem with a varnish implementation that we are testing to replace an ugly appliance. We were almost ready to place our server in a more real environment (some of our production sites), but I found out that there is something not working properly with the compression handling in my varnishd (varnish-2.0.5 - Debian) Varnish its returning the first object cached no matter if i ask for a clear object (no Accept-encoding specified) or a gzip/deflate object. If the object cached is a gzip object it will return that no matter if I later ask for a clear one later. According to what I have seen in the documentation varnish should keep both object versions (compressed and no-compressed) in the cache and deliver the one that its asked by the client. Step 1 I ask for a non-compressed object (no Accept-encoding specified). This works great GET -H TE: -sed http://foo.bar/test/prueba.php; 200 OK Cache-Control: max-age=20 Connection: close Date: Mon, 16 Nov 2009 16:56:06 GMT Via: 1.1 varnish Age: 0 Server: Apache Content-Length: 11013 Content-Type: text/html; charset=iso-8859-15 Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT Client-Date: Mon, 16 Nov 2009 16:56:06 GMT Client-Peer: 10.10.10.10:80 Client-Response-Num: 1 X-Varnish: 1655545411 The request goes like this in the log: 12 SessionOpen c 10.20.20.20 57909 :80 12 ReqStart c 10.20.20.20 57909 1655545411 12 RxRequest c GET 12 RxURL c /test/prueba.php 12 RxProtocol c HTTP/1.1 12 RxHeader c Connection: TE, close 12 RxHeader c Host: foo.bar 12 RxHeader c TE: 12 RxHeader c User-Agent: lwp-request/5.827 libwww-perl/5.831 12 VCL_call c recv 12 VCL_return c lookup 12 VCL_call c hash 12 VCL_return c hash 12 VCL_call c miss 12 VCL_return c fetch 14 BackendOpen b default 10.10.10.10 33484 10.30.30.30 80 12 Backend c 14 default default 14 TxRequest b GET 14 TxURL b /test/prueba.php 14 TxProtocol b HTTP/1.1 14 TxHeader b Host: foo.bar 14 TxHeader b User-Agent: lwp-request/5.827 libwww-perl/5.831 14 TxHeader b X-Varnish: 1655545411 14 TxHeader b X-Forwarded-For: 10.20.20.20 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1258390564 1.0 14 RxProtocol b HTTP/1.1 14 RxStatus b 200 14 RxResponse b OK 14 RxHeader b Date: Mon, 16 Nov 2009 16:56:01 GMT 14 RxHeader b Server: Apache 14 RxHeader b Cache-control: max-age=20 14 RxHeader b Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT 14 RxHeader b Connection: close 14 RxHeader b Transfer-Encoding: chunked 14 RxHeader b Content-Type: text/html; charset=iso-8859-15 12 ObjProtocol c HTTP/1.1 12 ObjStatus c 200 12 ObjResponse c OK 12 ObjHeader c Date: Mon, 16 Nov 2009 16:56:01 GMT 12 ObjHeader c Server: Apache 12 ObjHeader c Cache-control: max-age=20 12 ObjHeader c Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT 12 ObjHeader c Content-Type: text/html; charset=iso-8859-15 14 BackendClose b default 12 TTL c 1655545411 RFC 20 1258390566 0 0 20 0 12 VCL_call c fetch 12 VCL_return c deliver 12 Length c 11013 12 VCL_call c deliver 12 VCL_return c deliver 12 TxProtocol c HTTP/1.1 12 TxStatus c 200 12 TxResponse c OK 12 TxHeader c Server: Apache 12 TxHeader c Cache-control: max-age=20 12 TxHeader c Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT 12 TxHeader c Content-Type: text/html; charset=iso-8859-15 12 TxHeader c Content-Length: 11013 12 TxHeader c Date: Mon, 16 Nov 2009 16:56:06 GMT 12 TxHeader c X-Varnish: 1655545411 12 TxHeader c Age: 0 12 TxHeader c Via: 1.1 varnish 12 TxHeader c Connection: close 12 ReqEnd c 1655545411 1258390561.316438675 1258390566.327898026 0.000134945 5.010995150 0.000464201 12 SessionClose c Connection: close 12 StatSess c 10.20.20.20 57909 5 1 1 0 0 1 282 11013 Step 2 Then the next request will go with a (Accept-encoding: gzip), and returns me a clear object :( GET -H Accept-encoding: gzip -H TE: -sed http://foo.bar/test/prueba.php; 200 OK Cache-Control: max-age=20 Connection: close Date: Mon, 16 Nov 2009 16:56:09 GMT Via: 1.1 varnish Age: 3 Server: Apache Content-Length: 11013 Content-Type: text/html; charset=iso-8859-15 Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT Client-Date: Mon, 16 Nov 2009 16:56:08 GMT Client-Peer: 10.10.10.10:80 Client-Response-Num: 1 X-Varnish: 1655545412 1655545411 12 SessionOpen c 10.20.20.20 57910 :80 12 ReqStart c 10.20.20.20 57910 1655545412 12 RxRequestc GET 12 RxURLc /test/prueba.php 12 RxProtocol c HTTP/1.1 12 RxHeader c Connection: TE, close 12 RxHeader c Accept-Encoding: gzip 12 RxHeader c Host: foo.bar
Re: Compressed and uncompressed cached object handling
Are you returning a Vary: Accept-Encoding in your origin server's response headers? --Michael On Nov 17, 2009, at 4:01 PM, Daniel Rodriguez wrote: Hi guys, I'm having a problem with a varnish implementation that we are testing to replace an ugly appliance. We were almost ready to place our server in a more real environment (some of our production sites), but I found out that there is something not working properly with the compression handling in my varnishd (varnish-2.0.5 - Debian) Varnish its returning the first object cached no matter if i ask for a clear object (no Accept-encoding specified) or a gzip/deflate object. If the object cached is a gzip object it will return that no matter if I later ask for a clear one later. According to what I have seen in the documentation varnish should keep both object versions (compressed and no-compressed) in the cache and deliver the one that its asked by the client. Step 1 I ask for a non-compressed object (no Accept-encoding specified). This works great GET -H TE: -sed http://foo.bar/test/prueba.php; 200 OK Cache-Control: max-age=20 Connection: close Date: Mon, 16 Nov 2009 16:56:06 GMT Via: 1.1 varnish Age: 0 Server: Apache Content-Length: 11013 Content-Type: text/html; charset=iso-8859-15 Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT Client-Date: Mon, 16 Nov 2009 16:56:06 GMT Client-Peer: 10.10.10.10:80 Client-Response-Num: 1 X-Varnish: 1655545411 The request goes like this in the log: 12 SessionOpen c 10.20.20.20 57909 :80 12 ReqStart c 10.20.20.20 57909 1655545411 12 RxRequestc GET 12 RxURLc /test/prueba.php 12 RxProtocol c HTTP/1.1 12 RxHeader c Connection: TE, close 12 RxHeader c Host: foo.bar 12 RxHeader c TE: 12 RxHeader c User-Agent: lwp-request/5.827 libwww-perl/5.831 12 VCL_call c recv 12 VCL_return c lookup 12 VCL_call c hash 12 VCL_return c hash 12 VCL_call c miss 12 VCL_return c fetch 14 BackendOpen b default 10.10.10.10 33484 10.30.30.30 80 12 Backend c 14 default default 14 TxRequestb GET 14 TxURLb /test/prueba.php 14 TxProtocol b HTTP/1.1 14 TxHeader b Host: foo.bar 14 TxHeader b User-Agent: lwp-request/5.827 libwww-perl/5.831 14 TxHeader b X-Varnish: 1655545411 14 TxHeader b X-Forwarded-For: 10.20.20.20 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1258390564 1.0 14 RxProtocol b HTTP/1.1 14 RxStatus b 200 14 RxResponse b OK 14 RxHeader b Date: Mon, 16 Nov 2009 16:56:01 GMT 14 RxHeader b Server: Apache 14 RxHeader b Cache-control: max-age=20 14 RxHeader b Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT 14 RxHeader b Connection: close 14 RxHeader b Transfer-Encoding: chunked 14 RxHeader b Content-Type: text/html; charset=iso-8859-15 12 ObjProtocol c HTTP/1.1 12 ObjStatusc 200 12 ObjResponse c OK 12 ObjHeaderc Date: Mon, 16 Nov 2009 16:56:01 GMT 12 ObjHeaderc Server: Apache 12 ObjHeaderc Cache-control: max-age=20 12 ObjHeaderc Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT 12 ObjHeaderc Content-Type: text/html; charset=iso-8859-15 14 BackendClose b default 12 TTL c 1655545411 RFC 20 1258390566 0 0 20 0 12 VCL_call c fetch 12 VCL_return c deliver 12 Length c 11013 12 VCL_call c deliver 12 VCL_return c deliver 12 TxProtocol c HTTP/1.1 12 TxStatus c 200 12 TxResponse c OK 12 TxHeader c Server: Apache 12 TxHeader c Cache-control: max-age=20 12 TxHeader c Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT 12 TxHeader c Content-Type: text/html; charset=iso-8859-15 12 TxHeader c Content-Length: 11013 12 TxHeader c Date: Mon, 16 Nov 2009 16:56:06 GMT 12 TxHeader c X-Varnish: 1655545411 12 TxHeader c Age: 0 12 TxHeader c Via: 1.1 varnish 12 TxHeader c Connection: close 12 ReqEnd c 1655545411 1258390561.316438675 1258390566.327898026 0.000134945 5.010995150 0.000464201 12 SessionClose c Connection: close 12 StatSess c 10.20.20.20 57909 5 1 1 0 0 1 282 11013 Step 2 Then the next request will go with a (Accept-encoding: gzip), and returns me a clear object :( GET -H Accept-encoding: gzip -H TE: -sed http://foo.bar/test/prueba.php 200 OK Cache-Control: max-age=20 Connection: close Date: Mon, 16 Nov 2009 16:56:09 GMT Via: 1.1 varnish Age: 3 Server: Apache Content-Length: 11013 Content-Type: text/html; charset=iso-8859-15 Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT Client-Date: Mon, 16 Nov 2009 16:56:08 GMT Client-Peer: 10.10.10.10:80 Client-Response-Num: 1 X-Varnish: 1655545412 1655545411 12 SessionOpen c 10.20.20.20 57910 :80
Re: Compressed and uncompressed cached object handling
Hi Michael, That was the problem, the server was not returning the Vary: Accept-Encoding, I didn't notice that detail in the headers sent by the server. Its working perfect right now. Thank you very much. Regards, On Tue, Nov 17, 2009 at 11:42 AM, Michael S. Fischer mich...@dynamine.net wrote: Are you returning a Vary: Accept-Encoding in your origin server's response headers? --Michael On Nov 17, 2009, at 4:01 PM, Daniel Rodriguez wrote: Hi guys, I'm having a problem with a varnish implementation that we are testing to replace an ugly appliance. We were almost ready to place our server in a more real environment (some of our production sites), but I found out that there is something not working properly with the compression handling in my varnishd (varnish-2.0.5 - Debian) Varnish its returning the first object cached no matter if i ask for a clear object (no Accept-encoding specified) or a gzip/deflate object. If the object cached is a gzip object it will return that no matter if I later ask for a clear one later. According to what I have seen in the documentation varnish should keep both object versions (compressed and no-compressed) in the cache and deliver the one that its asked by the client. Step 1 I ask for a non-compressed object (no Accept-encoding specified). This works great GET -H TE: -sed http://foo.bar/test/prueba.php; 200 OK Cache-Control: max-age=20 Connection: close Date: Mon, 16 Nov 2009 16:56:06 GMT Via: 1.1 varnish Age: 0 Server: Apache Content-Length: 11013 Content-Type: text/html; charset=iso-8859-15 Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT Client-Date: Mon, 16 Nov 2009 16:56:06 GMT Client-Peer: 10.10.10.10:80 Client-Response-Num: 1 X-Varnish: 1655545411 The request goes like this in the log: 12 SessionOpen c 10.20.20.20 57909 :80 12 ReqStart c 10.20.20.20 57909 1655545411 12 RxRequest c GET 12 RxURL c /test/prueba.php 12 RxProtocol c HTTP/1.1 12 RxHeader c Connection: TE, close 12 RxHeader c Host: foo.bar 12 RxHeader c TE: 12 RxHeader c User-Agent: lwp-request/5.827 libwww-perl/5.831 12 VCL_call c recv 12 VCL_return c lookup 12 VCL_call c hash 12 VCL_return c hash 12 VCL_call c miss 12 VCL_return c fetch 14 BackendOpen b default 10.10.10.10 33484 10.30.30.30 80 12 Backend c 14 default default 14 TxRequest b GET 14 TxURL b /test/prueba.php 14 TxProtocol b HTTP/1.1 14 TxHeader b Host: foo.bar 14 TxHeader b User-Agent: lwp-request/5.827 libwww-perl/5.831 14 TxHeader b X-Varnish: 1655545411 14 TxHeader b X-Forwarded-For: 10.20.20.20 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1258390564 1.0 14 RxProtocol b HTTP/1.1 14 RxStatus b 200 14 RxResponse b OK 14 RxHeader b Date: Mon, 16 Nov 2009 16:56:01 GMT 14 RxHeader b Server: Apache 14 RxHeader b Cache-control: max-age=20 14 RxHeader b Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT 14 RxHeader b Connection: close 14 RxHeader b Transfer-Encoding: chunked 14 RxHeader b Content-Type: text/html; charset=iso-8859-15 12 ObjProtocol c HTTP/1.1 12 ObjStatus c 200 12 ObjResponse c OK 12 ObjHeader c Date: Mon, 16 Nov 2009 16:56:01 GMT 12 ObjHeader c Server: Apache 12 ObjHeader c Cache-control: max-age=20 12 ObjHeader c Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT 12 ObjHeader c Content-Type: text/html; charset=iso-8859-15 14 BackendClose b default 12 TTL c 1655545411 RFC 20 1258390566 0 0 20 0 12 VCL_call c fetch 12 VCL_return c deliver 12 Length c 11013 12 VCL_call c deliver 12 VCL_return c deliver 12 TxProtocol c HTTP/1.1 12 TxStatus c 200 12 TxResponse c OK 12 TxHeader c Server: Apache 12 TxHeader c Cache-control: max-age=20 12 TxHeader c Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT 12 TxHeader c Content-Type: text/html; charset=iso-8859-15 12 TxHeader c Content-Length: 11013 12 TxHeader c Date: Mon, 16 Nov 2009 16:56:06 GMT 12 TxHeader c X-Varnish: 1655545411 12 TxHeader c Age: 0 12 TxHeader c Via: 1.1 varnish 12 TxHeader c Connection: close 12 ReqEnd c 1655545411 1258390561.316438675 1258390566.327898026 0.000134945 5.010995150 0.000464201 12 SessionClose c Connection: close 12 StatSess c 10.20.20.20 57909 5 1 1 0 0 1 282 11013 Step 2 Then the next request will go with a (Accept-encoding: gzip), and returns me a clear object :( GET -H Accept-encoding: gzip -H TE: -sed http://foo.bar/test/prueba.php; 200 OK Cache-Control: max-age=20 Connection: close Date: Mon, 16 Nov 2009 16:56:09 GMT Via: 1.1 varnish Age: 3 Server: Apache Content-Length: 11013 Content-Type: text/html;
RE: Does vbulletin support varnish?
Out of the box vbulletin is php and runs best with fastcgi. You'll get the most value out of serving images on a separate domain that varnish answers. The cookies make it difficult. Best running example of a big vbulletin instance is http://hackint0sh.org/ which is nginx and fastcgi, no apache. Stefan From: varnish-misc-boun...@projects.linpro.no [mailto:varnish-misc-boun...@projects.linpro.no] On Behalf Of Debunk it Sent: November-16-09 8:06 PM To: varnish-misc@projects.linpro.no Subject: Does vbulletin support varnish? Hi all, Can someone confirm whether vbulletin supports varnish out of the box? Does vbulletin know how to send specific purge requests to varnish on updates? My concern is mainly about stale cache. I would prefer not to have to send purge requests manually. Any tips or help is greatly appreciated. D ___ varnish-misc mailing list varnish-misc@projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc
