Compressed and uncompressed cached object handling
Hi guys, I'm having a problem with a varnish implementation that we are testing to replace an ugly appliance. We were almost ready to place our server in a more real environment (some of our production sites), but I found out that there is something not working properly with the compression handling in my varnishd (varnish-2.0.5 - Debian) Varnish its returning the first object cached no matter if i ask for a clear object (no Accept-encoding specified) or a gzip/deflate object. If the object cached is a gzip object it will return that no matter if I later ask for a clear one later. According to what I have seen in the documentation varnish should keep both object versions (compressed and no-compressed) in the cache and deliver the one that its asked by the client. Step 1 I ask for a non-compressed object (no Accept-encoding specified). This works great GET -H TE: -sed http://foo.bar/test/prueba.php; 200 OK Cache-Control: max-age=20 Connection: close Date: Mon, 16 Nov 2009 16:56:06 GMT Via: 1.1 varnish Age: 0 Server: Apache Content-Length: 11013 Content-Type: text/html; charset=iso-8859-15 Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT Client-Date: Mon, 16 Nov 2009 16:56:06 GMT Client-Peer: 10.10.10.10:80 Client-Response-Num: 1 X-Varnish: 1655545411 The request goes like this in the log: 12 SessionOpen c 10.20.20.20 57909 :80 12 ReqStart c 10.20.20.20 57909 1655545411 12 RxRequest c GET 12 RxURL c /test/prueba.php 12 RxProtocol c HTTP/1.1 12 RxHeader c Connection: TE, close 12 RxHeader c Host: foo.bar 12 RxHeader c TE: 12 RxHeader c User-Agent: lwp-request/5.827 libwww-perl/5.831 12 VCL_call c recv 12 VCL_return c lookup 12 VCL_call c hash 12 VCL_return c hash 12 VCL_call c miss 12 VCL_return c fetch 14 BackendOpen b default 10.10.10.10 33484 10.30.30.30 80 12 Backend c 14 default default 14 TxRequest b GET 14 TxURL b /test/prueba.php 14 TxProtocol b HTTP/1.1 14 TxHeader b Host: foo.bar 14 TxHeader b User-Agent: lwp-request/5.827 libwww-perl/5.831 14 TxHeader b X-Varnish: 1655545411 14 TxHeader b X-Forwarded-For: 10.20.20.20 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1258390564 1.0 14 RxProtocol b HTTP/1.1 14 RxStatus b 200 14 RxResponse b OK 14 RxHeader b Date: Mon, 16 Nov 2009 16:56:01 GMT 14 RxHeader b Server: Apache 14 RxHeader b Cache-control: max-age=20 14 RxHeader b Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT 14 RxHeader b Connection: close 14 RxHeader b Transfer-Encoding: chunked 14 RxHeader b Content-Type: text/html; charset=iso-8859-15 12 ObjProtocol c HTTP/1.1 12 ObjStatus c 200 12 ObjResponse c OK 12 ObjHeader c Date: Mon, 16 Nov 2009 16:56:01 GMT 12 ObjHeader c Server: Apache 12 ObjHeader c Cache-control: max-age=20 12 ObjHeader c Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT 12 ObjHeader c Content-Type: text/html; charset=iso-8859-15 14 BackendClose b default 12 TTL c 1655545411 RFC 20 1258390566 0 0 20 0 12 VCL_call c fetch 12 VCL_return c deliver 12 Length c 11013 12 VCL_call c deliver 12 VCL_return c deliver 12 TxProtocol c HTTP/1.1 12 TxStatus c 200 12 TxResponse c OK 12 TxHeader c Server: Apache 12 TxHeader c Cache-control: max-age=20 12 TxHeader c Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT 12 TxHeader c Content-Type: text/html; charset=iso-8859-15 12 TxHeader c Content-Length: 11013 12 TxHeader c Date: Mon, 16 Nov 2009 16:56:06 GMT 12 TxHeader c X-Varnish: 1655545411 12 TxHeader c Age: 0 12 TxHeader c Via: 1.1 varnish 12 TxHeader c Connection: close 12 ReqEnd c 1655545411 1258390561.316438675 1258390566.327898026 0.000134945 5.010995150 0.000464201 12 SessionClose c Connection: close 12 StatSess c 10.20.20.20 57909 5 1 1 0 0 1 282 11013 Step 2 Then the next request will go with a (Accept-encoding: gzip), and returns me a clear object :( GET -H Accept-encoding: gzip -H TE: -sed http://foo.bar/test/prueba.php; 200 OK Cache-Control: max-age=20 Connection: close Date: Mon, 16 Nov 2009 16:56:09 GMT Via: 1.1 varnish Age: 3 Server: Apache Content-Length: 11013 Content-Type: text/html; charset=iso-8859-15 Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT Client-Date: Mon, 16 Nov 2009 16:56:08 GMT Client-Peer: 10.10.10.10:80 Client-Response-Num: 1 X-Varnish: 1655545412 1655545411 12 SessionOpen c 10.20.20.20 57910 :80 12 ReqStart c 10.20.20.20 57910 1655545412 12 RxRequestc GET 12 RxURLc /test/prueba.php 12 RxProtocol c HTTP/1.1 12 RxHeader c Connection: TE, close 12 RxHeader c Accept-Encoding: gzip 12 RxHeader c Host: foo.bar
Re: Compressed and uncompressed cached object handling
Are you returning a Vary: Accept-Encoding in your origin server's response headers? --Michael On Nov 17, 2009, at 4:01 PM, Daniel Rodriguez wrote: Hi guys, I'm having a problem with a varnish implementation that we are testing to replace an ugly appliance. We were almost ready to place our server in a more real environment (some of our production sites), but I found out that there is something not working properly with the compression handling in my varnishd (varnish-2.0.5 - Debian) Varnish its returning the first object cached no matter if i ask for a clear object (no Accept-encoding specified) or a gzip/deflate object. If the object cached is a gzip object it will return that no matter if I later ask for a clear one later. According to what I have seen in the documentation varnish should keep both object versions (compressed and no-compressed) in the cache and deliver the one that its asked by the client. Step 1 I ask for a non-compressed object (no Accept-encoding specified). This works great GET -H TE: -sed http://foo.bar/test/prueba.php; 200 OK Cache-Control: max-age=20 Connection: close Date: Mon, 16 Nov 2009 16:56:06 GMT Via: 1.1 varnish Age: 0 Server: Apache Content-Length: 11013 Content-Type: text/html; charset=iso-8859-15 Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT Client-Date: Mon, 16 Nov 2009 16:56:06 GMT Client-Peer: 10.10.10.10:80 Client-Response-Num: 1 X-Varnish: 1655545411 The request goes like this in the log: 12 SessionOpen c 10.20.20.20 57909 :80 12 ReqStart c 10.20.20.20 57909 1655545411 12 RxRequestc GET 12 RxURLc /test/prueba.php 12 RxProtocol c HTTP/1.1 12 RxHeader c Connection: TE, close 12 RxHeader c Host: foo.bar 12 RxHeader c TE: 12 RxHeader c User-Agent: lwp-request/5.827 libwww-perl/5.831 12 VCL_call c recv 12 VCL_return c lookup 12 VCL_call c hash 12 VCL_return c hash 12 VCL_call c miss 12 VCL_return c fetch 14 BackendOpen b default 10.10.10.10 33484 10.30.30.30 80 12 Backend c 14 default default 14 TxRequestb GET 14 TxURLb /test/prueba.php 14 TxProtocol b HTTP/1.1 14 TxHeader b Host: foo.bar 14 TxHeader b User-Agent: lwp-request/5.827 libwww-perl/5.831 14 TxHeader b X-Varnish: 1655545411 14 TxHeader b X-Forwarded-For: 10.20.20.20 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1258390564 1.0 14 RxProtocol b HTTP/1.1 14 RxStatus b 200 14 RxResponse b OK 14 RxHeader b Date: Mon, 16 Nov 2009 16:56:01 GMT 14 RxHeader b Server: Apache 14 RxHeader b Cache-control: max-age=20 14 RxHeader b Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT 14 RxHeader b Connection: close 14 RxHeader b Transfer-Encoding: chunked 14 RxHeader b Content-Type: text/html; charset=iso-8859-15 12 ObjProtocol c HTTP/1.1 12 ObjStatusc 200 12 ObjResponse c OK 12 ObjHeaderc Date: Mon, 16 Nov 2009 16:56:01 GMT 12 ObjHeaderc Server: Apache 12 ObjHeaderc Cache-control: max-age=20 12 ObjHeaderc Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT 12 ObjHeaderc Content-Type: text/html; charset=iso-8859-15 14 BackendClose b default 12 TTL c 1655545411 RFC 20 1258390566 0 0 20 0 12 VCL_call c fetch 12 VCL_return c deliver 12 Length c 11013 12 VCL_call c deliver 12 VCL_return c deliver 12 TxProtocol c HTTP/1.1 12 TxStatus c 200 12 TxResponse c OK 12 TxHeader c Server: Apache 12 TxHeader c Cache-control: max-age=20 12 TxHeader c Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT 12 TxHeader c Content-Type: text/html; charset=iso-8859-15 12 TxHeader c Content-Length: 11013 12 TxHeader c Date: Mon, 16 Nov 2009 16:56:06 GMT 12 TxHeader c X-Varnish: 1655545411 12 TxHeader c Age: 0 12 TxHeader c Via: 1.1 varnish 12 TxHeader c Connection: close 12 ReqEnd c 1655545411 1258390561.316438675 1258390566.327898026 0.000134945 5.010995150 0.000464201 12 SessionClose c Connection: close 12 StatSess c 10.20.20.20 57909 5 1 1 0 0 1 282 11013 Step 2 Then the next request will go with a (Accept-encoding: gzip), and returns me a clear object :( GET -H Accept-encoding: gzip -H TE: -sed http://foo.bar/test/prueba.php 200 OK Cache-Control: max-age=20 Connection: close Date: Mon, 16 Nov 2009 16:56:09 GMT Via: 1.1 varnish Age: 3 Server: Apache Content-Length: 11013 Content-Type: text/html; charset=iso-8859-15 Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT Client-Date: Mon, 16 Nov 2009 16:56:08 GMT Client-Peer: 10.10.10.10:80 Client-Response-Num: 1 X-Varnish: 1655545412 1655545411 12 SessionOpen c 10.20.20.20 57910 :80
Re: Compressed and uncompressed cached object handling
Hi Michael, That was the problem, the server was not returning the Vary: Accept-Encoding, I didn't notice that detail in the headers sent by the server. Its working perfect right now. Thank you very much. Regards, On Tue, Nov 17, 2009 at 11:42 AM, Michael S. Fischer mich...@dynamine.net wrote: Are you returning a Vary: Accept-Encoding in your origin server's response headers? --Michael On Nov 17, 2009, at 4:01 PM, Daniel Rodriguez wrote: Hi guys, I'm having a problem with a varnish implementation that we are testing to replace an ugly appliance. We were almost ready to place our server in a more real environment (some of our production sites), but I found out that there is something not working properly with the compression handling in my varnishd (varnish-2.0.5 - Debian) Varnish its returning the first object cached no matter if i ask for a clear object (no Accept-encoding specified) or a gzip/deflate object. If the object cached is a gzip object it will return that no matter if I later ask for a clear one later. According to what I have seen in the documentation varnish should keep both object versions (compressed and no-compressed) in the cache and deliver the one that its asked by the client. Step 1 I ask for a non-compressed object (no Accept-encoding specified). This works great GET -H TE: -sed http://foo.bar/test/prueba.php; 200 OK Cache-Control: max-age=20 Connection: close Date: Mon, 16 Nov 2009 16:56:06 GMT Via: 1.1 varnish Age: 0 Server: Apache Content-Length: 11013 Content-Type: text/html; charset=iso-8859-15 Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT Client-Date: Mon, 16 Nov 2009 16:56:06 GMT Client-Peer: 10.10.10.10:80 Client-Response-Num: 1 X-Varnish: 1655545411 The request goes like this in the log: 12 SessionOpen c 10.20.20.20 57909 :80 12 ReqStart c 10.20.20.20 57909 1655545411 12 RxRequest c GET 12 RxURL c /test/prueba.php 12 RxProtocol c HTTP/1.1 12 RxHeader c Connection: TE, close 12 RxHeader c Host: foo.bar 12 RxHeader c TE: 12 RxHeader c User-Agent: lwp-request/5.827 libwww-perl/5.831 12 VCL_call c recv 12 VCL_return c lookup 12 VCL_call c hash 12 VCL_return c hash 12 VCL_call c miss 12 VCL_return c fetch 14 BackendOpen b default 10.10.10.10 33484 10.30.30.30 80 12 Backend c 14 default default 14 TxRequest b GET 14 TxURL b /test/prueba.php 14 TxProtocol b HTTP/1.1 14 TxHeader b Host: foo.bar 14 TxHeader b User-Agent: lwp-request/5.827 libwww-perl/5.831 14 TxHeader b X-Varnish: 1655545411 14 TxHeader b X-Forwarded-For: 10.20.20.20 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1258390564 1.0 14 RxProtocol b HTTP/1.1 14 RxStatus b 200 14 RxResponse b OK 14 RxHeader b Date: Mon, 16 Nov 2009 16:56:01 GMT 14 RxHeader b Server: Apache 14 RxHeader b Cache-control: max-age=20 14 RxHeader b Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT 14 RxHeader b Connection: close 14 RxHeader b Transfer-Encoding: chunked 14 RxHeader b Content-Type: text/html; charset=iso-8859-15 12 ObjProtocol c HTTP/1.1 12 ObjStatus c 200 12 ObjResponse c OK 12 ObjHeader c Date: Mon, 16 Nov 2009 16:56:01 GMT 12 ObjHeader c Server: Apache 12 ObjHeader c Cache-control: max-age=20 12 ObjHeader c Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT 12 ObjHeader c Content-Type: text/html; charset=iso-8859-15 14 BackendClose b default 12 TTL c 1655545411 RFC 20 1258390566 0 0 20 0 12 VCL_call c fetch 12 VCL_return c deliver 12 Length c 11013 12 VCL_call c deliver 12 VCL_return c deliver 12 TxProtocol c HTTP/1.1 12 TxStatus c 200 12 TxResponse c OK 12 TxHeader c Server: Apache 12 TxHeader c Cache-control: max-age=20 12 TxHeader c Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT 12 TxHeader c Content-Type: text/html; charset=iso-8859-15 12 TxHeader c Content-Length: 11013 12 TxHeader c Date: Mon, 16 Nov 2009 16:56:06 GMT 12 TxHeader c X-Varnish: 1655545411 12 TxHeader c Age: 0 12 TxHeader c Via: 1.1 varnish 12 TxHeader c Connection: close 12 ReqEnd c 1655545411 1258390561.316438675 1258390566.327898026 0.000134945 5.010995150 0.000464201 12 SessionClose c Connection: close 12 StatSess c 10.20.20.20 57909 5 1 1 0 0 1 282 11013 Step 2 Then the next request will go with a (Accept-encoding: gzip), and returns me a clear object :( GET -H Accept-encoding: gzip -H TE: -sed http://foo.bar/test/prueba.php; 200 OK Cache-Control: max-age=20 Connection: close Date: Mon, 16 Nov 2009 16:56:09 GMT Via: 1.1 varnish Age: 3 Server: Apache Content-Length: 11013 Content-Type: text/html;
