Hi, I am having a problem with a varnish cache installation that I cannot find the answer to... I'm getting 503 (Service Unavailable) errors when trying to view an image served from a drupal 7 backend.
Here are details (ips, names are changed for privacy): varnish cache -> foo.gatech.edu (192.168.0.1), RHEL6, varnish-3.0.4-4.el6.art.x86_64 from the atomicorp repo drupal server -> bar.gatech.edu (192.168.0.2), RHEL6, apache 2.2, drupal 7, defined in vcl as backend 'bar', has 'foo.gatech.edu' as a ServerAlias web client (Chrome) -> 10.10.10.1 Users can access the site via http://foo.gatech.edu/ , which goes through the varnish cache, and works well, generally. The admins of the site can access the site via http://bar.gatech.edu/ , which does not go through the cache. When I use my web browser to try an fetch a JPEG image from http://foo.gatech.edu/home/images/50384 , I get a 503 error from varnish. This is odd for several reasons: - I can see a successful (HTTP 200 result) request for the image both in apache logs of bar.gatech.edu, and in the back-end (b) entries from the varnishlog on foo.gatech.edu - It happens for any default-sized being served from the drupal application on bar.gatech.edu, so it's not specific to one image. Images in the /home/images/ directory are served via the "Adaptive Image Styles" varnish module - I can always successfully fetch the image via http://bar.gatech.edu/home/images/50384 , bypassing the cache - It does *not* happen for smaller versions of the same image. So, http://foo.gatech.edu/home/images/50384/thumbnail_scaled returns a smaller jpeg version of the image, through the cache, successfully every time. Using HTTPFox (similar to HTTP Live Headers) on Firefox shows nothing significantly different between fetching the original image and the thumbnail version of the image from bar.gatec.edu directly Things I've tried: - restarting varnish, rebooting the box, etc. - using both file and malloc storage options in /etc/sysconfig/varnish - upping the backend-timeouts to very large values - eliminate any vhosts/hostname issues by changing my /etc/hosts so that foo.gatech.edu resolves to 198.168.0.2, and then verifying http://foo.gatech.edu/home/images/50384 returns a good image to my browser Here is the varnishlog output from a failed request... The key line is "11 FetchError c straight insufficient bytes" : ----------------------------------- 13 BackendOpen b bar 192.168.0.1 44950 192.168.0.2 80 13 TxRequest b GET 13 TxURL b /home/images/50384 13 TxProtocol b HTTP/1.1 13 TxHeader b Host: foo.gatech.edu 13 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 13 TxHeader b Pragma: no-cache 13 TxHeader b User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36 13 TxHeader b DNT: 1 13 TxHeader b Accept-Language: en-US,en;q=0.8 13 TxHeader b X-Forwarded-For: 10.10.10.1 13 TxHeader b X-Varnish: 1892649961 13 TxHeader b Accept-Encoding: gzip 13 RxProtocol b HTTP/1.1 13 RxStatus b 200 13 RxResponse b OK 13 RxHeader b Date: Wed, 16 Oct 2013 20:17:23 GMT 13 RxHeader b Server: Apache/2.2.3 (Red Hat) 13 RxHeader b X-Powered-By: PHP/5.3.14 ZendServer/5.0 13 RxHeader b X-Drupal-Cache: MISS 13 RxHeader b Expires: Sun, 19 Nov 1978 05:00:00 GMT 13 RxHeader b Last-Modified: Wed, 16 Oct 2013 20:17:23 +0000 13 RxHeader b Cache-Control: public, max-age=21600 13 RxHeader b ETag: "1381954643-1" 13 RxHeader b Content-Length: 55293 13 RxHeader b Content-Language: en 13 RxHeader b Vary: Accept-Encoding 13 RxHeader b Content-Encoding: gzip 13 RxHeader b Connection: close 13 RxHeader b Content-Type: image/jpeg 13 Fetch_Body b 4(length) cls -1 mklen 1 13 BackendClose b bar 11 SessionOpen c 10.10.10.1 64106 :80 11 ReqStart c 10.10.10.1 64106 1892649961 11 RxRequest c GET 11 RxURL c /home/images/50384 11 RxProtocol c HTTP/1.1 11 RxHeader c Host: foo.gatech.edu 11 RxHeader c Connection: keep-alive 11 RxHeader c Cache-Control: no-cache 11 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 11 RxHeader c Pragma: no-cache 11 RxHeader c User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36 11 RxHeader c DNT: 1 11 RxHeader c Accept-Encoding: gzip,deflate,sdch 11 RxHeader c Accept-Language: en-US,en;q=0.8 11 VCL_call c recv 11 VCL_return c lookup 11 VCL_call c hash 11 Hash c /home/images/50384 11 Hash c foo.gatech.edu 11 VCL_return c hash 11 VCL_call c miss fetch 11 Backend c 13 bar bar 11 TTL c 1892649961 RFC 21600 -1 -1 1381954643 0 1381954643 280299600 21600 11 VCL_call c fetch deliver 11 ObjProtocol c HTTP/1.1 11 ObjResponse c OK 11 ObjHeader c Date: Wed, 16 Oct 2013 20:17:23 GMT 11 ObjHeader c Server: Apache/2.2.3 (Red Hat) 11 ObjHeader c X-Powered-By: PHP/5.3.14 ZendServer/5.0 11 ObjHeader c X-Drupal-Cache: MISS 11 ObjHeader c Expires: Sun, 19 Nov 1978 05:00:00 GMT 11 ObjHeader c Last-Modified: Wed, 16 Oct 2013 20:17:23 +0000 11 ObjHeader c Cache-Control: public, max-age=21600 11 ObjHeader c ETag: "1381954643-1" 11 ObjHeader c Content-Language: en 11 ObjHeader c Vary: Accept-Encoding 11 ObjHeader c Content-Encoding: gzip 11 ObjHeader c Content-Type: image/jpeg 11 ObjHeader c X-Varnish-IP: 192.168.0.1 11 FetchError c straight insufficient bytes 11 Gzip c u F - 29537 55293 80 80 236227 11 VCL_call c error deliver 11 VCL_call c deliver deliver 11 TxProtocol c HTTP/1.1 11 TxStatus c 503 11 TxResponse c Service Unavailable 11 TxHeader c Server: Varnish 11 TxHeader c Content-Type: text/html; charset=utf-8 11 TxHeader c Content-Length: 829 11 TxHeader c Accept-Ranges: bytes 11 TxHeader c Date: Wed, 16 Oct 2013 20:17:23 GMT 11 TxHeader c X-Varnish: 1892649961 11 TxHeader c Age: 0 11 TxHeader c Via: 1.1 varnish 11 TxHeader c Connection: close 11 TxHeader c X-Cache: MISS 11 Length c 829 11 ReqEnd c 1892649961 1381954643.203719378 1381954643.275374174 0.000027418 0.071626902 0.000027895 11 SessionClose c error 11 StatSess c 10.10.10.1 64106 0 1 1 0 0 0 256 829 ----------------------------------- As I mentioned, I've Goolged around for this error "straight insufficient bytes" and didn't find any solution. I can find the line that returns that error in the varnish cache source code, but without knowning the internals of the src, it's Greek to me. Anyone got any guesses/suggestions? Any help will be greatly appreciated. Thanks, Adam A -- :: Adam Arrowood :: [email protected] :: Office of Information Technology/A&I :: Georgia Institute of Technology, Atlanta, GA USA
_______________________________________________ varnish-misc mailing list [email protected] https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
