I really am looking at what's happening as well. I have been looking at both
varnishlog and varnishtop and I see a lot of thumbnail image requests being
sent to the backend when there is still plenty of room for them in the cache,
so even though there are a lot of thumbnail images, I shouldn't see so many
backend requests for them. As I previously mentioned, I give Varnish 8 GB and
it used to stay full (based on RSS usage and looking at nukes vs. expires) but
now it hovers around only about 2 GB used. A related statistics is that there
used to be 600-700k objects in Varnish (based on our graphs of MAIN.n_object
via Collectd's varnish-default-struct.objects-object metric) but now there are
only roughly 40-70k objects in Varnish at any given time. So it's definitely
caching a lot fewer things than it was before the upgrade, and most of the
requested URLs for requests that have cookies are for a lot of images and
thumbnails. Images shouldn't be cached due to size and overall volume but
thumbnails should, which is why I strip cookies from the thumbnails. These
varnishtop commands break out /images and /images/thumb client requests,
showing IMHO too many regular images being cached and nowhere near enough
thumbnails:
# varnishtop -c -i VCL_call -q 'ReqURL ~ "/images/" and not ReqURL ~
"/images/thumb"'
349.47 VCL_call HASH
349.47 VCL_call RECV
349.47 VCL_call DELIVER
207.22 VCL_call HIT
116.40 VCL_call MISS
116.30 VCL_call PASS
# varnishtop -c -i VCL_call -q 'ReqURL ~ "/images/thumb"'
1859.60 VCL_call HASH
1859.60 VCL_call RECV
1859.60 VCL_call DELIVER
1424.83 VCL_call MISS
422.84 VCL_call HIT
218.82 VCL_call PASS
I'm still poking around trying to correlate caching of other types of URLs
based on whether or not the requests have cookies, if Cache-Control gets
returned, etc. but I just wanted to reply with this info. I do appreciate the
responses I'm getting! :)
-----Original Message-----
From: Dridi Boukelmoune [mailto:[email protected]]
Sent: Friday, December 9, 2016 10:11 AM
To: Justin Lloyd <[email protected]>
Cc: Dag Haavi Finstad <[email protected]>;
[email protected]
Subject: Re: Hit ratio dropped significantly after recent upgrades
> To reiterate on a point in another of my responses in this thread, I think it
> may be something about MediaWiki thumbnail images not being cached properly
> despite our current VCL in that regard not having changed from how it worked
> prior to the upgrade during which time we were seeing a very high (86%-ish)
> hit ratio from the same formula.
To reiterate on a point I made on a couple occasions, it's time to give
varnishlog a spin. Too much focus on VCL, and not enough on what's happening.
Dridi
_______________________________________________
varnish-misc mailing list
[email protected]
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
