On 2011-01-19, at 2:50 AM, "Tyler Love" <[email protected]> wrote:
> I know this is a commonly misunderstood aspect of Varnish, but I think I have > the hang of it. > > I am using varnish on a very high traffic website and have yet to find a > satisfactory approach to expiring objects from the cache. You need a reasonable default that balances performance and freshness. > > My goal is to increase ttls (1 day, maybe even a week), to maximize hit > rates, and expire objects when content is changed. This has proven to be more > difficult than I anticipated. > > Indeed. Admin wants long cache, business wants instant everything. > > The first approach was to use the "purge" function that varnish gives you. > The caveat that defated this approach was that purge is basically a memory > leak, that also slows down requests. > > Please correct me if I have a misunderstanding of purge but, when you call > purge you add either a url or a regex to a list stored within varnish, and > then it checks all incoming requests against this url and makes sure to not > serve them from the cache. True, it is a ban list. We do not get to manage the cache directly, we simply steer the ship. > > This make sense with an understanding of how objects are stored in varnish, > but the name "purge" is less than ideal and even misleading (I read this is > being changed in trunk?). > English words convey extra meaning, which unfortunately does not resonate with machines. > Second, banned items are not served in grace mode, which is behavior I do not > want. > > So purge if backend is healthy? Should be possible in vcl. > > The alternative is to set the ttl on an object to 0 (zero). The first caveat > was being fully aware of every single resource/url that needs to be called > for a given purge/expiration. One big problem area is caching on paginated > urls. Sounds like your app needs to set a header which you can use in recv. > > Second was the sheer volume of purges I need to send. At the scale I am at, I > have to purge approximately 400-800 urls per second (I have multiple > instances of varnish running by the way). > Your app needs to communicate with your vcl better. > Lastly, purging the objects with different compression (Accept-Encoding: > gzip, deflate, etc) in the cache. I now have to multiply the amount of urls > for every possible encoding I may or may not use. > Normalize these where possible or again it will get complex. > > > So there you have it, removing objects from varnish is tedious, and full of > pitfalls. I understand the architecture of varnish just well enough to know > why it works this way, but I still wish it was better. Again, if you are trying to manage the objects you aren't getting what you should from varnish. Your app and your vcl should get you a nice hit rate. > > If I am approaching this problem the wrong way, I am more than happy to hear > some thoughts. > > Tyler > ______________________________ Stefan Caunter Operations TorstarDigital 416.561.4871 _______________________________________________ varnish-misc mailing list [email protected] http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
