Apologies added this to a new thread instead of existing (as i have digest mode set)
On Wed, Jul 12, 2017 at 9:12 PM, Matthew Johnson <[email protected]> wrote: >> Date: Wed, 12 Jul 2017 11:20:52 +0200 >> From: Geoff Simmons <[email protected]> >> To: [email protected] >> Subject: Re: Value of builtin.vcl - vcl_recv on modern websites >> Message-ID: <[email protected]> >> Content-Type: text/plain; charset=windows-1252 >> >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA256 >> >> On 07/12/2017 09:14 AM, Matthew Johnson wrote: >>> >>> Since Varnish 1.0 the builtin (or default) vcl_recv has had this >>> statement: if (req.http.Authorization || req.http.Cookie) { >>> >>> /* Not cacheable by default */ >>> >>> return (pass); >>> >>> } >>> >>> My issue with this is the req.http.Cookie check, In any modern >>> website cookies are always present. >> >> While I'm sympathetic to all the things you said and don't mean to >> disregard it but cutting things short, the answer here is simple: >> there is no other choice for the default configuration of a caching >> proxy, because those two headers mean that the response may be >> personalized. > > Agreed. I think the area im looking to explore is how you move > forwards from the default configuration as there are a few paths that > can be taken. > >> >> Many uses of cookies don't have that effect, of course, but Varnish >> has no way of knowing that. As bad as the effect of the default config >> may seem on sites that use cookies on every request -- Varnish doesn't >> cache anything -- it would be much worse if someone sets up Varnish, >> doesn't think of the consequences of not changing the default >> configuration, and you end up seeing someone else's personal data in >> cached responses. >> >> This problem is not specific to Varnish, but to any server that tries >> to do what Varnish tries to do. I know from experience that it's >> generally futile to say so, but this situation really ought to lead to >> some widespread re-thinking throughout the industry. Forgive me for >> shouting, soapbox-style, but this gives me an opportunity to sound off >> on a pet peeve: >> >> ==> Maybe modern web site SHOULD NOT use cookies on every request! >> Because of the way cookies interfere with downstream caching. >> >> I have come to the conviction that many uses of cookies are a result >> of lazy thinking in app development. Many PHP devs, for example, are >> in the habit of saying session_start(); at the beginning of every >> script, without thinking twice about whether they really need it. I >> have seen uses of cookies where "just toss that thing into a cookie" >> was evidently the easy decision to make. I have seen cookies with >> values that are 3KB long. > > Most .Net sites are the same, Almost any application I come across > takes a "session first" approach. > > >> >> (Sometime over beer I'll tell you about that little database that >> someone wanted to transport over a cookie, a base64-encoded CSV file >> whose data was *also* base64-encoded, leading to a doubly >> base64-encoded cookie value, in every request.) > Beer sounds good. There are definitely many war stories on my side of > madness in the use of cookies. Database in cookie in a novel approach! > >> >> This is an instance of an issue that you encounter a lot with the use >> of Varnish in practice: app development that doesn't think outside of >> its own box in terms of functionality and performance. Rather than >> thinking about the benefits of handing off some of your work, by >> letting someone else serve your cached responses for you. >> >> HTTP was conceived from the beginning to enable caching as a means of >> solving performance problems in slow networks. A well-configured >> deployment of Varnish shows how beneficial that can be. But the >> universal and unreflected use of cookies is one of the forces >> presently at work that actively undermine that part of the equation. >> >>> A classic example of this is someone adding javascript via Google >>> Tag Manager which then sets a cookie. >> >> One might have hoped that the Googlers, of all people, would have more >> awareness of the trouble that they could cause by doing that. >> > Whilst Google do contribute to the issue with their own scripts, > Google Tag Manager allows non technical people to deploy additional > 3rd party javascript on a website. In the web performance space this > always leads to slow loading websites in the browser but the cookie > problem then plays into caching rules aswell. > >>> Do you still recommend configurations fall through to the >>> underlying vcl_recv logic? >>> >>> Options i can think of: >> >> In a project where I am able to work with the app devs, I have had >> good experience with working out a policy with them: if you MUST have >> cookies in every request (although I WISH YOU WOULD RECONSIDER THAT), >> then the caching proxy cannot make caching decisions on your behalf. >> Only you can know if your response is cacheable, despite the presence >> of cookie foo or bar, but is not cacheable if the cookie is baz. >> >> So if you want your response to be cached, you MUST say so in a >> Cache-Control header. The proxy will not cache any other responses. > > I consider this ideal if it's possible to take that approach. I am > often caught with customers using off the shelf platforms and less > than ideal control over their application. > >> >> Then we write VCL to bypass builtin's vcl_recv, and start Varnish with >> - -t 0 (default TTL is 0s). Responses are then cached only if they >> announce that they are cacheable. >> >> Of course, this has the effect that you're lamenting -- Varnish >> doesn't cache anything by default -- but in my experience, the result >> is that devs have become very good at thinking about the cacheability >> of their responses. >> >> That boils down to answering your question by saying no, you can't use >> builtin vcl_recv in a situation like that. When the cookies, like the >> Evil, are always and everywhere (to paraphrase a saying in Germany), >> and some cookies lead to cacheable responses while others don't, then >> there's no other option for a caching proxy. > > A great summary, I think that summarises Dridi views earlier around > the way HTTP has been implemented in applications. No perfect wins! > >> >> >> Best, >> Geoff >> - -- >> ** * * UPLEX - Nils Goroll Systemoptimierung >> >> Scheffelstra?e 32 >> 22301 Hamburg >> >> Tel +49 40 2880 5731 >> Mob +49 176 636 90917 >> Fax +49 40 42949753 >> >> http://uplex.de >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v2 >> >> iQIcBAEBCAAGBQJZZen0AAoJEOUwvh9pJNURnWEQAK9ucYSXEcrEwbmOrroBWoGK >> iR9a8OFhst1rVKFQ2vNTUpw+OYM8vf8SJYToyq2VxG5/f/uGsT6nSVRWGKThgeV1 >> geyyUQfbbDte1at4aFy6HX6LeCt62Si0L9KUZMZMkI5C6m6FKgA/5HecUchhuXdP >> Li17DXKvzrQyTBJpvbk2vBZGPVlErnVAUz75IeJMrD6t/WGO0PsZvkC/l8LZhqcD >> 2S2R9SHYMIyBrWSZm+YsI1DxMwvH6Gt84NRPpKcHQQ7TKEfvtOq0NwoqcNOB26EL >> KIfOuVJdbiMvD5D+BZud/7a7UzSpJz5klLMdTcJMN60MrHJjGcok/5KiG7TNNowj >> hMy5YUYpuIybsWzcvB5Ie/nteb0WyXt5+LYkxjP9dbN7AN3k+aU1PboSOyqYXO3u >> KK1al00LMKHfzMHs1vF3QHRt2Q1Udud6dCdHuw6TyJ7eWCc9YGgU8NyboMLkXBhO >> fBVNUQQjfNjaDWhKFvUIMEsGZhgvzzuMvjlZNhkc/lcDLmU8wVXyiMFSoBcuR1sX >> 7EM1wBUKcKix0wE4QPl9608ql/5LF3Ms+wqDpmS0ECgFIf1yKMWFZt9iHhMUbch2 >> 7fhr77vVjVD1K6nKHqDGOuLp4Cq+lfBJkd7PX2huQUV/hc00C8+NEieD77wuAwk7 >> OPiGNt5YqmDNjtZmFUVH >> =BlAn >> -----END PGP SIGNATURE----- >> >> >> >> ********************************************* _______________________________________________ varnish-misc mailing list [email protected] https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
