Re: Problem with cached entries w/ETag and request without If-None-Match header
but it feels like you are trying to accomplish something very unusual/weird Perhaps some more background information is in order here. What we are trying to accomplish is acceleration on a Zope/Plone CMS. The CacheFu product is primarily responsible for setting cache headers and issuing purge requests to Squid. CacheFu uses the ETag mechanism for marking variations of a cached entity based on logged in user and some other preferences. We would like to cache anonymous and authenticated versions of pages in Squid. The big problem we are facing is that if an authenticated user visits a URL, Squid will cache that page with the appropriate ETag. A subsequent request by an anonymous user will return the cached version of that page -- this may reveal information that should not be publicly available. If Squid's behavior were modified in the manner described, the anonymous request would not match the cached entity and would therefore result in a request to Zope/Plone which would serve the anonymous version of the page, which Squid would cache. This seems like a typical model for modern web applications. I now see from looking at the RFC that it does not support this type of behavior. In light of this, the patch should be extended to make the behavior a configurable option. It seems that others would be interested in this configuration as well. Alex Rousskov wrote: On 06/16/2009 08:13 AM, Jason Noble wrote: You're right, looks like I read that section too quickly. After reading the RFC more carefully, it appears that the case I'm having issues with is undefined. The closest thing I can find is 13.3.4: An HTTP/1.1 caching proxy, upon receiving a conditional request that includes both a Last-Modified date and one or more entity tags as cache validators, MUST NOT return a locally cached response to the client unless that cached response is consistent with all of the conditional header fields in the request. But I'm actually interested in the case where the caching proxy receives a non-conditional request. The current behavior of Squid is to return a cached entity. It seems to me that not returning cached entries that include conditional header fields would be more consistent with the behavior of not returning a locally cached response to the client unless that cached response is consistent with all of the conditional header fields in the request (i.e. the absence of conditional header fields does not match the presence of one or more conditional header fields.) If this case is indeed defined in the RFC, I'd be interested to know where. You are applying requirements for a conditional request [field] to an unconditional request and/or to a stored response. They are not applicable. There is simply no such thing as a conditional response field. You can sort of think of a Vary response field to be conditional (which is what Mark have noted) but, in your case, there is no Vary, right? Perhaps the server should add Vary for its specific application, but it feels like you are trying to accomplish something very unusual/weird. Why do you want Squid to ignore cached entities with ETags if a client does not use ETags or does not have ETags? Thank you, Alex. Mark Nottingham wrote: Selecting request headers are specified by Vary; If-None-Match is a conditional request header. Cheers, On 16/06/2009, at 12:44 AM, Jason Noble wrote: From RFC 2616 13.6: ... When the cache receives a subsequent request whose Request-URI specifies one or more cache entries including a Vary header field, the cache MUST NOT use such a cache entry to construct a response to the new request unless all of the selecting request-headers present in the new request match the corresponding stored request-headers in the original request. ... For the case in question, all selecting request headers do not match the stored request headers. Therefore, the cache must not use the stored entry to construct a response. --Jason Mark Nottingham wrote: What requirement in RFC2616 does this violate? On 13/06/2009, at 3:02 AM, Jason Noble wrote: I recently ran into a bug on Squid 2.7 regarding cached content with ETags. Currently, if all cached entries for a URL include ETags, and a request is received for said URL with no If-None-Match header, Squid will serve a cached entry. This behavior does not follow RFC 2616. I have attached a patch that prevents Squid from serving the cached entries in said case here: http://www.squid-cache.org/bugs/show_bug.cgi?id=2677 I would appreciate any feedback regarding this patch. Thanks, Jason -- Mark Nottingham m...@yahoo-inc.com -- Mark Nottingham m...@yahoo-inc.com
Re: Problem with cached entries w/ETag and request without If-None-Match header
As mentioned in my previous e-mail, we are working to implement caching of anonymous/authenticated versions of pages in a Zope/Plone environment. In a discussion in 2005 revolving around this same subject, you recommended to a user that they use ETags instead of Vary: Cookie in this situation ( http://www.eu.squid-cache.org/mail-archive/squid-users/200510/0491.html ) due to purging issues with Vary. We have indeed gone the ETag/If-None-Match route, but are experiencing problems this way as well. It appears to be the case that Squid (and indeed RFC2616) currently provide no way to handle this case. The point of this patch is to provide a mechanism for Squid support such behavior. If there is a way to achieve this behavior without modification to Squid, or the client we would be very interested in that technique. Thanks, Jason Henrik Nordstrom wrote: fre 2009-06-12 klockan 13:02 -0400 skrev Jason Noble: I recently ran into a bug on Squid 2.7 regarding cached content with ETags. Currently, if all cached entries for a URL include ETags, and a request is received for said URL with no If-None-Match header, Squid will serve a cached entry. This behavior does not follow RFC 2616. I have attached a patch that prevents Squid from serving the cached entries in said case here: http://www.squid-cache.org/bugs/show_bug.cgi?id=2677 I would appreciate any feedback regarding this patch. As others have already pointed out it's plain wrong. Please read RFC2616 13.6 Caching Negotiated Responses again for more details on how this works in HTTP. It's a quite simple and effective scheme when used right. In short Selecting request headers (as indicated by Vary) select which response may be returned in response to the request. If there is multiple matching responses then the most recent among them should be used. Conditional request headers determine if that response is to be a 200, 304 or in case of origin servers only 412. An unconditional request is just that, and will always respond with the matching 200 response. Clients can not select which response to return based on etag, only by using negotiation request headers as indicated by Vary, such as Accept, Accept-Language, Accept-Encoding, Cookie etc.. Regards Henrik
Re: Problem with cached entries w/ETag and request without If-None-Match header
ons 2009-07-01 klockan 16:45 -0400 skrev Jason Noble: As mentioned in my previous e-mail, we are working to implement caching of anonymous/authenticated versions of pages in a Zope/Plone environment. Ah.. As you probably know by now HTTP cache model isn't really designed for that.. content are meant to be either anonymous or restricted, not both.. Cookie based authentication and caching do not work well together. In a discussion in 2005 revolving around this same subject, you recommended to a user that they use ETags instead of Vary: Cookie in this situation ( http://www.eu.squid-cache.org/mail-archive/squid-users/200510/0491.html ) due to purging issues with Vary. We have indeed gone the ETag/If-None-Match route, but are experiencing problems this way as well. I think you misunderstood me there. In both approaches mentioned in the message above Vary is meant to be used. It appears to be the case that Squid (and indeed RFC2616) currently provide no way to handle this case. The point of this patch is to provide a mechanism for Squid support such behavior. If there is a way to achieve this behavior without modification to Squid, or the client we would be very interested in that technique. The first part of that response talks about a Squid implementation bug, outside of RFC2616 where you can use the Vary header as a kind of object version tag. Each time Squid sees a new (cachable) Vary header content old cached entries for that URL will get lost as the Vary index key function gets changed as it's based on last known Vary header for the resource. Second part mentions that Vary support in Squid is a lot better with the ETag support, then available as a patch and nowdays a standard part of Squid-2 (but not yet Squid-3). What I meant was that you should do this combination: * Send appropriate ETag values in the responses * Handle If-None-Match * In the Vary header, apart from the headers the response varies on (i.e. Accept-Encoding, Cookie), also include a document version identifier as a custom header such as X-Change- where the first version of the object will have X-Change-1, second version X-Change-2 etc... The dummy version/change header as such do not need to be present in the response, just mentioned in Vary. The first two optimizes the cache use, and if done right makes life easier for your server as well. The second makes sure that once Squid learns that one variant of the object have changed it will loose track of the other cached variants (cache misses on next request until cached again). But please note that this is actually an implementation bug/shortcoming and may change in future.. But either way it won't do very well, unless your Cookie based authentication is the only cookies you use AND you make sure to clear the cookies on anonymous users AND non-anonymous content is marked non-cachable. Regards Henrik
Re: Problem with cached entries w/ETag and request without If-None-Match header
fre 2009-06-12 klockan 13:02 -0400 skrev Jason Noble: I recently ran into a bug on Squid 2.7 regarding cached content with ETags. Currently, if all cached entries for a URL include ETags, and a request is received for said URL with no If-None-Match header, Squid will serve a cached entry. This behavior does not follow RFC 2616. I have attached a patch that prevents Squid from serving the cached entries in said case here: http://www.squid-cache.org/bugs/show_bug.cgi?id=2677 I would appreciate any feedback regarding this patch. As others have already pointed out it's plain wrong. Please read RFC2616 13.6 Caching Negotiated Responses again for more details on how this works in HTTP. It's a quite simple and effective scheme when used right. In short Selecting request headers (as indicated by Vary) select which response may be returned in response to the request. If there is multiple matching responses then the most recent among them should be used. Conditional request headers determine if that response is to be a 200, 304 or in case of origin servers only 412. An unconditional request is just that, and will always respond with the matching 200 response. Clients can not select which response to return based on etag, only by using negotiation request headers as indicated by Vary, such as Accept, Accept-Language, Accept-Encoding, Cookie etc.. Regards Henrik
Re: Problem with cached entries w/ETag and request without If-None-Match header
On 06/16/2009 08:13 AM, Jason Noble wrote: You're right, looks like I read that section too quickly. After reading the RFC more carefully, it appears that the case I'm having issues with is undefined. The closest thing I can find is 13.3.4: An HTTP/1.1 caching proxy, upon receiving a conditional request that includes both a Last-Modified date and one or more entity tags as cache validators, MUST NOT return a locally cached response to the client unless that cached response is consistent with all of the conditional header fields in the request. But I'm actually interested in the case where the caching proxy receives a non-conditional request. The current behavior of Squid is to return a cached entity. It seems to me that not returning cached entries that include conditional header fields would be more consistent with the behavior of not returning a locally cached response to the client unless that cached response is consistent with all of the conditional header fields in the request (i.e. the absence of conditional header fields does not match the presence of one or more conditional header fields.) If this case is indeed defined in the RFC, I'd be interested to know where. You are applying requirements for a conditional request [field] to an unconditional request and/or to a stored response. They are not applicable. There is simply no such thing as a conditional response field. You can sort of think of a Vary response field to be conditional (which is what Mark have noted) but, in your case, there is no Vary, right? Perhaps the server should add Vary for its specific application, but it feels like you are trying to accomplish something very unusual/weird. Why do you want Squid to ignore cached entities with ETags if a client does not use ETags or does not have ETags? Thank you, Alex. Mark Nottingham wrote: Selecting request headers are specified by Vary; If-None-Match is a conditional request header. Cheers, On 16/06/2009, at 12:44 AM, Jason Noble wrote: From RFC 2616 13.6: ... When the cache receives a subsequent request whose Request-URI specifies one or more cache entries including a Vary header field, the cache MUST NOT use such a cache entry to construct a response to the new request unless all of the selecting request-headers present in the new request match the corresponding stored request-headers in the original request. ... For the case in question, all selecting request headers do not match the stored request headers. Therefore, the cache must not use the stored entry to construct a response. --Jason Mark Nottingham wrote: What requirement in RFC2616 does this violate? On 13/06/2009, at 3:02 AM, Jason Noble wrote: I recently ran into a bug on Squid 2.7 regarding cached content with ETags. Currently, if all cached entries for a URL include ETags, and a request is received for said URL with no If-None-Match header, Squid will serve a cached entry. This behavior does not follow RFC 2616. I have attached a patch that prevents Squid from serving the cached entries in said case here: http://www.squid-cache.org/bugs/show_bug.cgi?id=2677 I would appreciate any feedback regarding this patch. Thanks, Jason -- Mark Nottingham m...@yahoo-inc.com -- Mark Nottingham m...@yahoo-inc.com
Re: Problem with cached entries w/ETag and request without If-None-Match header
You're right, looks like I read that section too quickly. After reading the RFC more carefully, it appears that the case I'm having issues with is undefined. The closest thing I can find is 13.3.4: An HTTP/1.1 caching proxy, upon receiving a conditional request that includes both a Last-Modified date and one or more entity tags as cache validators, MUST NOT return a locally cached response to the client unless that cached response is consistent with all of the conditional header fields in the request. But I'm actually interested in the case where the caching proxy receives a non-conditional request. The current behavior of Squid is to return a cached entity. It seems to me that not returning cached entries that include conditional header fields would be more consistent with the behavior of not returning a locally cached response to the client unless that cached response is consistent with all of the conditional header fields in the request (i.e. the absence of conditional header fields does not match the presence of one or more conditional header fields.) If this case is indeed defined in the RFC, I'd be interested to know where. Thanks, Jason Mark Nottingham wrote: Selecting request headers are specified by Vary; If-None-Match is a conditional request header. Cheers, On 16/06/2009, at 12:44 AM, Jason Noble wrote: From RFC 2616 13.6: ... When the cache receives a subsequent request whose Request-URI specifies one or more cache entries including a Vary header field, the cache MUST NOT use such a cache entry to construct a response to the new request unless all of the selecting request-headers present in the new request match the corresponding stored request-headers in the original request. ... For the case in question, all selecting request headers do not match the stored request headers. Therefore, the cache must not use the stored entry to construct a response. --Jason Mark Nottingham wrote: What requirement in RFC2616 does this violate? On 13/06/2009, at 3:02 AM, Jason Noble wrote: I recently ran into a bug on Squid 2.7 regarding cached content with ETags. Currently, if all cached entries for a URL include ETags, and a request is received for said URL with no If-None-Match header, Squid will serve a cached entry. This behavior does not follow RFC 2616. I have attached a patch that prevents Squid from serving the cached entries in said case here: http://www.squid-cache.org/bugs/show_bug.cgi?id=2677 I would appreciate any feedback regarding this patch. Thanks, Jason -- Mark Nottingham m...@yahoo-inc.com -- Mark Nottingham m...@yahoo-inc.com
Re: Problem with cached entries w/ETag and request without If-None-Match header
From RFC 2616 13.6: ... When the cache receives a subsequent request whose Request-URI specifies one or more cache entries including a Vary header field, the cache MUST NOT use such a cache entry to construct a response to the new request unless all of the selecting request-headers present in the new request match the corresponding stored request-headers in the original request. ... For the case in question, all selecting request headers do not match the stored request headers. Therefore, the cache must not use the stored entry to construct a response. --Jason Mark Nottingham wrote: What requirement in RFC2616 does this violate? On 13/06/2009, at 3:02 AM, Jason Noble wrote: I recently ran into a bug on Squid 2.7 regarding cached content with ETags. Currently, if all cached entries for a URL include ETags, and a request is received for said URL with no If-None-Match header, Squid will serve a cached entry. This behavior does not follow RFC 2616. I have attached a patch that prevents Squid from serving the cached entries in said case here: http://www.squid-cache.org/bugs/show_bug.cgi?id=2677 I would appreciate any feedback regarding this patch. Thanks, Jason -- Mark Nottingham m...@yahoo-inc.com
Re: Problem with cached entries w/ETag and request without If-None-Match header
Selecting request headers are specified by Vary; If-None-Match is a conditional request header. Cheers, On 16/06/2009, at 12:44 AM, Jason Noble wrote: From RFC 2616 13.6: ... When the cache receives a subsequent request whose Request-URI specifies one or more cache entries including a Vary header field, the cache MUST NOT use such a cache entry to construct a response to the new request unless all of the selecting request-headers present in the new request match the corresponding stored request-headers in the original request. ... For the case in question, all selecting request headers do not match the stored request headers. Therefore, the cache must not use the stored entry to construct a response. --Jason Mark Nottingham wrote: What requirement in RFC2616 does this violate? On 13/06/2009, at 3:02 AM, Jason Noble wrote: I recently ran into a bug on Squid 2.7 regarding cached content with ETags. Currently, if all cached entries for a URL include ETags, and a request is received for said URL with no If-None- Match header, Squid will serve a cached entry. This behavior does not follow RFC 2616. I have attached a patch that prevents Squid from serving the cached entries in said case here: http://www.squid-cache.org/bugs/show_bug.cgi?id=2677 I would appreciate any feedback regarding this patch. Thanks, Jason -- Mark Nottingham m...@yahoo-inc.com -- Mark Nottingham m...@yahoo-inc.com
Re: Problem with cached entries w/ETag and request without If-None-Match header
What requirement in RFC2616 does this violate? On 13/06/2009, at 3:02 AM, Jason Noble wrote: I recently ran into a bug on Squid 2.7 regarding cached content with ETags. Currently, if all cached entries for a URL include ETags, and a request is received for said URL with no If-None-Match header, Squid will serve a cached entry. This behavior does not follow RFC 2616. I have attached a patch that prevents Squid from serving the cached entries in said case here: http://www.squid-cache.org/bugs/show_bug.cgi?id=2677 I would appreciate any feedback regarding this patch. Thanks, Jason -- Mark Nottingham m...@yahoo-inc.com
