Re: [go-nuts] Standard library ETag/Last-Modified conditional request logic and best practices?

2019-08-06 Thread mark . mcdx
Thanks Devon!

So just to clarify our request flow is:

Client > CDN > Go Reverse Proxy > Origin

Our Go Reverse Proxy has historically been responsible for adding caching 
headers (e.g. Cache-Control and Surrogate-Control) when the origins have 
failed to do so (as a way to ensure things are cached appropriately).

It's unclear to me why you should be setting an etag header if you're a 
> proxy. 
>

That's why when it came to looking at setting serve stale defaults for our 
origins (e.g. stale-while-revalidate and stale-if-error) I realized that 
somewhere along the chain an appropriate ETag/Last-Modified should be set 
and that's why I started wondering if our proxy should be responsible for 
setting them.

Even then I felt like setting Last-Modified was way outside the 
responsibility of our proxy, but that maybe setting of ETag would have 
sufficed.

Unless you're serving from the filesystem handler (which does 
> implement IMS/INM), you'll need to implement these yourself. 
>

I think your other related answers might explain to me why the go reverse 
proxy doesn't support conditional requests, in that it's NOT a 'caching 
proxy' and so being able to handle that revalidation logic wouldn't make 
sense.
 

> Note that you _could_ simply proxy this to the origin and let it 
> handle the validation. This is often overkill for what people actually 
> need, but it is guaranteed to work. 
>

OK, so as we are indeed just proxying the request pretty much 'as is' to 
the origin, i.e. the CDN is making the revalidation conditional request 
when our stale-while-revalidate TTL expires, I'm guessing (I appreciate 
this is the 'basics' of how a proxy works, but I want to talk it through in 
case I'm mistaken in any way!) the go proxy will transparently keep that 
information for the origin to respond with the appropriate 
ETag/Last-Modified, and the go proxy again will transparently pass back 
their response through to the CDN to then update its cache if it indeed got 
a `200 OK` from origin or to continue serving stale if the origin returned 
a `304 Not Modified` (and in either case I expect the origin should send 
ETag/Last-Modified headers regardless of 200/304 status').

A hash function over the body of the response would constitute strong 
> validation. I'm not sure why you'd need to mix in the path; there's 
> nothing wrong with serving the exact same content between two 
> endpoints, and the ETag is tied to a response object. 
>

Ah ok, so I was thinking along these lines, but was getting confused 
between content that is cached vs content that is rendered at 'runtime' 
(e.g. I was getting confused with the response containing a 

[go-nuts] Standard library ETag/Last-Modified conditional request logic and best practices?

2019-08-06 Thread mark . mcdx
Hello,

I'm using Go's standard library reverse proxy and I'm trying to figure out 
if the standard library HTTP web server (e.g. http.ListenAndServe) 
implements the relevant conditional request handling logic for 
ETag/Last-Modified headers.

I did some Googling and noticed the HTTP file system request handler 
(https://golang.org/src/net/http/fs.go) does implement that logic, but I 
couldn't find the same for the HTTP web server.

I also couldn't find any examples of setting ETags/Last-Modified (other 
than this basic implementation for setting ETags: 
https://github.com/go-http-utils/etag/blob/master/etag.go).

What's confusing me there is the concept of "strong" and "weak" validation 
and how certain scenarios might influence whether an ETag is marked as 
either strong or weak (depending on the implementation -- see 
https://developer.mozilla.org/en-US/docs/Web/HTTP/Conditional_requests#Validators).

So to recap, my questions are (and I appreciate some of these are outside 
the scope of just Go -- so apologies if that's not allowed in this forum):

1. Should I set ETag/Last-Modified in a proxy? Last-Modified feels like 
it's not the responsibility of the proxy but the origin, where as an ETag 
is something I feel is "ok" to do in the proxy as a 'fallback' (as we 
already set 'serve stale' caching headers on behalf of our origins if they 
neglect to include them).

2. Do I need to implement `If-None-Match` and `If-Modified-Since` 
behaviours myself (i.e. is it not provided by the Go standard library's 
HTTP web server)?

3. I was planning on setting an ETag header on the response from within 
httputil.ReverseProxy#ModifyResponse but wasn't sure if that would be the 
correct place to set it.

4. What constitutes a strong/weak validator (e.g. would a simple hash 
function generating a digest of the URL path + response body suffice)?

Thanks for any help/insights/opinions y'all can share with me.

Kind regards,
Mark

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/b9428dbf-55d7-4ae2-ab11-523f94102071%40googlegroups.com.