On 08/16/2010 01:17 PM, Henrik Nordström wrote:
mån 2010-08-16 klockan 11:43 -0600 skrev Alex Rousskov:
I am revisiting this issue in hope to enable HTTP/1.1 to clients. If
Squid properly dechunks requests on the client side, what should happen
to that dechunked request on the server side? Let's start with the most
general case were we do _not_ know the origin server version and where
there are no matching ACLs to control Squid behavior.
I see several options:
1. Always send a chunked request (because the client did so and perhaps
the client knows that the server is HTTP/1.1 and can handle a chunked
request.)
But the client doesn't really know.. it knows Squid. HTTP version,
transfer encoding etc is hop-by-hop, not end-to-end. Well, there is Via,
but not sent on 100 Continue for example.
Specs says "MUST include a valid Content-Length header field unless the
server is known to be HTTP/1.1 compliant".
But we should at least have a knob for tuning this. I don't really agree
with the specs here. As you note if the client sends chunked then in
reality it's quite likely the server supports chunked as the client most
likely have some out of band information about the server capabilities.
2. Always send dechunked request with a Content-Length field, after
buffering all the chunks and calculating content length (because we do
not know whether the server is HTTP/1.1 and can handle a chunked request).
Not really doable. Request may be huge and buffer space limited.
I am entirely fine with dechunking and converting to content-length
should we already have the data buffered. But not to use it as default
mechanism.
Limited buffering plays somewhat badly with 100 Continue. I am not very
comfortable with sending a 411 after 100 Continue even if not strictly
illegal. Very unlikely clients deal well with such situation as specs
says the 100 Continue indicates explicitly that the origin server is
willing to accept the body.
3. Always add an "Expect: 100-continue" if not already there, send the
headers, and then pick option #1 or #2, depending on the server version
if we get a 100 Continue response.
This should work out reasonably well I think, but adds a fair amount of
complexity.
If we get 100 Continue then the server is supposed to be 1.1. But MAY be
an intermediary seding 100 Continue (not strictly legal..)
If we add 100-continue then we must also retry without 100-continue if
receiving a 417. Which also means that we need to buffer the request
while it's forwarded, which risks running into the same limited
buffering issue, requiring us to send a 411 response if buffer space has
been exceeded and receiving a 417 response.
Any other options?
I don't think a single hard answer can be given. More likely a mix and
some knobs to tune to real world brokenness will be needed. But in the
short run blindly forwarding quite liekly works.
Since Squid is a program and not a human being, we do need to hard-code
a single default. Clearly, there will be ACLs to change the behavior,
but if no options apply, we still need to do something.
Yu have more-or-less said "no" to every option I listed :-). Correct me
if I am wrong, but I sense that option #1 (always send chunked request)
is the "least bad" default. I will try to implement that unless you stop me.
Should the choice depend on whether we are a direct
or interception proxy?
Not really. End result is pretty much the same, except that the client
have even less knowledge about the state of things when being
intercepted.
An interception client is arguably more likely to know the next hop
capabilities because it thinks it is talking directly to that hop.
Similarly, we are less likely to be blamed for screwing things up if we
just repeat what the intercepted client did.
Thank you,
Alex.
