Well, you guys convinced me this is more of an nginx bug (a very serious nginx bug, IMO!) than a uwsgi bug. The fact that nginx ignores the Content-Length of the data it receives from the upstream is very strange to me (http://forum.nginx.org/read.php?2,204853,205805#msg-205805).
I could argue that uwsgi's protocol is not really HTTP 1.0, it's a different protocol which happens to have a single packet that looks precisely like an HTTP 1.0 response, and this problem could be solved by adding the "EOT" marker that C Anthony described to the uwsgi protocol, not to the actual content, but I understand now it would be better to try and engage nginx's community about this than solve it particularly for uwsgi. The bottom line here is that if you use nginx, you must do end-to-end body-length declaration from your app to the end client. nginx can't be trusted to discover a disconnection in the body phase, and it can have disastrous results, especially with transparent caching. - Yaniv On Fri, Sep 30, 2011 at 2:35 AM, C Anthony Risinger <[email protected]> wrote: > On Thu, Sep 29, 2011 at 3:25 PM, Evgeny Turnaev <[email protected]> > wrote: > > 2011/9/29 Yaniv Aknin <[email protected]>: > >> > >> I'm happy you understand me now. > >> The problem is if I fix it the HTTP way, it means my WSGI app has to do > >> things that are the webserver's job (like gzip compression). > >> For example, I tried sending a 1,000 byte document with correct > >> Content-Length from my app, via nginx with gzip compression on. If I > kill > >> the connection at the 500th byte, nginx happily ignores the fact it has > a > >> 500 byte document with Content-Length of 1,000 bytes, removes the > >> Content-Length header, does gzip and adds chunked transfer encoding. > > > > This is indeed an nginx issue. Btw did you tried latest nginx? > > If it not works in latest nginx then possible workaround: > > * write a patch for nginx > > * disable gzip and pass content-length provided by wsgi app directly > > (maybe with uwsgi_pass_header) to browser the browser will show > > that a page don`t loaded.. dont know if it will cache. but still this > > is 200 ok. if you want 500 you have to teach > > nginx recognize that this is incomplete response. > > Also wsgi application may not provide already gzipped or transfer > > encoded content > > this is described in > > http://www.python.org/dev/peps/pep-0333/#other-http-features > > and > http://www.python.org/dev/peps/pep-0333/#handling-the-content-length-header > > > >> 2. Answering "fix your app so uwsgi will not die" is not a serious > response > >> in my opinion. Maybe my app had a bug, maybe uWSGI had a bug, maybe any > of > >> the libraries they use had a bug, maybe my kernel had a bug, maybe my > system > >> administrator made a mistake. This is a possibility that I don't think > you > >> can seriously ignore, and not necessarily something I can fix. I prefer > a > >> 502 in production instead of an incorrect 200 OK anyday. > >> This is a serious and complicated issue, I'd be happy to meet in IRC and > >> further explain this to those interested. > > > > Sorry Yaniv, maybe i misunderstood you point a bit. > > But still: If you app have a bug: try/catch it. If a bug in kernel: > > nothing will help you really. > > The system administrators do really make errors :) > > Although i agree with you: sad that nginx doesn't support checking of > > body response against content-length header. > > Also i think this issue can be solved with a proper support of ETag by > > nginx - so that ETag of incomplete response and complete will differ, > > but this is more like workaround. > > i don't think the problem *strictly* rests with nginx, but rather with > the well-known semantics of the HTTP/1.0 protocol: > > http://www.w3.org/Protocols/HTTP/1.0/spec.html#BodyLength > "... If a Content-Length header field is present, its value in bytes > represents the length of the Entity-Body. Otherwise, the body length > is determined by the closing of the connection by the server. ... > Closing the connection cannot be used to indicate the end of a request > body, since it leaves no possibility for the server to send back a > response. Therefore, HTTP/1.0 requests containing an entity body must > include a valid Content-Length header field. If a request contains an > entity body and Content-Length is not specified, and the server does > not recognize or cannot calculate the length from other fields, then > the server should send a 400 (bad request) response." > > ... so in all correctness, nginx should fail with 400, but nobody > honors this because the front-end client-connection is really > HTTP/1.1, ie. keepalive/chunked (i don't think a missing > content-length is treated as 400 even if client is 1.0). > > immediate issue is that nginx doesn't natively support HTTP/1.1 to > backends ... though i imagine this could technically be implemented by > uwsgi by special casing an nginx connection -- a workaround to relying > on a bottomless 1.0 connection. uwsgi would need to [re]implement > chunked encoding between the nginx module and uwsgi, out-of-band, and > reassemble/remove the frames before actually handing each chunk to > nginx, yes? > > ... sounds annoying :-) is there a way you (the app developer) could > append a finalizing "EOT" marker to your response? perhaps you could > strip the marker on 200/OK, and have nginx redirect/kill if it never > shows before socket close. i know nginx can do some interesting > matching via configuration file alone, but it might complicate/bloat > your setup, if it's possible at all. > > -- > > C Anthony > _______________________________________________ > uWSGI mailing list > [email protected] > http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi >
_______________________________________________ uWSGI mailing list [email protected] http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
