Evgeny, again, you missed my point. I don't want uWSGI to add an HTTP header. I want it to fix the uwsgi protocol, not by adding HTTP headers, by fixing the uwsgi protocol.
In HTTP, there is a way to prevent a broken connection to the client: add content-length or use chunked-encoding. In the uwsgi protocol, there is no such way, and /me adding HTTP headers will not fix the uwsgi protocol/. As I feared, this problem will not go away even if I add HTTP Content-Length to the response from my app; this must be fixed at the uwsgi level. Take, for example, the following scenario: I have a uWSGI app behind nginx. nginx is in charge of gzip compression, so it will discard any content-length header I provide and replace it with chunked transfer encoding. Now my app is outputting its output, and uwsgi is passing this output to nginx. Suddenly uwsgi dies, I will address "why it died" at the end of this email. Now, what happened? - because the uwsgi protocol has no framing, /not HTTP framing, uwsgi framing/, nginx has no way of knowing something went wrong - nginx happily takes the response I wrote, /ignores and removes my Content-Length header and adds chunked transfer encoding/, and returns 200 OK to the client, possibly with an ETag and with long cache expiry - the client receives this incorrect response, thinks it's OK and caches it - Now even if I reload my uWSGI, the client will forever be stuck with bad data in its cache! If it will send a request with If-None-Match with this ETag, my app will (correctly) reply with 304 Unmodified, because this is the correct ETag and I have no way of knowing something went wrong! So yes, this scenario is much much worse than a single 502. If the entity doing the caching is an ISP transparant proxy, it could affect hundreds of thousands of people, and I will never know about it - all because uwsgi has no framing! I am attaching a test application I used to explore this case, which proves that even if my app sends a correct Content-Length, it will not solve this problem. Now, two general remarks: 1. Roberto, please try opening a TCP socket in Linux, sniff the connection, and kill -9 (or SIGSEGV, or exit()) the process. You will see that a FIN is sent, not a RST. 2. Answering "fix your app so uwsgi will not die" is not a serious response in my opinion. Maybe my app had a bug, maybe uWSGI had a bug, maybe any of the libraries they use had a bug, maybe my kernel had a bug, maybe my system administrator made a mistake. This is a possibility that I don't think you can seriously ignore, and not necessarily something I can fix. I prefer a 502 in production instead of an incorrect 200 OK anyday. This is a serious and complicated issue, I'd be happy to meet in IRC and further explain this to those interested. - Yaniv On Thu, Sep 29, 2011 at 9:32 PM, Evgeny Turnaev <[email protected]> wrote: > 2011/9/29 Yaniv Aknin <[email protected]>: > > I don't think I explained myself well. > > You're telling me to solve a problem which I think exists with the uwsgi > > protocol, by doing stuff with the HTTP protocol. But the real way to > solve > > it is by augmenting the uwsgi protocol. I'm not talking about uWSGI > adding a > > "Content-Length" HTTP header, because indeed it can't know the true > length > > of the response and it shouldn't do the webserver's job. But I AM talking > > about uWSGI adding a "uwsgi-content-length" header, because the uwsgi > > protocol is up to uWSGI. > > In other words, the type of failure I'm talking about can not happen in a > > server like mod_wsgi, because it's in-process; there can be no > communication > > failure between mod_wsgi and Apache. However, uWSGI lives outside of > nginx, > > and has it's own protocol (which isn't HTTP, it just happens to look like > > HTTP for the case of modifier1==72, with 'TT' length and 'P' modifier2). > The > > uwsgi protocol can and should protect against uwsgi-nginx failure, even > if > > the HTTP protocol didn't add Content-Length or chunked encoding yet (for > > example, precisely because I want the webserver to handle this). > > I hope this makes it clearer, > > - Yaniv > > Why should uwsgi add content-length because you application get killed? > How can uWSGI add content-length if application produces infinity > chunks with yield? > How will 502 will help you in production more than incomplete request? > Why should uwsgi protocol protect against uwsgi kill/crash? think of > it - is there any protocol > that protects you from just uplugging ethernet cable off? well ok with > content-length you can know > the length of body. so for how long nginx would wait for complete > body? 1 minute? 5 minutes? 5 seconds? what if next yield is withing 6 > second? > > Hmm it seems very strange to me that uwsgi should do buffering and > discard chunked transfer encoding > for the purpose of application that occasionally get killed. > Why dont you do it right so it wont get killed? or maybe you > application should provide content-length header? > > > > On Thu, Sep 29, 2011 at 8:48 PM, Roberto De Ioris <[email protected]> > wrote: > >> > >> > -- > >> > >> Obviously you can bypass the problem doing the cache write within the > app > >> itself (using memcached or something similar). Memcached is supported > out > >> of the box by nginx. > >> > >> -- > >> Roberto De Ioris > >> http://unbit.it > >> _______________________________________________ > >> uWSGI mailing list > >> [email protected] > >> http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi > > > > > > _______________________________________________ > > uWSGI mailing list > > [email protected] > > http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi > > > > > > > > -- > -------------------------------------------- > Турнаев Евгений Викторович > +7 906 875 09 43 > -------------------------------------------- > _______________________________________________ > uWSGI mailing list > [email protected] > http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi >
testapp.py
Description: Binary data
_______________________________________________ uWSGI mailing list [email protected] http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
