I've worked out that mod_python can be made to work, but can't be fixed until:
http://issues.apache.org/jira/browse/MODPYTHON-125 is fixed. More details in a few days. Graham Clark Rawlins wrote .. > Thanks for the reply and the explanation. > > I am reading the content using the sax api. I don't know if it uses > sized reads or not but based on what you are saying here it must use > sized reads. I haven't had any problems yet but that is probably > related in part to the client and server I am using and the data I am > passing, and I can see based on your explanation that in general it may > not be safe. > > If I understand the situation, to handle this situation correctly the > mod_python code would need to provide some kind of buffering between the > python read and the read to the request header. That buffering would > need to figure out what size the next chunk was and read that size and > then satisfy reads from python from the buffer it read into. Does that > sound right? Can you point out the mechanism Apache supplies to figure > out what the next chunk size is? > > > On Fri, 2007-02-23 at 01:12 -0500, Graham Dumpleton wrote: > > Clark Rawlins wrote .. > > > Hello, > > > > > > I have a client I need to handle post requests for that uses > > > Content-Encoding: xml and Transfer-Encoding: chunked. > > > > > > Now the Content-Encoding is no problem I plan on using sax to parse > the > > > content. The chunked transfer encoding is where I have run into issues. > > > > > > I tracked it down to the part of the mod-python code that handles > > > reading the request body from apache. > > > > > > In two places it uses: > > > rc = ap_setup_client_block(self->request_rec, REQUEST_CHUNKED_ERROR); > > > > > > As a test, I modified the code to use REQUEST_CHUNKED_DECHUNK and it > > > worked great for me. Now I imagine that this may cause some problems > > > for scripts using mod-python to handle requests but I couldn't think > of > > > any that wouldn't already be quite serious with a request of MAX_INT > > > length. What if anything am I missing and if I am not, could this > > > change be incorporated into the main tree? > > > > To make it short, mod_python can't be made to handle chunked incoming > > content from a client and the change you suggest will cause other problems. > > > > This is because when using REQUEST_CHUNKED_DECHUNK it is important > > that the read buffer be large enough to handle any chunk being read. > Ie., > > from Apache source code: > > > > * 1. Call ap_setup_client_block() near the beginning of the request > > * handler. This will set up all the necessary properties, and will > > * return either OK, or an error code. If the latter, the module should > > * return that error code. The second parameter selects the policy > to > > * apply if the request message indicates a body, and how a chunked > > * transfer-coding should be interpreted. Choose one of > > * > > * REQUEST_NO_BODY Send 413 error if message has any body > > * REQUEST_CHUNKED_ERROR Send 411 error if body without Content-Length > > * REQUEST_CHUNKED_DECHUNK If chunked, remove the chunks for me. > > * REQUEST_CHUNKED_PASS If chunked, pass the chunk headers with > body. > > * > > * In order to use the last two options, the caller MUST provide a > buffer > > * large enough to hold a chunk-size line, including any extensions. > > > > If chunked is used the mod_python source can't know what the buffer size > > it would need to use is as that is going to be dependent on the higher > level > > application and what it is doing. > > > > Thus a general purpose solution like mod_python can't be used and what > is > > really needed is a lower level Python interface to Apache API so you > can control > > things properly specific for your case. I have talked about the need > for such > > a better Python/Apache interface previously on the list but nothing public > is > > probably going to be available in that space in the near future. > > > > BTW, I am basing this analysis on reading Apache code. If you are seeing > > different results and things are working then I can only assume it is > because > > you are using read() with a size. If you use read() without a size, you > will > > probably not get any content. Even when specifying an argument you might > > get problems as mod_python will always read up to that size and not simply > > return what might be available at that time from a chunk. > > > > Anyway, I'll have a think about the problem some more as will affect > some other > > code I am working on. :-) > > > > Graham > > > > > Index: src/requestobject.c > > > =================================================================== > > > --- src/requestobject.c (revision 510030) > > > +++ src/requestobject.c (working copy) > > > @@ -864,7 +864,7 @@ > > > if (! self->request_rec->read_length) { > > > > > > /* then do some initial setting up */ > > > - rc = ap_setup_client_block(self->request_rec, > > > REQUEST_CHUNKED_ERROR); > > > + rc = ap_setup_client_block(self->request_rec, > > > REQUEST_CHUNKED_DECHUNK); > > > if(rc != OK) { > > > PyObject *val = PyInt_FromLong(rc); > > > if (val == NULL) > > > @@ -964,7 +964,7 @@ > > > if (! self->request_rec->read_length) { > > > > > > /* then do some initial setting up */ > > > - rc = ap_setup_client_block(self->request_rec, > > > REQUEST_CHUNKED_ERROR); > > > + rc = ap_setup_client_block(self->request_rec, > > > REQUEST_CHUNKED_DECHUNK); > > > > > > if (rc != OK) { > > > PyObject *val = PyInt_FromLong(rc);
