Clark Rawlins wrote ..
> Hello,
>
> I have a client I need to handle post requests for that uses
> Content-Encoding: xml and Transfer-Encoding: chunked.
>
> Now the Content-Encoding is no problem I plan on using sax to parse the
> content. The chunked transfer encoding is where I have run into issues.
>
> I tracked it down to the part of the mod-python code that handles
> reading the request body from apache.
>
> In two places it uses:
> rc = ap_setup_client_block(self->request_rec, REQUEST_CHUNKED_ERROR);
>
> As a test, I modified the code to use REQUEST_CHUNKED_DECHUNK and it
> worked great for me. Now I imagine that this may cause some problems
> for scripts using mod-python to handle requests but I couldn't think of
> any that wouldn't already be quite serious with a request of MAX_INT
> length. What if anything am I missing and if I am not, could this
> change be incorporated into the main tree?
To make it short, mod_python can't be made to handle chunked incoming
content from a client and the change you suggest will cause other problems.
This is because when using REQUEST_CHUNKED_DECHUNK it is important
that the read buffer be large enough to handle any chunk being read. Ie.,
from Apache source code:
* 1. Call ap_setup_client_block() near the beginning of the request
* handler. This will set up all the necessary properties, and will
* return either OK, or an error code. If the latter, the module should
* return that error code. The second parameter selects the policy to
* apply if the request message indicates a body, and how a chunked
* transfer-coding should be interpreted. Choose one of
*
* REQUEST_NO_BODY Send 413 error if message has any body
* REQUEST_CHUNKED_ERROR Send 411 error if body without Content-Length
* REQUEST_CHUNKED_DECHUNK If chunked, remove the chunks for me.
* REQUEST_CHUNKED_PASS If chunked, pass the chunk headers with body.
*
* In order to use the last two options, the caller MUST provide a buffer
* large enough to hold a chunk-size line, including any extensions.
If chunked is used the mod_python source can't know what the buffer size
it would need to use is as that is going to be dependent on the higher level
application and what it is doing.
Thus a general purpose solution like mod_python can't be used and what is
really needed is a lower level Python interface to Apache API so you can control
things properly specific for your case. I have talked about the need for such
a better Python/Apache interface previously on the list but nothing public is
probably going to be available in that space in the near future.
BTW, I am basing this analysis on reading Apache code. If you are seeing
different results and things are working then I can only assume it is because
you are using read() with a size. If you use read() without a size, you will
probably not get any content. Even when specifying an argument you might
get problems as mod_python will always read up to that size and not simply
return what might be available at that time from a chunk.
Anyway, I'll have a think about the problem some more as will affect some other
code I am working on. :-)
Graham
> Index: src/requestobject.c
> ===================================================================
> --- src/requestobject.c (revision 510030)
> +++ src/requestobject.c (working copy)
> @@ -864,7 +864,7 @@
> if (! self->request_rec->read_length) {
>
> /* then do some initial setting up */
> - rc = ap_setup_client_block(self->request_rec, REQUEST_CHUNKED_ERROR);
> + rc = ap_setup_client_block(self->request_rec,
> REQUEST_CHUNKED_DECHUNK);
> if(rc != OK) {
> PyObject *val = PyInt_FromLong(rc);
> if (val == NULL)
> @@ -964,7 +964,7 @@
> if (! self->request_rec->read_length) {
>
> /* then do some initial setting up */
> - rc = ap_setup_client_block(self->request_rec, REQUEST_CHUNKED_ERROR);
> + rc = ap_setup_client_block(self->request_rec,
> REQUEST_CHUNKED_DECHUNK);
>
> if (rc != OK) {
> PyObject *val = PyInt_FromLong(rc);
