In the most recent versions of cURL, the Content-Length header is now
parsed as an integer instead of a floating point number, for good
reasons. When cURL isn't able to parse the Content-Length value into a
representable integer, it seems that it proceeds as if there was no
Content-Length header. The intent is noble, because cURL is trying to be
robust and keep going even when the response is very, very large.
However, there are some problems with this approach:
Firstly, it's not clear how cURL's going to know when the response is
over. For a typical response with a Content-Length header, if it's not a
chunked response entity, and "Connection: close" semantics aren't in
play, then it seems like there will be no way for cURL to know when the
response is finished. How will it not include the any subsequent
(pipelined?) response headers in the entity? How will it know when to
tell the application that the request is complete?
Secondly, it seems like the behaviour is different when the
Content-Length can be represented but is negative. In this case cURL
seems to close the connection and fail the request. Presumably, this is
for reasons similar to the first case, since there can be no way to know
when such a request is finished. But, of course, a negative
Content-Length is meaningless anyway. (It looks like the intended
behaviour is to close the connection when the Content-Length is
negative, but it looks like negative values may trigger the
"unrepresentable" behaviour instead since the ASCII-to-integer parser
doesn't seem to handle negative numbers.)
Thirdly, when a maximum file size is set with the
CURLOPT_MAXFILESIZE_LARGE option, but then cURL can't parse a
Content-Length, it seems safe to assume that either the Content-Length
is garbage, so the request needs to fail, or that the Content-Length is
too large for cURL to represent, which means it must be larger than the
maximum file size, so the request needs to fail. In the very least, it
would seem that at least when this option is set, cURL should always
close the connection and fail the request.
So my question is, why doesn't cURL close the connection and fail the
request in all of these cases?
Also, BTW, it's interesting that cURL does not enforce the maximum file
size when there is no Content-Length. This is surprising (it was a
surprise to me when I first encountered it), and it leaves the
application with the task of counting the bytes returning in response
data callbacks so it can also enforce the maximum file size for the
unpredictable cases where the server didn't advertise the response size.
It would seem natural for cURL to do this itself when a maximum file
size is set on the request.
--
Brad Spencer
-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
