[issue25919] http.client PUT method ignores error responses sent immediatly after headers

[R. David Murray]

Changes by R. David Murray :


--
nosy:  -r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25919] http.client PUT method ignores error responses sent immediatly after headers

[Martin Panter]

Martin Panter added the comment:

As I see it, if the client receives an early response (when 100-continue is not 
used) but needs to make more requests, the only choices are:

* Complete sending the request if possible, after which the connection may 
still be usable for a second request. But I’m not sure how many servers would 
support this; for your Microsoft server this would not be useful.

* Abort the connection and start a new one.

Perhaps the race condition would be more obvious if you sent the upload as a 
single 300 MiB bytes() object rather than a file. Then the sendall() call will 
be a 300 MiB chunk rather than 8 KiB. If the response is received after 
sendall() starts, I expect you will see the original two minute delay again. If 
this were plain TCP, you could replace sendall() with fine-grained select() and 
send() calls in a loop.

Here is a demonstration of the SSL renegotiation issue. I used separate client 
and server terminal windows. I don’t know if there is a way to force 
renegotiation purely in Python’s SSL module, so I used an external Open SSL 
server.

1: In the server window, start an SSL server on port 4433:
$ openssl s_server -cert Lib/test/keycert.pem 

2: In the client window, start a request in the Python interpreter. It will 
pause to read stdin:
>>> import http.client, ssl, sys
>>> context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
>>> context.load_verify_locations("/usr/lib/python3.5/test/keycert.pem")
>>> conn = http.client.HTTPSConnection('localhost', 4433, context=context)
>>> conn.request("PUT", "/", headers={"Content-Length": "4"}, 
>>> body=sys.stdin.buffer)

3: In the server, the request header is received. Type lowercase “r” to force a 
renegotiation:
Secure Renegotiation IS supported
PUT / HTTP/1.1
Host: localhost:4433
Accept-Encoding: identity
Content-Length: 8

r
SSL_do_handshake -> 1

4: In Python, type in upload data and repeat Ctrl+D to signal EOF until the 
request() call stops reading from stdin and you get the Python prompt back:
abc  <== 3 letters + newline + Ctrl+D
>>> 

5: Notice that the no body has been uploaded to the server.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25919] http.client PUT method ignores error responses sent immediatly after headers

[Martin Panter]

Martin Panter added the comment:

Curl uses “Expect: 100-continue”. This is the relevant output from it:

$ curl -v -T 300mb --no-fail 
https://api.onedrive.com/v1.0/drives/me/root:/test.file:/content
[. . .]
> PUT /v1.0/drives/me/root:/test.file:/content HTTP/1.1
> Host: api.onedrive.com
> Content-Range: bytes 0-314572799/314572800
> User-Agent: curl/7.43.0
> Accept: */*
> Connection: TE
> TE: gzip
> Content-Length: 314572800
> Expect: 100-continue
> 
< HTTP/1.1 413 Request Entity Too Large
< Content-Length: 0
< Server: Microsoft-HTTPAPI/2.0
< P3P: CP="BUS CUR CONo FIN IVDo ONL OUR PHY SAMo TELo"
< X-MSNSERVER: DM2302PAP179
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< X-ThrowSite: 7c11.e25d
< Date: Sun, 03 Jan 2016 20:04:27 GMT
* HTTP error before end of send, stop sending
< 
* Closing connection 0

See Issue 1346874 about support for 100-continue mode.

When I tried your test code on Linux, it hung for two minutes, and then raised 
ConnectionResetError. I guess the server refused to receive data, and then 
timed the connection out. The patch should handle this in most cases, but I 
think there could still a race between select() returning an empty rlist, and 
then the server sending a request and causing the upload to hang. Also, there 
is nothing to protect from sendall() partially uploading a chunk, and then 
hanging.

It should be more robust to check for both send and receive sides of the socket 
with select(), and only use send(), not sendall(), to avoid blocking.

I would consider this a new feature, so I don’t think it should go into 3.5. It 
would be good to add documentation and test cases as well.

Wiktor’s problem is similar to the situations in Issue 5542. There, the 
original poster seemed happy to wait for a broken pipe error, and then inspect 
a 401 Unauthorized response. This apparently was made to work with plain HTTP 
over TCP, but not HTTPS.

I wonder if having request() silently return is the best API design. Is there a 
possibility that the caller would want to distinguish a successful upload from 
an aborted one? I can’t think of one, but it seems plausible.

Another related enhancement I would like is the ability to detect an error 
response (e.g. 408 Request Timeout) or dropped connection _before_ even sending 
a request. This is useful with persistent connections; see items 1 and 2 in 
.

--
nosy: +martin.panter
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25919] http.client PUT method ignores error responses sent immediatly after headers

[Martin Panter]

Martin Panter added the comment:

For non-blocking SSL sockets, the Python documentation suggests the results of 
select() do not directly indicate if data can be sent or received: 
. I guess this 
would also apply to blocking SSL sockets.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25919] http.client PUT method ignores error responses sent immediatly after headers

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
title: htp.client PUT method ignores error responses sent immediatly after 
headers -> http.client PUT method ignores error responses sent immediatly after 
headers

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com