[issue24964] Add tunnel CONNECT response headers to httplib / http.client

[Alexey Namyotkin]

Alexey Namyotkin  added the comment:

Thanks, Terry. I signed it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24964] Add tunnel CONNECT response headers to httplib / http.client

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Alexey, to repeat what I said to Thomas above: please sign a contributor 
agreement for your patches to be considered.
https://www.python.org/psf/contrib/
https://www.python.org/psf/contrib/contrib-form/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24964] Add tunnel CONNECT response headers to httplib / http.client

[Terry J. Reedy]

Change by Terry J. Reedy :


--
versions: +Python 3.11 -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24964] Add tunnel CONNECT response headers to httplib / http.client

[Alexey Namyotkin]

Change by Alexey Namyotkin :


--
pull_requests: +24786
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/26152

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24964] Add tunnel CONNECT response headers to httplib / http.client

[Alexey]

Change by Alexey :


--
nosy: +Namyotkin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24964] Add tunnel CONNECT response headers to httplib / http.client

[Martin Panter]

Martin Panter added the comment:

1) The real problem is when _tunnel() internally calls getresponse(), it 
notices the connection cannot be reused for another request, and closes the 
socket object. Perhaps I should rethink my logic; maybe move sock and detach() 
to HTTPResponse.

2) With some rough experimentation, passing tunnel through the HTTPConnection 
(plain text HTTP) constructor seems to work for me. However if you meant 
HTTPSConnection (over TLS) instead, you will probably need to manually do the 
wrap_socket() step. Maybe that’s why your connection is being dropped.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24964] Add tunnel CONNECT response headers to httplib / http.client

[Thomas Belhalfaoui]

Thomas Belhalfaoui added the comment:

Martin, I went through your patch and made some simple tests, and I have a 
couple of questions.

1) When I run the following code, I get a "Bad file descriptor" :

conn = httplib.HTTPConnection("uk.proxymesh.com", 31280)
conn.set_tunnel("www.google.com", 80)
conn.request("GET", "/")
resp = conn.getresponse()
print(resp.read())

So I tweaked the "getresponse" function so that it does not call "self.close()" 
(i.e. the connection stays open after the CONNECT request) in that case, and it 
seems to works fine.

2) I added "self.sock, _ = tunnel" in HTTPConnection constructor, to try your 
use case, but I get "http.client.RemoteDisconnected: Remote end closed 
connection without response".

Do you think it makes sense or am I missing something ?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24964] Add tunnel CONNECT response headers to httplib / http.client

[Martin Panter]

Martin Panter added the comment:

This is the patch I had in mind. It looks like it only implements the detach() 
method, so we would still need to add support for passing in the tunnel details 
to the HTTPSConnection constructor.

This patch would allow doing stuff at a lower level than the existing tunnel 
functionality. The patch includes a test case for getting the proxy’s response 
header fields, and another test case illustrating how a plain text HTTP 2 
upgrade could work.

--
keywords: +patch
stage:  -> needs patch
Added file: http://bugs.python.org/file40371/http-detach.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24964] Add tunnel CONNECT response headers to httplib / http.client

[Martin Panter]

Changes by Martin Panter :


--
stage: test needed -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24964] Add tunnel CONNECT response headers to httplib / http.client

[Thomas Belhalfaoui]

Thomas Belhalfaoui added the comment:

Terry: Thanks for the form, I just filled it.

Martin: Thanks for sending your patch. I will dive into it, and try to figure 
out how to add support for passing in the tunnel details to the HTTPSConnection 
constructor.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24964] Add tunnel CONNECT response headers to httplib / http.client

[Terry J. Reedy]

Terry J. Reedy added the comment:

Thomas, please sign a contributor agreement for your patches to be considered.
https://www.python.org/psf/contrib/
https://www.python.org/psf/contrib/contrib-form/

--
nosy: +terry.reedy
stage:  -> test needed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24964] Add tunnel CONNECT response headers to httplib / http.client

[Thomas]

Thomas added the comment:

Martin: Thanks for your quick answer (and sorry for sending the whole file) !
I think it is indeed a good idea to detach the proxy connection and treat it as 
any other connection, as you did in your patch. It would be great if you would 
be able to dig it up !

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24964] Add tunnel CONNECT response headers to httplib / http.client

[Martin Panter]

Martin Panter added the comment:

Such a change would involve adding a new API, so should go into a new version 
of Python.

Thomas: a diff rather than a full copy of the changed file would be more 
convenient. Also, if this gets accepted, test cases and documentation would be 
needed.

It is also useful to get the header of an unsuccessful CONNECT response. For 
example, see Issue 7291, where the Proxy-Authenticate header of the proxy’s 407 
response needs to be accessible. In that issue, I started working on a patch 
tht may also be useful here. From memory, usage would be a bit like this:

proxy_conn = HTTPConnection(proxy)
proxy_conn.request(CONNECT, website:443)
proxy_resp = proxy_conn.getresponse()
if proxy_resp.status == PROXY_AUTHENTICATION_REQUIRED:
# Handle proxy_resp.msg[Proxy-Authenticate]
...
# Handle proxy_resp.msg[X-ProxyMesh-IP]
...
tunnel = proxy_conn.detach()  # Returns socket and any buffered data
website_conn = HTTPSConnection(website, tunnel=tunnel)
website_conn.request(GET, /)
...
website_conn.close()

Thomas, let me know if this would be useful for you, and I can try and dig up 
my patch.

--
nosy: +martin.panter
versions:  -Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24964
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com