[issue15769] urllib.request.urlopen with cafile or capath set overrides any previous Handlers

[Martin Panter]

Martin Panter added the comment:

Nothing has been fixed; I don’t see any evidence that this is “out of date”. 
Here is a more complete test:

import urllib.request
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor())
urllib.request.install_opener(opener)
request = 'https://httpbin.org/cookies/set?name=value'
# Download certifiate chain for https://httpbin.org/ into cacert.pem (I used 
Firefox)
sock = urllib.request.urlopen(request, cafile = 'cacert.pem')
sock.read()  # b'{\n  "cookies": {}\n}\n' (No cookies!)
sock.close()
sock = urllib.request.urlopen(request)  # Default SSL settings
sock.read()  # b'{\n  "cookies": {\n"name": "value"\n  }\n}\n'

I’m not comfortable with applying a patch that “doesn’t mangle [the global 
state] too much”. Anyway, this is the same as Issue 18543, which has slightly 
more recent discussion.

--
nosy: +martin.panter
resolution: out of date -> duplicate
status: pending -> closed
superseder:  -> urllib.parse.urlopen shouldn't ignore installed opener when 
called with any SSL argument

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15769] urllib.request.urlopen with cafile or capath set overrides any previous Handlers

[Christian Heimes]

Christian Heimes added the comment:

The ticket hasn't moved in about four years. Does the issue still persist?

--
nosy: +christian.heimes
resolution:  -> out of date
stage: needs patch -> patch review
status: open -> pending
versions: +Python 3.6, Python 3.7 -Python 3.2, Python 3.3, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15769] urllib.request.urlopen with cafile or capath set overrides any previous Handlers

[Senthil Kumaran]

Senthil Kumaran added the comment:

I could verify this bug and also looks like a tricky one. Because when we are 
sending the cacert (the second time), we create a new HTTPSHandler and then 
build the opener again using that handler. 

I thought, we can use the existing opener object itself like this - 

-opener = build_opener(https_handler)
+if _opener is None:
+opener = build_opener(https_handler)
+else:
+opener = _opener
+opener.add_handler(https_handler)


But even then the problem is, the existing default HTTPSHandler will be invoked 
before the newly added one, as add_handler does a bisect.insert to add new 
handlers.

Thinking what's the best way to insert this new updated HTTPS handler in front 
of the existing ones (without resorting to any hacks).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15769
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15769] urllib.request.urlopen with cafile or capath set overrides any previous Handlers

[Brian Turek]

Brian Turek added the comment:

I was actually going to come up with a patch that does the same thing orsenthil 
mentioned until I too was stymied by the use of bisect.insort

Does anyone know why bisect.insort was used instead of just list.append?  I 
don't see an obvious reason so I think just having add_handler insert new 
handlers at the beginning of all the respective lists would be an easy fix.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15769
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15769] urllib.request.urlopen with cafile or capath set overrides any previous Handlers

[R. David Murray]

R. David Murray added the comment:

It has been a while since I looked at the code, but if I remember correctly 
there is a (somewhat non-obvious) mechanism for assigning priority to handlers, 
so that you can control the order of application.  If I'm right the insort is 
about that priority, and it also ought to be possible to use that priority to 
fix the problem.  But I remember whatever it was I was looking at as being a 
bit hard to understand, so I could be way off base (or thinking of some other 
part of the code base).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15769
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15769] urllib.request.urlopen with cafile or capath set overrides any previous Handlers

[Antoine Pitrou]

Antoine Pitrou added the comment:

-opener = build_opener(https_handler)
+if _opener is None:
+opener = build_opener(https_handler)
+else:
+opener = _opener
+opener.add_handler(https_handler)

Well, isn't it a bad idea to mutate the global opener?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15769
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15769] urllib.request.urlopen with cafile or capath set overrides any previous Handlers

[Brian Turek]

Brian Turek added the comment:

So I'm not saying the attached patch is the *best* solution but it doesn't 
mangle the existing urllib.request._opener too much.

--
keywords: +patch
Added file: http://bugs.python.org/file26977/request.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15769
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15769] urllib.request.urlopen with cafile or capath set overrides any previous Handlers

[Brian Turek]

New submission from Brian Turek:

Using the code snippet below:

cj = http.cookiejar.CookieJar()
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor())
urllib.request.install_opener(opener)
request = urllib.request.Request(url, data, headers) # url is https://something
sock = urllib.request.urlopen(request, cafile = 'cacert.pem')

One would expect to establish a verified HTTPS connection to the host and the 
cookies stored in the cookie jar.  Unfortunately urllib.request.urlopen, when 
called with cafile or capath set, calls urllib.request.build_opener without 
HTTPCookieProcessor included in the chain.  This results in never being able to 
support cookies in a verified HTTPS connection.

--
components: Library (Lib)
messages: 168915
nosy: caligatio
priority: normal
severity: normal
status: open
title: urllib.request.urlopen with cafile or capath set overrides any previous 
Handlers
type: behavior
versions: Python 3.2, Python 3.3, Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15769
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15769] urllib.request.urlopen with cafile or capath set overrides any previous Handlers

[R. David Murray]

R. David Murray added the comment:

I thought that was an issue when I was looking at the code the other day, but I 
didn't get around to testing it.  Thanks for the report.

--
nosy: +orsenthil, pitrou, r.david.murray
stage:  - needs patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15769
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com