[issue20916] ssl.enum_certificates() will not return all certificates trusted by Windows

[Chi Hsuan Yen]

Changes by Chi Hsuan Yen :


--
nosy: +Chi Hsuan Yen

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20916] ssl.enum_certificates() will not return all certificates trusted by Windows

[Christian Heimes]

Changes by Christian Heimes :


--
assignee:  -> christian.heimes
components: +SSL
nosy: +christian.heimes
versions: +Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20916] ssl.enum_certificates() will not return all certificates trusted by Windows

[Christian Heimes]

Changes by Christian Heimes :


--
nosy:  -christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20916] ssl.enum_certificates() will not return all certificates trusted by Windows

[ppperry]

Changes by ppperry :


--
components: +Windows
nosy: +paul.moore, tim.golden, zach.ware

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20916] ssl.enum_certificates() will not return all certificates trusted by Windows

[Christian Heimes]

Changes by Christian Heimes :


--
assignee: christian.heimes -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20916] ssl.enum_certificates() will not return all certificates trusted by Windows

[John Nagle]

John Nagle added the comment:

Amusingly, I'm getting this failure on verisign.com on Windows 7 with Python 
2.7.9:

HTTP error - [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed 
(_ssl.c:581)..)  The current Verisign root cert (Class 3 public) is, indeed, 
not in the Windows 7 cert store. Verisign has a newer root cert.

That error message ought to be improved. Tell the user which cert was rejected.

python.org, with a DigiCert certificate, works fine.

I'm going to use the Mozilla certificate store explicitly.

--
nosy: +nagle

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20916
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20916] ssl.enum_certificates() will not return all certificates trusted by Windows

[Antoine Pitrou]

Changes by Antoine Pitrou pit...@free.fr:


--
nosy: +steve.dower

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20916
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20916] ssl.enum_certificates() will not return all certificates trusted by Windows

[James Teh]

Changes by James Teh ja...@nvaccess.org:


--
nosy: +jteh

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20916
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20916] ssl.enum_certificates() will not return all certificates trusted by Windows

[Adam Goodman]

Adam Goodman added the comment:

What Martin said is correct, IMO.

The actual problem I'd like to correct is: If I - for example - create an 
HTTPSConnection with cert validation enabled, and set to use the default OS 
trust mechanism, then the validation process should trigger Windows' root CA 
download mechanism if necessary (i.e. rather than just rejecting the CA cert if 
it hasn't already been locally cached).

I don't expect that ssl.enum_certificates() ever will return all the 
certificates that are (implicitly, via the update mechanism) trusted by 
Windows; that's probably not feasible. I chose that as the title of the issue 
because it seemed to be the most concrete root-cause, but maybe that wasn't 
ideal.

(I'll file a separate issue for that traceback I ran into, if I get a chance to 
dig into it more)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20916
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20916] ssl.enum_certificates() will not return all certificates trusted by Windows

[Martin v . Löwis]

Martin v. Löwis added the comment:

I notice that this issue doesn't contain actual problem statement; Adam only 
reported what he did and what happened, but not what should have happened 
instead.

I personally don't think that the problem stated in the title 
(ssl.enum_certificates() will not return all certificates trusted by Windows) 
is a bug - this is correct behavior. It would be unreasonable to expect that 
enum_certificates() triggers a download of the entire MS root list, when 
Microsoft has established as a policy that download should be on demand, 
triggered by verification.

What I would agree *is* a bug is that the certificate verification fails; it 
should trigger the root download, as is platform convention (hopefully then 
also conforming to the group policy setting where you can disable root 
certificate download).

Please leave out unrelated bugs (e.g. a failure to fetch certain certificate 
attributes) from this bug report. Report them separately instead.

--
nosy: +loewis

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20916
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20916] ssl.enum_certificates() will not return all certificates trusted by Windows

[Antoine Pitrou]

Antoine Pitrou added the comment:

If this is a Microsoft decision, perhaps it should be documented, then.

--
nosy: +pitrou

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20916
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20916] ssl.enum_certificates() will not return all certificates trusted by Windows

[Adam Goodman]

New submission from Adam Goodman:

Starting with Vista, Microsoft began shipping only a very minimal set of root 
CA certificates with Windows. Microsoft does trust many other authorities, but 
for these, Windows relies on the Update Root Certificates feature: 
http://technet.microsoft.com/en-us/library/cc749331(WS.10).aspx

... if the application is presented with a certificate issued by a 
certification authority in a PKI that is not directly trusted, the Update Root 
Certificates feature (if it is not turned off) will contact the Windows Update 
Web site to see if Microsoft has added the certificate of the root CA to its 
list of trusted root certificates. If the CA has been added to the Microsoft 
list of trusted authorities, its certificate will automatically be added to the 
set of trusted root certificates on the user's computer.

Critically, this update mechanism is only invoked if you're using CryptoAPI 
functions to validate a specific chain; if you just ask Windows to enumerate 
the certificates it knows about, it won't pull anything down from Windows 
Update.

(Some concrete numbers: on a clean installation of Windows 8.1, running 
certmgr.msc shows 18 certificates listed in the Trusted Root Certification 
Authorities; by contrast, OS X comes with over 200 trusted roots).

To confirm this is an issue, I did the following:

1. Start with a clean Windows 8.1 VM image (I used the one from from 
http://www.modern.ie/en-us/virtualization-tools#downloads). It is critical that 
the image be completely clean - i.e. you have never visited https://python.org 
in any web browser, etc.

2. Install Python 3.4.0 RC 3 (32-bit)

3. Run the attached script (which just does a request to https://python.org/ 
with cert validation enabled). It prints out 14 CA certificate subjects, then 
fails with ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify 
failed (_ssl.c:598). At the time I'm reporting this issue, python.org uses a 
certificate that chains back to the DigiCert High Assurance EV Root CA, which 
is not included in clean installations of Windows.

4. Browse to https://python.org in Internet Explorer

5. Run the attached script again. Now it prints out 17 CA certificate subjects, 
and the HTTPS request completes successfully.

Right now, the only idea I have for resolving this would require significant 
architectural changes - instead of pulling the certificates from Windows into 
an OpenSSL context, hook the OpenSSL verify callbacks to retrieve the leaf and 
intermediate certificates provided by the server, then use CryptoAPI functions 
(probably CertGetCertificateChain?) to have Windows perform the actual chain 
validation.

--
components: Extension Modules
messages: 213452
nosy: Adam.Goodman, christian.heimes
priority: normal
severity: normal
status: open
title: ssl.enum_certificates() will not return all certificates trusted by 
Windows
type: behavior
versions: Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20916
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20916] ssl.enum_certificates() will not return all certificates trusted by Windows

[Adam Goodman]

Changes by Adam Goodman akg...@duosecurity.com:


Added file: http://bugs.python.org/file34405/win_ca_test.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20916
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20916] ssl.enum_certificates() will not return all certificates trusted by Windows

[Christian Heimes]

Christian Heimes added the comment:

Thanks for you tests!

Yes, I was aware of the situation in general. Personally I think it is an 
unfortunate decision of Microsoft to download root CA certs on demand. When I 
developed the feature I only experimented with a fresh but fully patched VM of 
Windows 7 Professional. The VM had more root CAs installed so I didn't think 
it's going to bite the majority users for common sites. In retrospective I 
*might* have trigger cert downloads accidentally...

I also tried to implement a OpenSSL's verify hook but my code was far from 
ready for 3.4 beta. I'll have to implement a proper solution for Python 3.5. 
The situation on OSX and Windows isn't perfect.

KB931125 lists a way to trigger a full download of all known root certs. Do you 
still have a fresh VM around? I won't have time to test the tool from KB931125 
before 3.4.0 is released.

--
assignee:  - christian.heimes
stage:  - needs patch
versions: +Python 3.5 -Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20916
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20916] ssl.enum_certificates() will not return all certificates trusted by Windows

[Adam Goodman]

Adam Goodman added the comment:

I just tried installing the root certificate update from KB931125 on a clean 
VM. Now I have 369 trusted root CAs, according to certmgr.msc. (I imagine it 
would be unreasonable to expect all windows python users to do this, though...)

The https request to python.org does succeed - but the code I'd thrown together 
to print out the SSLContext state breaks somehow:

 Traceback (most recent call last):
   File C:/Users/IEUser/Desktop/win_ssl_test.py, line 8, in module
 ca_certs = context.get_ca_certs()
 ssl.SSLError: unknown error (_ssl.c:636)

Peeking at _ssl.c, it seems like it might be failing to decode some attributes 
on one (or more) of the certs...?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20916
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com