[issue27354] SSLContext.load_verify_locations cannot handle paths on Windows which cannot be encoded using mbcs

STINNER Victor added the comment: >> On Python 3.5, PyUnicode_FSConverter() uses MBCS, which is CP-1552 on your >> system. > > Will the behavior of Python 3.6 be different? Could you point me to relevant > notes or code? See the PEP 529. -- ___

[issue27354] SSLContext.load_verify_locations cannot handle paths on Windows which cannot be encoded using mbcs

Ilya Kulakov added the comment: > On Python 3.5, PyUnicode_FSConverter() uses MBCS, which is CP-1552 on your > system. Will the behavior of Python 3.6 be different? Could you point me to relevant notes or code? > If I understood correctly, it's possible to work around the issue by encoding

[issue27354] SSLContext.load_verify_locations cannot handle paths on Windows which cannot be encoded using mbcs

STINNER Victor added the comment: > I'm closing this issue was WONTFIX 3.5 If I understood correctly, it's possible to work around the issue by encoding the filename manually to utf-8. ssl_function(filename.encode('utf-8')) -- ___ Python tracker

[issue27354] SSLContext.load_verify_locations cannot handle paths on Windows which cannot be encoded using mbcs

Christian Heimes added the comment: Ilya, The exception "TypeError: cafile should be a valid filesystem path" is raised by Python, not by OpenSSL. On Python 3.5, PyUnicode_FSConverter() uses MBCS, which is CP-1552 on your system. CP-1552 cannot convert Arabic character set and fails to

[issue27354] SSLContext.load_verify_locations cannot handle paths on Windows which cannot be encoded using mbcs

Changes by Christian Heimes : -- stage: needs patch -> resolved status: open -> closed ___ Python tracker ___

[issue27354] SSLContext.load_verify_locations cannot handle paths on Windows which cannot be encoded using mbcs

Ilya Kulakov added the comment: Christian, If you have windows under your hand and can try an alike path, you should see the problem right away if it's still there. I think the original problem was unnecessary PyUnicode_FSConverter: it failed to encode string into mbcs, while OpenSSL did not

[issue27354] SSLContext.load_verify_locations cannot handle paths on Windows which cannot be encoded using mbcs

Christian Heimes added the comment: 3.6 uses PyUnicode_FSConverter() to convert the cafile and capath arguments, then passes PyBytes_AS_STRING() to OpenSSL. What needs to change to support non-ASCII chars on Windows? -- assignee: christian.heimes -> steve.dower versions: -Python 3.5

[issue27354] SSLContext.load_verify_locations cannot handle paths on Windows which cannot be encoded using mbcs

Steve Dower added the comment: I haven't tracked it down in 1.1, but in 1.0.2 OpenSSL handles ASCII, UTF-8 and mbcs/ANSI paths explicitly: https://github.com/openssl/openssl/blob/OpenSSL_1_0_2-stable/crypto/bio/bss_file.c#L138 So for 3.6 and later, if we're encoding the paths with fsencode(),

[issue27354] SSLContext.load_verify_locations cannot handle paths on Windows which cannot be encoded using mbcs

STINNER Victor added the comment: Steve Dower added the comment: > TESTFN_UNENCODABLE will be invalid utf8 now, as the name is chosen by > attempting to encode a list of names and using the first one to fail. Oh, maybe we should reject filenames not encodable to utf8? Filenames containg

[issue27354] SSLContext.load_verify_locations cannot handle paths on Windows which cannot be encoded using mbcs

Steve Dower added the comment: TESTFN_UNENCODABLE will be invalid utf8 now, as the name is chosen by attempting to encode a list of names and using the first one to fail. No code pages have emoji in them AFAIK, so a test with one of those would do. ISTR looking at this function though and

[issue27354] SSLContext.load_verify_locations cannot handle paths on Windows which cannot be encoded using mbcs

STINNER Victor added the comment: > Is this still an issue? I'm not 100% sure that SSL_CTX_load_verify_locations() accepts a path encoded to UTF-8 on Windows. To be 100% sure, it's "simple": try a filename not encoded to the ANSI code page on Windows. The best would be to have an unit test

[issue27354] SSLContext.load_verify_locations cannot handle paths on Windows which cannot be encoded using mbcs

Christian Heimes added the comment: Is this still an issue? -- assignee: -> christian.heimes components: +SSL nosy: +christian.heimes stage: -> needs patch type: -> behavior versions: +Python 3.6, Python 3.7 ___ Python tracker

[issue27354] SSLContext.load_verify_locations cannot handle paths on Windows which cannot be encoded using mbcs

Ilya Kulakov added the comment: I checked the source code of OpenSSL, specifically the `bss_file.c:file_fopen` function (https://github.com/openssl/openssl/blob/OpenSSL_1_0_2h/crypto/bio/bss_file.c#L118-L167). As you can see it support UTF-8 encoded strings under Windows. Python must follow

[issue27354] SSLContext.load_verify_locations cannot handle paths on Windows which cannot be encoded using mbcs

Ilya Kulakov added the comment: Viktor, I also came across this thread but it is rather old (we're using OpenSSL 1.0.2h). And it would only explain if _neither_ of my methods had worked. But as you can see, the last one (passing UTF-8 encoded bytes) works. --

[issue27354] SSLContext.load_verify_locations cannot handle paths on Windows which cannot be encoded using mbcs

STINNER Victor added the comment: The Python ssl module is a wrapper to the OpenSSL library. For this issue, we are talking about the function SSL_CTX_load_verify_locations(): https://www.openssl.org/docs/manmaster/ssl/SSL_CTX_load_verify_locations.html OpenSSL expects a byte string for CAfile

[issue27354] SSLContext.load_verify_locations cannot handle paths on Windows which cannot be encoded using mbcs

Ilya Kulakov added the comment: I believe this is a bug, because path suitable for os.path (or pathlib), should be equally suitable for load_verify_locations. -- ___ Python tracker

[issue27354] SSLContext.load_verify_locations cannot handle paths on Windows which cannot be encoded using mbcs

New submission from Ilya Kulakov: On Windows 8.1 x64 with Python 3.5.1 I was able to reproduce the issue by attempting to load a file at "C:\Users\غازي\AppData\Local\Temp\_غازي_70e5wbxo\cacert.pem". locale.getdefaultlocale() > ('en_US', 'cp1252') locale.getpreferredencoding()