[issue23327] zipimport to import from non-ascii pathname on Windows

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Thank you for your patch Swapneel, but this issue was fixed in issue25758.

--
nosy: +serhiy.storchaka
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23327] zipimport to import from non-ascii pathname on Windows

[Eric Snow]

Changes by Eric Snow ericsnowcurren...@gmail.com:


--
nosy: +eric.snow, superluser

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23327
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23327] zipimport to import from non-ascii pathname on Windows

[Eric Snow]

Changes by Eric Snow ericsnowcurren...@gmail.com:


--
type: crash - behavior
versions: +Python 3.5, Python 3.6

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23327
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23327] zipimport to import from non-ascii pathname on Windows

[Swapneel Ambre]

Swapneel Ambre added the comment:

Attaching a combined patch. I updated testUnencodable testcase from 
test_zipimport.py. Verified that without my fix, the testcase fails and it 
passes with my fix.

--
Added file: http://bugs.python.org/file37883/zipimport_fix_withtest.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23327
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23327] zipimport to import from non-ascii pathname on Windows

[STINNER Victor]

STINNER Victor added the comment:

 Please see zipimport_fix.patch for the possible fix.

The solution looks good. Can you please try to convert  zipimport_test.py to a 
patch for test_zipimport.py and combine it with   zipimport_fix.patch  to 
create a complete patch?

You should also sign the contributor agreement:
https://www.python.org/psf/contrib/contrib-form/

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23327
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23327] zipimport to import from non-ascii pathname on Windows

[STINNER Victor]

STINNER Victor added the comment:

 I do agree that it is a corner case scenario and using ASCII names is a 
 better option :-)

Since the patch is short, I see no problem to fix this issue.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23327
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23327] zipimport to import from non-ascii pathname on Windows

[Swapneel Ambre]

New submission from Swapneel Ambre:

On Windows, using zipimport module APIs like get_filename on a file with 
non-ascii characters in the full path fails with 

UnicodeEncodeError: 'mbcs' codec can't encode characters in position 0--1: 
invalid character ( Full output attached in errorlog.txt ).

The issue is that Modules/zipimport.c has a function compile_source which tries 
to run PyUnicode_EncodeFSDefault on the pathname. On Windows, the default 
encoding is 'mbcs' which cannot handle unicode characters.

This has already been fixed in the import machinery on python 3 ( see issue 
http://bugs.python.org/issue13758, http://bugs.python.org/issue11619). The 
solution is to pass the pathname as Unicode directly to the compiler.

--
components: Unicode, Windows
files: errorlog.txt
messages: 234786
nosy: amswap, ezio.melotti, haypo, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: zipimport to import from non-ascii pathname on Windows
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file37872/errorlog.txt

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23327
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23327] zipimport to import from non-ascii pathname on Windows

[Swapneel Ambre]

Swapneel Ambre added the comment:

I have tried to fix this by calling Py_CompileStringObject instead of 
Py_CompileString , thus avoiding the need to Encode the pathname. Please see 
zipimport_fix.patch for the possible fix.

--
keywords: +patch
Added file: http://bugs.python.org/file37874/zipimport_fix.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23327
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23327] zipimport to import from non-ascii pathname on Windows

[Swapneel Ambre]

Swapneel Ambre added the comment:

I am attaching the test script I have used to reproduce the issue.

--
Added file: http://bugs.python.org/file37873/zipimport_test.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23327
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23327] zipimport to import from non-ascii pathname on Windows

[STINNER Victor]

STINNER Victor added the comment:

I don't understand the issue: does it only concern the name of the ZIP file? Or 
also paths inside the ZIP?

In both cases, the workaround is to use only ASCII names.

I spent a lot of times on supporting any Unicode name, everyone in Python. I 
didn't expect that people have so different and crazy use cases :-)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23327
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23327] zipimport to import from non-ascii pathname on Windows

[Swapneel Ambre]

Swapneel Ambre added the comment:

Sorry I was not very clear about the use case. 

The name of the zipfile or any parent directory name could contain non-ascii 
characters. Consider a use case where you want to ship some product with third 
party module shipped as an egg file (say example.egg) along with your product. 
You don't have control over where the product files gets installed. Someone 
could install the product files under say C:\的\product_name. So both your 
product (exe or python files) and the egg files are installed under a path with 
non-ascii characters in it. Any import statements trying to import modules from 
the egg file will fail with UnicodeEncodeError as zipimport will try to use 
PyUnicode_EncodeFSDefault with 'mbcs' encoding on Windows. 

I hope the use case is clearer now. I do agree that it is a corner case 
scenario and using ASCII names is a better option :-)

I will create a complete patch and sign contributor agreement.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23327
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com