[issue16218] Python launcher does not support unicode characters

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
stage: commit review -> resolved

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[Andrew Svetlov]

Andrew Svetlov added the comment:

Thanks!

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[Roundup Robot]

Roundup Robot added the comment:

New changeset 41658a4fb3cc by Victor Stinner in branch '3.2':
Issue #16218, #16414, #16444: Backport FS_NONASCII, TESTFN_UNDECODABLE,
http://hg.python.org/cpython/rev/41658a4fb3cc

New changeset 4d40c1ce8566 by Victor Stinner in branch '3.3':
(Merge 3.2) Issue #16218, #16414, #16444: Backport FS_NONASCII,
http://hg.python.org/cpython/rev/4d40c1ce8566

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[STINNER Victor]

STINNER Victor added the comment:

 I assign the issue to you than. Is it ok?

Sure.

I backported all changesets related to this issue to Python 3.2 and 3.3. So I 
can finally close this issue.

--
assignee: haypo - 
resolution:  - fixed
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[STINNER Victor]

STINNER Victor added the comment:

Status of the different issues:

#16416, Mac OS X: 3.2, 3.3, 3.4
#16455, FreeBSD and Solaris: 3.4
#16218, set_main_loader: 3.3, 3.4
#16218, test_cmd_line_script: 3.4 (3.3 has an old copy of the test)
#16414, add support.TESTFN_NONASCII: 3.4
#16444, use support.TESTFN_NONASCII: 3.4

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[Andrew Svetlov]

Andrew Svetlov added the comment:

Victor, are you done all work for the issue?
Can it be closed?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[STINNER Victor]

STINNER Victor added the comment:

The issue is now fixed on all platforms for Python 3.4. Please keep the
issue open until all changes are backported to Python 3.3 or even Python
3.2.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[Andrew Svetlov]

Andrew Svetlov added the comment:

I assign the issue to you than. Is it ok?

--
assignee: asvetlov - haypo

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[STINNER Victor]

STINNER Victor added the comment:

 New changeset c25635b137cc by Victor Stinner in branch 'default':
 Issue #16455: On FreeBSD and Solaris, if the locale is C, the
 http://hg.python.org/cpython/rev/c25635b137cc

This changeset should fix this issue on FreeBSD and Solaris: see the issue 
#16455 for more information.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[koobs]

koobs added the comment:

Back to green for all branches on FreeBSD, thank you Victor

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[Stefan Krah]

Stefan Krah added the comment:

The Mountain Lion bots still fail. :)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[STINNER Victor]

STINNER Victor added the comment:

 Back to green for all branches on FreeBSD, thank you Victor

FreeBSD buildbots are green because I disabled the test on undecodable bytes! 
See issue #16455 which proposes a fix for FreeBSD and OpenIndiana.

 The Mountain Lion bots still fail. :)

Yeah I know, see the issue #16416 which has a patch. I plan to commit it to 
3.4, wait for buildbots, and then backport to 3.3.

--

Python 3.3 handles non-ASCII almost everywhere. Python 3.4 will probably handle 
non-ASCII everywhere.

Handling *undecodable* bytes is really hard. We cannot use the same code for 
UNIX and Windows. If we store data as bytes, it solves the issue, but we don't 
support any Unicode character on Windows anymore. If we store data as Unicode, 
it's the opposite (ok for Windows, decode error on UNIX).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[Roundup Robot]

Roundup Robot added the comment:

New changeset 6017f09ead53 by Victor Stinner in branch '3.3':
Issue #16218, #16444: Backport improvment on tests for non-ASCII characters
http://hg.python.org/cpython/rev/6017f09ead53

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[STINNER Victor]

STINNER Victor added the comment:

 Non-ASCII filenames were already supported with UTF-8 locale encoding.

 Test the example in msg173373.  It fails without fix.

Oh, I didn't understand that, sorry. I created #16444 to test also UTF-8 locale 
encoding with undecodable filenames (undecodable from UTF-8 in *strict* mode, 
not by os.fsencode() which uses surrogateescape).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[Roundup Robot]

Roundup Robot added the comment:

New changeset 6b8a8bc6ba9c by Victor Stinner in branch 'default':
Issue #16444, #16218: Use TESTFN_UNDECODABLE on UNIX
http://hg.python.org/cpython/rev/6b8a8bc6ba9c

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[STINNER Victor]

STINNER Victor added the comment:

If there's not another revision of the test patch in the wings, can 
56df0d4f0011 also be applied to 3.3, as tests are still failing on at least 
koobs-freebsd and koobs-freebsd-clang buildbots.

I just applied the patch of the issue #16444. I will check 3.4 buildbots, and 
then backport to older Python versions (at least 3.3).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[Antoine Pitrou]

Antoine Pitrou added the comment:

 If there's not another revision of the test patch in the wings, can
 56df0d4f0011 also be applied to 3.3, as tests are still failing on at
 least koobs-freebsd and koobs-freebsd-clang buildbots.

Let me insist on what koobs just said. The Windows buildbots are still 
broken on 3.3, so this either needs fixing or reverting.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[Jesús Cea Avión]

Jesús Cea Avión added the comment:

OpenIndiana 3.3 and 3.x buildbot broken too for a week.

I suggest to revert this patch and use the custom buildbots to debug it 
before committing again. A week, and counting, it is about time.

Feel free to hammer my OpenIndiana custom buildbots.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[koobs]

koobs added the comment:

If there's not another revision of the test patch in the wings, can 
56df0d4f0011 also be applied to 3.3, as tests are still failing on at least 
koobs-freebsd and koobs-freebsd-clang buildbots.

--
nosy: +koobs

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I think here should be used something like CommonTest.test_nonascii_abspath() 
in Lib/test/test_genericpath.py.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[STINNER Victor]

STINNER Victor added the comment:

 It tests nothing on utf-8 locale (test passed even when bug is not fixed).

The issue is about Windows and UTF-8 is never used as filesystem encoding on 
Windows.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[STINNER Victor]

STINNER Victor added the comment:

The test is still failing on Mac OS X:




==
FAIL: test_non_ascii (test.test_cmd_line_script.CmdLineTest)
--
Traceback (most recent call last):
  File 
/Volumes/bay2/buildslave/cpython/3.x.snakebite-mountainlion-amd64/build/Lib/test/test_cmd_line_script.py,
 line 380, in test_non_ascii
rc, stdout, stderr = assert_python_ok(script_name)
  File 
/Volumes/bay2/buildslave/cpython/3.x.snakebite-mountainlion-amd64/build/Lib/test/script_helper.py,
 line 54, in assert_python_ok
return _assert_python(True, *args, **env_vars)
  File 
/Volumes/bay2/buildslave/cpython/3.x.snakebite-mountainlion-amd64/build/Lib/test/script_helper.py,
 line 46, in _assert_python
stderr follows:\n%s % (rc, err.decode('ascii', 'ignore')))
AssertionError: Process return code is 2, stderr follows:
/Volumes/bay2/buildslave/cpython/3.x.snakebite-mountainlion-amd64/build/python.exe:
 can't open file './@test_63568_tmp.py': [Errno 2] No such file or directory

http://buildbot.python.org/all/builders/AMD64%20Mountain%20Lion%20%5BSB%5D%203.x/builds/410/steps/test/logs/stdio

--

If I remember correctly, the command line is always decoded from 
UTF-8/surrogateescape on Mac OS X. That's why we have the function 
_Py_DecodeUTF8_surrogateescape() (for bootstrap reasons).

Such example should not work if the locale encoding is not UTF-8 on Mac OS X:
---
arg = _Py_DecodeUTF8_surrogateescape(...);
filename = _Py_wchar2char(arg);
fp = fopen(filename, r);
---

run_file() uses a different strategy:

unicode = PyUnicode_FromWideChar(filename, wcslen(filename));
if (unicode != NULL) {
bytes = PyUnicode_EncodeFSDefault(unicode);
Py_DECREF(unicode);
}
if (bytes != NULL)
filename_str = PyBytes_AsString(bytes);
else {
PyErr_Clear();
filename_str = encoding error;
}

run_file() looks to be right. Py_Main() should use similar code.

We should probably not encode and then decode the filename in each function, 
but this is another problem.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

 The issue is about Windows and UTF-8 is never used as filesystem encoding
 on Windows.

The issue exists on Linux as I reported in msg173373.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[STINNER Victor]

STINNER Victor added the comment:

It skipped on locales which does not support £ (cp1006, cp1250, cp1251, 
cp737, cp852, cp855, cp866, cp874, cp949, euc_kr, gb2312, gbk, hz, iso2022_kr, 
iso8859_10, iso8859_11, iso8859_16, iso8859_2, iso8859_4, iso8859_5, iso8859_6, 
johab, koi8_r, koi8_u, mac_arabic, mac_farsi, ptcp154, tis_620).  But the bug 
is actual on such locales.

This issue is not specific to this test: I create the issue #16414 to improve 
the situation.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[STINNER Victor]

STINNER Victor added the comment:

 It tests nothing on utf-8 locale (test passed even when bug is not fixed).
 The issue is about Windows and UTF-8 is never used as filesystem encoding on 
 Windows.
 The issue exists on Linux as I reported in msg173373.

I don't understand your problem. Non-ASCII filenames were already supported 
with UTF-8 locale encoding. The new test checks that there is no regression 
with UTF-8 locale encoding. The test pass without the fix because it was not 
supported.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

 Non-ASCII filenames were already supported with UTF-8 locale encoding.

Test the example in msg173373.  It fails without fix.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[STINNER Victor]

STINNER Victor added the comment:

I created the issue #16416 to fix the Mac OS X case.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[STINNER Victor]

STINNER Victor added the comment:

test_cmd_line_script.test_non_utf8() is failing on Mac OS X since the changeset 
95d1adf144ee.

==
FAIL: test_non_utf8 (test.test_cmd_line_script.CmdLineTest)
--
Traceback (most recent call last):
  File 
/Volumes/bay2/buildslave/cpython/3.x.snakebite-mountainlion-amd64/build/Lib/test/test_cmd_line_script.py,
 line 381, in test_non_utf8
rc, out, _ = assert_python_ok(*run_args)
  File 
/Volumes/bay2/buildslave/cpython/3.x.snakebite-mountainlion-amd64/build/Lib/test/script_helper.py,
 line 54, in assert_python_ok
return _assert_python(True, *args, **env_vars)
  File 
/Volumes/bay2/buildslave/cpython/3.x.snakebite-mountainlion-amd64/build/Lib/test/script_helper.py,
 line 46, in _assert_python
stderr follows:\n%s % (rc, err.decode('ascii', 'ignore')))
AssertionError: Process return code is 2, stderr follows:
/Volumes/bay2/buildslave/cpython/3.x.snakebite-mountainlion-amd64/build/python.exe:
 can't open file 'unprintable file name': [Errno 92] Illegal byte sequence

http://buildbot.python.org/all/builders/AMD64%20Mountain%20Lion%20%5BSB%5D%203.x/builds/404/steps/test/logs/stdio

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[STINNER Victor]

STINNER Victor added the comment:

 @unittest.skipIf(sys.platform.startswith('freebsd') and
  sys.getfilesystemencoding() == 'ascii',
  'skip on FreeBSD with ASCII filesystem encoding')

Such skip is not a good idea. Many OS uses the Latin1 encoding when the C 
locale is used (even if ASCII encoding is announced :-/): Solaris, FreeBSD, Mac 
OS X, etc.

pythonrun_filename_decoding_test_2.patch: 'surrogateescape' error handler is 
not used on Windows (and must not be used), whereas the initial issue was 
reported on Windows.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[STINNER Victor]

STINNER Victor added the comment:

I propose a test with a single non-ASCII character, which should be supported 
by more code pages/locale encodings. It checks also the value of __file__. I 
only ran the test on Linux with UTF-8 locale encoding.

--
Added file: http://bugs.python.org/file27888/test_non_ascii.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[Andrew Svetlov]

Andrew Svetlov added the comment:

I like the last patch from Victor. It works on Windows also.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[Roundup Robot]

Roundup Robot added the comment:

New changeset 56df0d4f0011 by Andrew Svetlov in branch 'default':
Issue #16218: Fix test for issue again
http://hg.python.org/cpython/rev/56df0d4f0011

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[Antoine Pitrou]

Antoine Pitrou added the comment:

How does the test which has been committed even test the Python launcher? It 
only calls assert_python_ok(), which should use the regular Python interpreter.

--
nosy: +pitrou

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[Andrew Svetlov]

Andrew Svetlov added the comment:

Well. Fix (and test) is related to bug in python itself (./Python/pythonrun.c) 
pylauncher should be tested also, you are right.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Such test is not enough.

1. It skipped on locales which does not support £ (cp1006, cp1250, cp1251, 
cp737, cp852, cp855, cp866, cp874, cp949, euc_kr, gb2312, gbk, hz, iso2022_kr, 
iso8859_10, iso8859_11, iso8859_16, iso8859_2, iso8859_4, iso8859_5, iso8859_6, 
johab, koi8_r, koi8_u, mac_arabic, mac_farsi, ptcp154, tis_620).  But the bug 
is actual on such locales.

2. It tests nothing on utf-8 locale (test passed even when bug is not fixed).

We should test every filename which can be used in file system, even if it can 
not be decoded using current locale or UTF-8 encoding.  On Unix filenames are 
bytes sequences and we should use 
non_ascii_bytes.decode(sys.getfilesystemencoding(), 'surrogateescape') as 
script name.  On Windows it possible will be chr(k) where k is minimal code  
127 such that chr(k).encode('mbcs') is not fails (I am not sure).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[Ramchandra Apte]

Changes by Ramchandra Apte maniandra...@gmail.com:


--
title: Python launcher does not support non ascii characters - Python launcher 
does not support unicode characters

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[Roundup Robot]

Roundup Robot added the comment:

New changeset 884c2e93d3f7 by Andrew Svetlov in branch 'default':
Issue #16218: Fix broken test for supporting nonascii characters in python 
launcher
http://hg.python.org/cpython/rev/884c2e93d3f7

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[Andrew Svetlov]

Andrew Svetlov added the comment:

I like to follow Stefan suggestion.
New test is simple and it works.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[Stefan Krah]

Stefan Krah added the comment:

I think this is what went wrong on Windows in the previous test (see
Lib/test/test_cmd_line_script.py:43):

 s = '\udcf1\udcea\udcf0\udce8\udcef\udcf2'
 f = open(s, w)
 f.write(print('%s\\n' % __file__))
 f.close()

C:\Users\stefan\pydev\cpythonPCbuild\amd64\python_d.exe ï�
encoding error

So __file__ isn't set correctly, which looks like a bug to me. I'm not sure
whether it should be part of this issue or if we should open a new one.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[Roundup Robot]

Roundup Robot added the comment:

New changeset 95d1adf144ee by Andrew Svetlov in branch 'default':
Issue #16218: skip test if filesystem doesn't support required encoding
http://hg.python.org/cpython/rev/95d1adf144ee

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16218] Python launcher does not support unicode characters

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Andrew, you shod a flea.

1. Now the test skipped on non Cyrillic-compatible locales (such as 
en_US.ISO-8859-1).
2. On UTF-8 locale the test does not test the bug (it passed even without the 
patch).

Here is a new patch. It should fail on FreeBSD with ASCII locale (because there 
is a yet not fixed bug), and I don't know how it will behave on Windows. 
Temporary you can explicitly skip the test for such case:

@unittest.skipIf(sys.platform.startswith('freebsd') and
 sys.getfilesystemencoding() == 'ascii',
 'skip on FreeBSD with ASCII filesystem encoding')

--
Added file: 
http://bugs.python.org/file27854/pythonrun_filename_decoding_test_2.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16218
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com