[issue16414] Add support.NONASCII to test non-ASCII characters

2013-01-02 Thread Roundup Robot
Roundup Robot added the comment: New changeset 41658a4fb3cc by Victor Stinner in branch '3.2': Issue #16218, #16414, #16444: Backport FS_NONASCII, TESTFN_UNDECODABLE, http://hg.python.org/cpython/rev/41658a4fb3cc New changeset 4d40c1ce8566 by Victor Stinner in branch '3.3': (Merge 3.2) Issue

[issue16414] Add support.NONASCII to test non-ASCII characters

2013-01-02 Thread STINNER Victor
Changes by STINNER Victor victor.stin...@gmail.com: -- versions: +Python 3.2, Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16414 ___

[issue16414] Add support.NONASCII to test non-ASCII characters

2012-11-06 Thread Serhiy Storchaka
Serhiy Storchaka added the comment: I tested brute.py for all supported in Python encodings: No character for encoding cp1006:surrogateescape :-( No character for encoding cp720:surrogateescape :-( No character for encoding cp864:surrogateescape :-( No character for encoding

[issue16414] Add support.NONASCII to test non-ASCII characters

2012-11-06 Thread STINNER Victor
STINNER Victor added the comment: I tested brute.py for all supported in Python encodings: Oh thanks, interesting result. I completed the encoding list and the character list: see brute2.py. I added joker characters: U+00A0 and U+20AC which match requierements for most locale encodings.

[issue16414] Add support.NONASCII to test non-ASCII characters

2012-11-06 Thread Roundup Robot
Roundup Robot added the comment: New changeset de8cf1ece068 by Victor Stinner in branch 'default': Issue #16414: Add support.FS_NONASCII and support.TESTFN_NONASCII http://hg.python.org/cpython/rev/de8cf1ece068 -- nosy: +python-dev ___ Python tracker

[issue16414] Add support.NONASCII to test non-ASCII characters

2012-11-06 Thread Roundup Robot
Roundup Robot added the comment: New changeset 0e9fbdda3c92 by Victor Stinner in branch 'default': Issue #16414: Fix support.TESTFN_UNDECODABLE and test_genericpath.test_nonascii_abspath() http://hg.python.org/cpython/rev/0e9fbdda3c92 -- ___ Python

[issue16414] Add support.NONASCII to test non-ASCII characters

2012-11-06 Thread Serhiy Storchaka
Serhiy Storchaka added the comment: Why were you add '- ' suffix to TESTFN_NONASCII? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16414 ___

[issue16414] Add support.NONASCII to test non-ASCII characters

2012-11-06 Thread Serhiy Storchaka
Serhiy Storchaka added the comment: I don't see U+00A0 and U+20AC in the changeset. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16414 ___ ___

[issue16414] Add support.NONASCII to test non-ASCII characters

2012-11-06 Thread Roundup Robot
Roundup Robot added the comment: New changeset 55710b8c6670 by Victor Stinner in branch 'default': Issue #16414: Fix typo in support.TESTFN_NONASCII (useless space) http://hg.python.org/cpython/rev/55710b8c6670 -- ___ Python tracker

[issue16414] Add support.NONASCII to test non-ASCII characters

2012-11-06 Thread STINNER Victor
Changes by STINNER Victor victor.stin...@gmail.com: Added file: http://bugs.python.org/file27912/brute2.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16414 ___

[issue16414] Add support.NONASCII to test non-ASCII characters

2012-11-06 Thread Roundup Robot
Roundup Robot added the comment: New changeset 7f90305d9f23 by Victor Stinner in branch 'default': Issue #16414: Test more characters for support.FS_NONASCII http://hg.python.org/cpython/rev/7f90305d9f23 -- ___ Python tracker rep...@bugs.python.org

[issue16414] Add support.NONASCII to test non-ASCII characters

2012-11-06 Thread Roundup Robot
Roundup Robot added the comment: New changeset fce9e892c65d by Victor Stinner in branch 'default': Issue #16414: Fix test_os on Windows, don't test os.listdir() with undecodable http://hg.python.org/cpython/rev/fce9e892c65d -- ___ Python tracker

[issue16414] Add support.NONASCII to test non-ASCII characters

2012-11-06 Thread STINNER Victor
STINNER Victor added the comment: Why were you add '- ' suffix to TESTFN_NONASCII? Oops, the space was a mistake. I add - just for the readability of the generated filename. I don't see U+00A0 and U+20AC in the changeset. Oh, I forgot to update the patch with the latest results of

[issue16414] Add support.NONASCII to test non-ASCII characters

2012-11-06 Thread STINNER Victor
STINNER Victor added the comment: Handling non-ASCII paths is always a pain. I don't plan to backport support.FS_NONASCII to Python 3.3 right now, but I may backport it later. -- resolution: - fixed status: open - closed versions: -Python 3.3 ___

[issue16414] Add support.NONASCII to test non-ASCII characters

2012-11-05 Thread STINNER Victor
New submission from STINNER Victor: Attached patch adds support.NONASCII to have a portable non-ASCII character that can be used to test non-ASCII strings. The patch uses it in some existing functions. I wrote the patch on the default branch, we may start to use it since Python 3.2.

[issue16414] Add support.NONASCII to test non-ASCII characters

2012-11-05 Thread Serhiy Storchaka
Serhiy Storchaka added the comment: I think you should ensure that os.fsdecode(os.fsencode(character)) == character. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16414 ___

[issue16414] Add support.NONASCII to test non-ASCII characters

2012-11-05 Thread Serhiy Storchaka
Serhiy Storchaka added the comment: If NONASCII is None I suggest the followed fallback code for i in range(0x80, 0x): character = chr(i) if character.isprintable(): try: if os.fsdecode(os.fsencode(character)) == character:

[issue16414] Add support.NONASCII to test non-ASCII characters

2012-11-05 Thread Chris Jerdonek
Chris Jerdonek added the comment: +# NONASCII: non-ASCII character encodable by os.fsencode(), +# or None if there is no such character. +NONASCII = None Can you use a name that reflects that this is a specific type of non-ASCII character having a special property (e.g. FS_NONASCII)? I think

[issue16414] Add support.NONASCII to test non-ASCII characters

2012-11-05 Thread STINNER Victor
STINNER Victor added the comment: If NONASCII is None I suggest the followed fallback code I prefer to not brute force Unicode because it would slow down any test, even tests not using FS_NONASCII. I wrote attached brute.py script to compute an exhaustive list of non-ASCII characters

[issue16414] Add support.NONASCII to test non-ASCII characters

2012-11-05 Thread STINNER Victor
STINNER Victor added the comment: I tested support_non_ascii-2.patch on Windows with cp932 ANSI code page (FS encoding), and on Linux with ASCII, ISO-8859-1, ISO-8859-15 and UTF-8 locale encodings. -- ___ Python tracker rep...@bugs.python.org

[issue16414] Add support.NONASCII to test non-ASCII characters

2012-11-05 Thread STINNER Victor
STINNER Victor added the comment: I think you should ensure that os.fsdecode(os.fsencode(character)) == character. Chosen characters respect this property, but it doesn't hurt to add such check. Can you use a name that reflects that this is a specific type of non-ASCII character having a

[issue16414] Add support.NONASCII to test non-ASCII characters

2012-11-05 Thread STINNER Victor
Changes by STINNER Victor victor.stin...@gmail.com: Removed file: http://bugs.python.org/file27892/support_non_ascii.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16414 ___