[issue9630] Reencode filenames when setting the filesystem encoding

2010-09-29 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: Forget my previous message, I forgot important points. So the only reason why you have to go through all those hoops is to * allow the complete set of Python supported encoding names for the PYTHONFSENCODING * make sure

[issue9630] Reencode filenames when setting the filesystem encoding

2010-09-29 Thread Marc-Andre Lemburg
Marc-Andre Lemburg m...@egenix.com added the comment: STINNER Victor wrote: STINNER Victor victor.stin...@haypocalc.com added the comment: Forget my previous message, I forgot important points. So the only reason why you have to go through all those hoops is to * allow the complete

[issue9630] Reencode filenames when setting the filesystem encoding

2010-09-29 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: Le mercredi 29 septembre 2010 13:45:15, vous avez écrit : Marc-Andre Lemburg m...@egenix.com added the comment: STINNER Victor wrote: STINNER Victor victor.stin...@haypocalc.com added the comment: Forget my previous

[issue9630] Reencode filenames when setting the filesystem encoding

2010-09-29 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: I commited redecode_modules_path-4.patch as r85115 in Python 3.2. -- resolution: - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9630

[issue9630] Reencode filenames when setting the filesystem encoding

2010-09-29 Thread Marc-Andre Lemburg
Marc-Andre Lemburg m...@egenix.com added the comment: STINNER Victor wrote: STINNER Victor victor.stin...@haypocalc.com added the comment: Le mercredi 29 septembre 2010 13:45:15, vous avez écrit : Marc-Andre Lemburg m...@egenix.com added the comment: STINNER Victor wrote: STINNER

[issue9630] Reencode filenames when setting the filesystem encoding

2010-09-24 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: New version of the patch: - reencode sys.path_importer_cache (and remove the last FIXME) - fix different reference leaks - catch PyIter_Next() failures - create a subfunction to reencode sys.modules: it's easier to review and

[issue9630] Reencode filenames when setting the filesystem encoding

2010-09-24 Thread STINNER Victor
Changes by STINNER Victor victor.stin...@haypocalc.com: Removed file: http://bugs.python.org/file18561/reencode_modules_path-2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9630 ___

[issue9630] Reencode filenames when setting the filesystem encoding

2010-09-24 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: I would rename the feature to something like redecode-modules Yes, right. I will rename the functions before commiting the patch. -- ___ Python tracker rep...@bugs.python.org

[issue9630] Reencode filenames when setting the filesystem encoding

2010-09-24 Thread Marc-Andre Lemburg
Marc-Andre Lemburg m...@egenix.com added the comment: STINNER Victor wrote: STINNER Victor victor.stin...@haypocalc.com added the comment: New version of the patch: - reencode sys.path_importer_cache (and remove the last FIXME) - fix different reference leaks - catch PyIter_Next()

[issue9630] Reencode filenames when setting the filesystem encoding

2010-09-24 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: Why is this needed ? Short answer: to support filesystem encoding different than utf-8. See #8611 for a longer explanation. Example: $ pwd /home/SHARE/SVN/py3ké $ PYTHONFSENCODING=ascii ./python test_fs_encoding.py Fatal Python

[issue9630] Reencode filenames when setting the filesystem encoding

2010-09-24 Thread Antoine Pitrou
Antoine Pitrou pit...@free.fr added the comment: It is not possible to set Py_FilesystemDefaultEncoding before loading the first module. initfsencoding() loads codecs and encodings modules to check the codec name. Not sure it's related, but there seems to be a bug: $ ./python -c import sys;

[issue9630] Reencode filenames when setting the filesystem encoding

2010-09-24 Thread Antoine Pitrou
Antoine Pitrou pit...@free.fr added the comment: Some things about your patch: - as Amaury said, functions should be named redecode* rather than reencode* - please use -1 for error return, not 1 - have you tried to measure if it made Python startup slower? --

[issue9630] Reencode filenames when setting the filesystem encoding

2010-09-24 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: Not sure it's related, but there seems to be a bug: It's not a bug, it's a feature :-) If you specify a non-existing locale, the GNU libc fails back to ascii. $ locale -a C français french fr_FR fr...@euro fr_FR.iso88591

[issue9630] Reencode filenames when setting the filesystem encoding

2010-09-24 Thread Antoine Pitrou
Changes by Antoine Pitrou pit...@free.fr: -- nosy: +brett.cannon, loewis ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9630 ___ ___

[issue9630] Reencode filenames when setting the filesystem encoding

2010-09-24 Thread Marc-Andre Lemburg
Marc-Andre Lemburg m...@egenix.com added the comment: STINNER Victor wrote: STINNER Victor victor.stin...@haypocalc.com added the comment: Why is this needed ? Short answer: to support filesystem encoding different than utf-8. See #8611 for a longer explanation. Example: $ pwd

[issue9630] Reencode filenames when setting the filesystem encoding

2010-09-24 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: Some things about your patch: - as Amaury said, functions should be named redecode* rather than reencode* Yes, as written before (msg117269), I will do it in my next patch. - please use -1 for error return, not 1 Ok. - have

[issue9630] Reencode filenames when setting the filesystem encoding

2010-09-24 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: Le vendredi 24 septembre 2010 14:35:29, Marc-Andre Lemburg a écrit : Thanks for the explanation. So the only reason why you have to go through all those hoops is to * allow the complete set of Python supported encoding names

[issue9630] Reencode filenames when setting the filesystem encoding

2010-09-05 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: Do we really want to support this kind of configuration? There is also a problem is the directory name is b'py3k\xe9': at startup (utf-8 encoding), the name is decoded to 'py3k\udce9'. When the locale encoding is set to

[issue9630] Reencode filenames when setting the filesystem encoding

2010-09-04 Thread Arfrever Frehtes Taifersar Arahesis
Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com: -- nosy: +Arfrever ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9630 ___

[issue9630] Reencode filenames when setting the filesystem encoding

2010-09-04 Thread Amaury Forgeot d'Arc
Amaury Forgeot d'Arc amaur...@gmail.com added the comment: Python is installed in a directory called b'py3k\xc3\xa9' and your locale is C Do we really want to support this kind of configuration? -- ___ Python tracker rep...@bugs.python.org

[issue9630] Reencode filenames when setting the filesystem encoding

2010-09-03 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: Another possibility is to use _Py_char2wchar() + PyUnicode_FromWideChar() / _Py_wchar2char() + PyUnicode_AsWideChar() to decode / encode filenames. These functions use the locale encoding. This solution was possible in Python 3.1,

[issue9630] Reencode filenames when setting the filesystem encoding

2010-08-19 Thread Amaury Forgeot d'Arc
Amaury Forgeot d'Arc amaur...@gmail.com added the comment: I would rename the feature to something like redecode-modules: the filenames were decoded with the wrong encoding, and must be decoded again. -- nosy: +amaury.forgeotdarc ___ Python tracker

[issue9630] Reencode filenames when setting the filesystem encoding

2010-08-19 Thread Amaury Forgeot d'Arc
Amaury Forgeot d'Arc amaur...@gmail.com added the comment: Some thoughts: since the modules were successfully imported, surely it means that their filenames where correctly computed and encoded? So why is the __filename__ attribute wrong? -- ___

[issue9630] Reencode filenames when setting the filesystem encoding

2010-08-19 Thread STINNER Victor
STINNER Victor victor.stin...@haypocalc.com added the comment: since the modules were successfully imported, surely it means that their filenames where correctly computed and encoded? So why is the __filename__ attribute wrong? Python starts with 'utf-8' encoding. If the new encoding is

[issue9630] Reencode filenames when setting the filesystem encoding

2010-08-17 Thread STINNER Victor
Changes by STINNER Victor victor.stin...@haypocalc.com: -- title: Reencode filenames of all module and code objects when setting the filesystem encoding - Reencode filenames when setting the filesystem encoding ___ Python tracker