[issue3824] test_tarfile fails on cygwin (unicode decode error)

Changes by Ezio Melotti : -- resolution: -> out of date stage: -> resolved status: pending -> closed type: -> behavior ___ Python tracker

[issue3824] test_tarfile fails on cygwin (unicode decode error)

Changes by Serhiy Storchaka storch...@gmail.com: -- status: open - pending ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3824 ___ ___

[issue3824] test_tarfile fails on cygwin (unicode decode error)

Amaury Forgeot d'Arc amaur...@gmail.com added the comment: grp.getgrgid() now calls .decode('utf8', errors=surrogateescape). Even if cygwin does not correctly copy strings from the Windows registry, tarinfo.gname should now contain a string that will at least round trip and give the same value

[issue3824] test_tarfile fails on cygwin (unicode decode error)

Amaury Forgeot d'Arc [EMAIL PROTECTED] added the comment: Doesn't getgrgid() return the untranslated content of /etc/group? Then the encoding of this file is relevant. On cygwin, mkgroup -l is often (exclusively?) used to generate this /etc/group, extracting the user definitions from the

[issue3824] test_tarfile fails on cygwin (unicode decode error)

Hirokazu Yamamoto [EMAIL PROTECTED] added the comment: Doesn't getgrgid() return the untranslated content of /etc/group? Then the encoding of this file is relevant. Yes, /etc/group contains なし as gr_name in MBCS,（なし means nothing）and I can print it with puts() in grpmodule.c, so it shouldn't be

[issue3824] test_tarfile fails on cygwin (unicode decode error)

Martin v. Löwis [EMAIL PROTECTED] added the comment: Doesn't getgrgid() return the untranslated content of /etc/group? Then the encoding of this file is relevant. That certainly depends on the implementation of getgrgid. On some systems, it uses NIS, LDAP, or a relational database in addition

[issue3824] test_tarfile fails on cygwin (unicode decode error)

Roumen Petrov [EMAIL PROTECTED] added the comment: What is test result if the environment variable LANG is set to C ? -- nosy: +rpetrov ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3824 ___

[issue3824] test_tarfile fails on cygwin (unicode decode error)

Hirokazu Yamamoto [EMAIL PROTECTED] added the comment: What is test result if the environment variable LANG is set to C ? There is no change. ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3824 ___

[issue3824] test_tarfile fails on cygwin (unicode decode error)

Changes by Hirokazu Yamamoto [EMAIL PROTECTED]: Removed file: http://bugs.python.org/file11455/experimental_mbcstowcs_codec.patch ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3824 ___

[issue3824] test_tarfile fails on cygwin (unicode decode error)

Hirokazu Yamamoto [EMAIL PROTECTED] added the comment: I'm not cygwin user, but cygwin seems not to support multibyte function. Following program outputs 5 on VC6 as expected, but 10 on cygwin. Hmm... #include stdio.h #include stdlib.h #include locale.h int main(int argc, char* argv[]) {

[issue3824] test_tarfile fails on cygwin (unicode decode error)

Martin v. Löwis [EMAIL PROTECTED] added the comment: In this case, I think there is nothing we can do. Perhaps it is useful to put a comment into the test, pointing out that this is likely to break on Cygwin, and refer to this issue. I don't see that as a problem: it's just a test that fails,

[issue3824] test_tarfile fails on cygwin (unicode decode error)

Hirokazu Yamamoto [EMAIL PROTECTED] added the comment: Sorry, the patch didn't work... I didn't understand Martin's word. And nl_langinfo(CODESET) is useless on cygwin because it's always US-ASCII. ___ Python tracker [EMAIL PROTECTED]

[issue3824] test_tarfile fails on cygwin (unicode decode error)

Martin v. Löwis [EMAIL PROTECTED] added the comment: I didn't mean to suggest that a new codec is created; instead, mbstowcs should be called directly in grpmodule.c. By default, mbstowcs will use ASCII, so it is likely to fail - you would need to call setlocale first.

[issue3824] test_tarfile fails on cygwin (unicode decode error)

Hirokazu Yamamoto [EMAIL PROTECTED] added the comment: Sorry, probably I saw illusion... If uses cp932 codec, still test_tarfile.py reports error. :-( == ERROR: test_tar_size (__main__.WriteTest)

[issue3824] test_tarfile fails on cygwin (unicode decode error)

Amaury Forgeot d'Arc [EMAIL PROTECTED] added the comment: Is PyUnicode_DecodeMBCS available on cygwin? I get compilation errors when I try your patch. -- nosy: +amaury.forgeotdarc ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue3824

[issue3824] test_tarfile fails on cygwin (unicode decode error)

Hirokazu Yamamoto [EMAIL PROTECTED] added the comment: Yes, when I did it last night, I thought I could compile it and saw OK on test_tarfile.py, but probably I dreamed. :-( #define PyUnicode_FromString(s) PyUnicode_Decode(s, strlen(s), cp932, strict) or following patch should work.

[issue3824] test_tarfile fails on cygwin (unicode decode error)

New submission from Hirokazu Yamamoto [EMAIL PROTECTED]: I noticed test_tarfile on py3k fails like this. == ERROR: test_directory_size (__main__.WriteTest) --

[issue3824] test_tarfile fails on cygwin (unicode decode error)

Martin v. Löwis [EMAIL PROTECTED] added the comment: I think you should use the locale's encoding to process the data, ie. either mbstowcs, then Unicode from wchar_t, or decode with the nl_langinfo(CODESET) encoding. You might have to set the locale before this can work (which isn't