[issue850997] mbcs encoding ignores errors
STINNER Victor victor.stin...@haypocalc.com added the comment: Close this issue: nothing special on the buildbots. -- resolution: - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue850997 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue850997] mbcs encoding ignores errors
Tim Golden m...@timgolden.me.uk added the comment: I'm unlikely to get to it soon. If there's no urgency I can look at it later. FWIW, it's not something I'm especially familiar with. On 12/06/2010 01:02, STINNER Victor wrote: STINNER Victorvictor.stin...@haypocalc.com added the comment: Tim: are you interested in testing this patch? -- nosy: +tim.golden ___ Python trackerrep...@bugs.python.org http://bugs.python.org/issue850997 ___ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue850997 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue850997] mbcs encoding ignores errors
STINNER Victor victor.stin...@haypocalc.com added the comment: Patch version 4: - encode_mbcs() uses WC_NO_BEST_FIT_CHARS flag in strict mode. Examples: ğ and ł are not more replaced by g and l - encode_mbcs() doesn't set *repr to NULL on encode error: the caller does anyway destroy it - write more documentation about mbcs, especially about the error handlers and the changes in Python 3.2 -- Added file: http://bugs.python.org/file17690/mbcs_errors-py3k-4.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue850997 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue850997] mbcs encoding ignores errors
STINNER Victor victor.stin...@haypocalc.com added the comment: I commited the last patch to py3k: r82037. Let see how the buildbots react :-) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue850997 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue850997] mbcs encoding ignores errors
STINNER Victor victor.stin...@haypocalc.com added the comment: Tim: are you interested in testing this patch? -- nosy: +tim.golden ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue850997 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue850997] mbcs encoding ignores errors
STINNER Victor victor.stin...@haypocalc.com added the comment: Update the patch (I commited the patch on tarfile module): version 3. -- Added file: http://bugs.python.org/file17635/mbcs_errors-py3k-3.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue850997 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue850997] mbcs encoding ignores errors
Changes by STINNER Victor victor.stin...@haypocalc.com: Removed file: http://bugs.python.org/file17456/mbcs_errors-py3k.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue850997 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue850997] mbcs encoding ignores errors
Changes by STINNER Victor victor.stin...@haypocalc.com: Removed file: http://bugs.python.org/file17622/mbcs_errors-py3k-2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue850997 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue850997] mbcs encoding ignores errors
STINNER Victor victor.stin...@haypocalc.com added the comment: I worked again on the patch. I opened new issues to prepare the new mbcs codec: - #8966: ctypes: remove implicit conversion between unicode and bytes - #8967: Create PyErr_GetWindowsMessage() function - #8969: Windows: use (mbcs in) strict mode to encode/decode filenames, and enable os.fsencode() #8967 can be used to get the translated message of a mbcs encode error. PyErr_GetWindowsMessage() returns a PyUnicodeObject, whereas make_translate_exception() and PyUnicodeTranslateError_SetReason() expect a char*. Another patch is requied: translate_reason_unicode.patch (attached to this issue, not tested). But I don't think that the message is very important for now :-) #8784 (tarfile/Windows: Don't use mbcs as the default encoding) is still open. -- Added file: http://bugs.python.org/file17618/translate_reason_unicode.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue850997 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue850997] mbcs encoding ignores errors
STINNER Victor victor.stin...@haypocalc.com added the comment: New version of the patch: - decode_mbcs() calls raise_translate_exception() to set the error (in the previous patch, I'm not sure that the error was set) - include #8784 patch (tarfile uses utf-8 as the default encoding) - ctypes: use mbcs is strict mode instead of ignore mode. This is just a workaround, the real fix is to remove the implicit conversion between bytes and characters: see #8966 The patch requires #8969 patch (use mbcs in strict mode to encode/decode filenames). -- Added file: http://bugs.python.org/file17622/mbcs_errors-py3k-2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue850997 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue850997] mbcs encoding ignores errors
STINNER Victor victor.stin...@haypocalc.com added the comment: Updated version of the patch for py3k: - don't accept ignore error handler anymore - there is a FIXME near mbcs_decode_error: The whole test suite pass with these patch. -- Added file: http://bugs.python.org/file17456/mbcs_errors-py3k.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue850997 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue850997] mbcs encoding ignores errors
Changes by STINNER Victor victor.stin...@haypocalc.com: -- nosy: +haypo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue850997 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue850997] mbcs encoding ignores errors
STINNER Victor victor.stin...@haypocalc.com added the comment: I patched py3k with mbcs_errors.patch (only encode_mbcs, not the decoder function) and most test pass: I opened #8784 for test_tarfile failure. I don't think that it's a problem that mbcs only supports few error handlers, eg. 'strict', 'replace' and 'errors' (but not 'ignore' nor 'surrogateescape'). mbcs should be avoided anyway :-) It is kept for backward compatibility (with Python2). Python3 tries to avoid it by using the Unicode functions of Windows API. I don't know exactly where mbcs is still used in Python3. If mbcs becomes more strict and raise new errors, I would like to say that the problem comes from the program, not in the encodig, and the program should be fixed (especilly if the program is the Python standard library). About the backward compatibility with Python 3.2: I don't know exactly if this change would be a problem or not. I bet that few people use (directly or indirectly) mbcs with Python 3.1 (on Windows), and few peple (or nobody) would notice this change. And as I wrote, if someone notices a problem: the problem should be fixed in the function using mbcs, not in the codec. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue850997 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue850997] mbcs encoding ignores errors
STINNER Victor victor.stin...@haypocalc.com added the comment: Since this change breaks backward compatibility, it's a very bad idea to change mbcs codec in Python 2.7: remove this version from this issue. -- versions: -Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue850997 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue850997] mbcs encoding ignores errors
Changes by Ezio Melotti ezio.melo...@gmail.com: -- nosy: +ezio.melotti versions: +Python 2.7, Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue850997 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue850997] mbcs encoding ignores errors
Daniel Diniz aja...@gmail.com added the comment: Is this behavior still present? If so, is it still interesting to change it? -- components: +Unicode keywords: +patch nosy: +ajaksu2 stage: - test needed type: - feature request ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue850997 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue850997] mbcs encoding ignores errors
Changes by Thomas Heller thel...@ctypes.org: -- nosy: -theller ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue850997 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue850997] mbcs encoding ignores errors
Mark Hammond mhamm...@users.sourceforge.net added the comment: It is still present, but I'm not sure what problems can be seen due to this so can't comment on its desirability. It would also introduce a backwards compatability concern but I've not enough experience to know how much of a problem that would be in practice either. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue850997 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
