[issue8390] tarfile: use surrogates for undecode fields

Changes by STINNER Victor victor.stin...@haypocalc.com: -- status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8390 ___ ___

[issue8390] tarfile: use surrogates for undecode fields

Lars Gustäbel l...@gustaebel.de added the comment: I think it is a good suggestion to use surrogateescape as the default, because (I hope) it produces the fewest errors and is the best choice if tarfile is used in connection with Python's filesystem calls. - When reading tar headers,

[issue8390] tarfile: use surrogates for undecode fields

STINNER Victor victor.stin...@haypocalc.com added the comment: Thank you for your review. I commited the patch as r80824 (I fixed the documentation, :versionadded = :versionchanged), blocked as r80825 (3.2). -- Unfortunately, POSIX says nothing about how to store bad filenames in a pax

[issue8390] tarfile: use surrogates for undecode fields

Lars Gustäbel l...@gustaebel.de added the comment: Yes, I will soon have ;-) Please give me a few days... -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8390 ___

[issue8390] tarfile: use surrogates for undecode fields

STINNER Victor victor.stin...@haypocalc.com added the comment: A better fix is maybe to store fields as bytes, but it would break the compatibility and unicode is pratical in Python3. -- ___ Python tracker rep...@bugs.python.org

[issue8390] tarfile: use surrogates for undecode fields

STINNER Victor victor.stin...@haypocalc.com added the comment: My patch changes test_uname_unicode() of test_tarfile for the GNU and ustar formats (but not PAX). In GNU and ustar formats, the fields can be encoded in any encoding, and may contain invalid byte sequences. --

[issue8390] tarfile: use surrogates for undecode fields

STINNER Victor victor.stin...@haypocalc.com added the comment: lars: Do you have an opinion about this suggestion? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8390 ___

[issue8390] tarfile: use surrogates for undecode fields

Changes by STINNER Victor victor.stin...@haypocalc.com: -- nosy: +lars.gustaebel ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8390 ___ ___

[issue8390] tarfile: use surrogates for undecode fields

New submission from STINNER Victor victor.stin...@haypocalc.com: When reading a tar archive, tarfile decodes fields using replace error handler by default. The result is that we loose informations if there is an undecodable character. Since the PEP 383, undecodable filenames are stored using