New submission from Bodo Graumann <m...@bodograumann.de>: This is a follow up on https://bugs.python.org/issue37751 concerning normalization of codec names.
First of all, the changes made therein are not documented correctly. In the implementation | Normalization works as follows: all non-alphanumeric | characters except the dot used for Python package names are | collapsed and replaced with a single underscore, e.g. ' -;#' | becomes '_'. Leading and trailing underscores are removed.” Cf. [encodings/__init__.py](https://github.com/python/cpython/blob/bb3e0c240bc60fe08d332ff5955d54197f79751c/Lib/encodings/__init__.py#L47-L50) The documentation however only states that: | Search functions are expected to take one argument, being the encoding name in all lower case letters with hyphens and spaces converted to underscores Cf. https://docs.python.org/3/library/codecs.html#codecs.register Secondly, this change breaks lots of iconv codecs with the python-iconv binding. E.g. `ASCII//TRANSLIT` is now normalized to `ascii_translit`, which iconv does not understand. Codec names which use hyphens also break and iinm not all of them have aliases in iconv without hyphens. Cf. [python-iconv #4](https://github.com/bodograumann/python-iconv/issues/4) How about first looking up the given name and only then, if the given name could not be found, looking for the codec by its normalized name? ---------- components: Unicode messages: 398042 nosy: bodograumann, ezio.melotti, vstinner priority: normal severity: normal status: open title: Codec name normalization breaks custom codecs type: behavior versions: Python 3.9 _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue44723> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
