On Tue, Feb 2, 2021 at 12:23 AM Victor Stinner <vstin...@python.org> wrote: > > Hi Inada-san, > > I followed the discussions on your different PEP and I like overall > your latest PEP :-) I have some minor remarks. > > On Mon, Feb 1, 2021 at 6:55 AM Inada Naoki <songofaca...@gmail.com> wrote: > > The warning is disabled by default. New ``-X warn_encoding`` > > command-line option and ``PYTHONWARNENCODING`` environment variable > > are used to enable the warnings. > > Maybe "warn implicit encoding" or "warn omit encoding" (not sure if > it's make sense written like that in english ;-)) would be more > explicit. >
Yes, it's explicit. So I used `PYTHONWARNDEFAULTENCODING` first. But I feel it's unreadable. That's why I shorten the option name. I wait to see more feedback about naming. > > > Options to enable the warning > > ------------------------------ > > > > ``-X warn_encoding`` option and the ``PYTHONWARNENCODING`` > > environment variable are added. They are used to enable the > > ``EncodingWarning``. > > > > ``sys.flags.encoding_warning`` is also added. The flag represents > > ``EncodingWarning`` is enabled. > > Nitpick: I would prefer using the same name for the -X option and the > sys.flags attribute (ex: sys.flags.warn_encoding). > OK, I will change the flag name same to option name. > > > ``encoding="locale"`` option > > ---------------------------- > > > > ``io.TextIOWrapper`` accepts ``encoding="locale"`` option. It means > > same to current ``encoding=None``. But ``io.TextIOWrapper`` doesn't > > emit ``EncodingWarning`` when ``encoding="locale"`` is specified. > > Can you please define if os.device_encoding(fd) is called if > encoding="locale" is used? It seems so, so it's not obvious from the > PEP. > OK. > > In Python 3.10, I added _locale._get_locale_encoding() function which > is exactly what the encoding used by open() when no encoding is > specified (encoding=None) and when os.device_encoding(fd) returns > None. See _Py_GetLocaleEncoding() for the C implementation > (Python/fileutils.c). > > Maybe we should add a public locale.get_locale_encoding() function? On > Unix, this function uses nl_langinfo(CODESET) *without* setting > LC_CTYPE locale to the user preferred locale. > I can not imagine any use case. Isn't it just confusing? > I understand that encoding=locale.get_locale_encoding() would be > different from encoding="locale": > encoding=locale.get_locale_encoding() doesn't call > os.device_encoding(), right? > Yes. > > Maybe the PEP should also explain (in a "How to teach this" section?) > when encoding="locale" is better than a specific encoding, like > encoding="utf-8" or encoding="cp1252". In my experience, it's mostly > for the inter-operability which other applications which also use the > current locale encoding. > This option is for experts who are publishing cross-platform libraries, frameworks, etc. For students, I am suggesting another idea that make UTF-8 mode more accessible. > > > Add ``io.LOCALE_ENCODING = "locale"`` constant too. This constant can > > be used to avoid confusing ``LookupError: unknown encoding: locale`` > > error when the code is run in old Python accidentally. > > I'm not sure that it is useful. I like a simple "locale" literal > string. If there is a constant is io, people may start to think that > it's specific and will add "import io" just to get the string > "locale". > > I don't think that we should care too much about the error message > rased by old Python versions. > This constant not only for replacing "locale" litera. As example code in the PEP, it can be used to test wheather TextIOWrapper supports `encoding="locale"` . `open(fn, encoding=getattr(io, "LOCALE_ENCODING", None))` works both for Python ~3.9 and Python 3.10~. > > > > Opt-in warning > > --------------- > > > > Although ``DeprecationWarning`` is suppressed by default, emitting > > ``DeprecationWarning`` always when ``encoding`` option is omitted > > would be too noisy. > > The PEP is not very clear. Does "-X warn_encoding" only emits the > warning, or does it also display it by default? Does it add a warning > filter for EncodingWarning? > This section is not the spec. This section is the rationale for adding EncodingWarning instead of using DeprecationWarning. As spec saying, EncodingWarning is a subclass of Warning. So it is displayed by default. But it is not emitted by default. When -X encoding_warning (or -X warn_default_encoding) is used, the warning is emitted and shown unless the user suppresses warnings. > > The PEP has no "Backward compatibility" section. Is it possible to > monkey-patch Python to implement this PEP (maybe only partially) on > old Python versions? I'm asking to prepare existing projects for > future EncodingWarning. > This PEP doesn't have "backward compatibility" section because the PEP doesn't break any backward compatibility. Unless the option is enabled, no warnings are emitted by the PEP, like `-b` option and BytesWarning. And if developers want to support Python ~3.9 and use -X warn_default_encoding on 3.10, they need to write `encoding=getattr(io, "LOCALE_ENCODING", None)`, as written in the spec. > The main question is if it's possible to use encoding="locale" on > Python 3.6-3.9 (maybe using some ugly hacks). No. > By the way, your PEP has > no target Python version ;-) Do you want to get it in Python 3.10 or > 3.11? > Oh, I'm sorry. I want to make it in 3.10. Precisely speaking, add option and warning in 3.10. But no need to fix all warnings in stdlib and tests by 3.10. I expect we find many codes using locale-specific encoding without reason in stdlib and we need to fix them until 3.11 or even 3.12. After that, we can recommend using the option for many third-party libraries. My long-term plan (out of scope of this PEP) is: * Python 3.10 : Add the optional warning * Python 3.12 : Fix all warnings in stdlib, start the option for third parties. * Python 3.16 : Discuss changing the default encoding. If we decided to change, enable the warning by default. * Python 3.20 : Change the default encoding to UTF-8. Regards, -- Inada Naoki <songofaca...@gmail.com> _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/CEUFKVXP4IQQHVCZ6CB3IBWBOMEZMHH4/ Code of Conduct: http://python.org/psf/codeofconduct/
