Eryk Sun <eryk...@gmail.com> added the comment:
> I was one of the people who mistakenly thought that Python 3 operating > in the new Windows Terminal was going to magically leave us sitting > happily in completely UTF-8 compatible territory on Windows, not > realizing the complex long-term dependencies and regressions that > still remain problematic. Windows Terminal provides a tab-based UI that supports font fallback and complex scripts, which is a significant improvement over the builtin terminal that conhost.exe provides. Each tab is a headless (conpty) console session that's hosted by an instance of OpenConsole.exe. The latter is based on the same source tree as the system conhost.exe, but typically it's a more recent build. The console host is what implements most of the API for client applications. That the host is in headless mode and connected to an alternate terminal doesn't matter from the perspective of client applications. The console has been Unicode (UCS-2) back to its initial release in 1993. Taking advantage of this requires reading and writing wide-character strings via ReadConsoleW and WriteConsoleW, as Python's does in 3.6+ (except not for os.read and os.write). Many console applications instead use encoded byte strings with ReadFile / ReadConsoleA and WriteFile / WriteConsoleA. Updating the console host to support UTF-8 for this has been a drawn-out process. It finally has full support for writing UTF-8 in Windows 10, including splitting a sequence across multiple writes. But reading non-ASCII UTF-8 is still broken. "more.com" uses the console input codepage to decode the file, so a workaround is to run `chcp.com 65001` and run Python in UTF-8 mode, e.g. `py -X utf8=1`. Since reading non-ASCII UTF-8 is broken, you'll have to switch back to the old input codepage if you need to enter non-ASCII characters in an app that reads from the console via ReadFile or ReadConsoleA. ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue44275> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
