Jira (PUP-6432) Puppet::Util::Execution.execute cannot handle Unicode output on Windows

[Ethan Brown (JIRA)]

Title: Message Title

Ethan Brown commented on PUP-6432   Re: Puppet::Util::Execution.execute cannot handle Unicode output on Windows A few more notes: cmd.exe has a command line parameter to enable Unicode - not entirely sure yet what behavioral changes it will make: /U Causes the output of internal commands to a pipe or file to be Unicode Changing Puppets codepage to 65001 with chcp 65001 or programatically with SetConsoleOutputCP might not work properly. This would have to be done before Ruby initializes, which I don't believe is even possible. Note that there are supposedly many bugs with using that codepage, that at least anecdotally are WONTFIX per http://stackoverflow.com/a/1371069/87793 Michael Kaplan has a very detailed post about using _O_U16TEXT and _O_U8TEXT with the CRT call _setmode at http://archives.miloush.net/michkap/archive/2008/03/18/8306597.html - not that some of the problems revolve around rendering on Windows, which we don't typically care about unless logoutput is on for the exec. As long as bytes are written correctly in both directions, we should generally be OK. He summarizes conventional, but incorrect Windows limitations as: Unicode I/O with ReadConsoleW/WriteConsoleW is always supported; but default raster font will break it. Changing console codepage is always possible, but DBCS codepages are supported under DBCS system locales (not necessarily matching). The console does not support complex script languages such as Arabic or the various Indic languages that can only be rendered with Uniscribe. Unicode I/O is supported through Win32, but with several limitations: a user has to select a truetype font in console; no redirection of input/output is supported; CJK languages are supported only under CJK system locales. The _setmode solution is also referenced in http://stackoverflow.com/a/9051543/87793 Another good explanation of the run command and capture output problem is at http://stackoverflow.com/a/7651522 - there is a .NET shim which can be used to launch a console in the correct way to capture UTF8 - see https://github.com/NeoSmart/UtfRedirect Ruby seems to call setmode with O_BINARY when initializing stdio? https://github.com/ruby/ruby/blob/6e6a8c02aff7207745808cdaa4b12e8f1e9ace82/win32/win32.c#L2504 Add Comment   This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.