> From: Costas Argyris <costas.argy...@gmail.com> > Date: Sat, 18 Mar 2023 16:37:20 +0000 > > This is a proposed patch to enable UTF-8 support in GNU Make running on > Windows host.
Thanks. > Today, the make process on Windows is using the legacy system code page > because of the "A" functions > called in the source code. This means that any UTF-8 input to make on > Windows will break. A few > examples follow: Yes, this misfeature of using the system codepage is known, together with the consequences. > The attached patch incorporates the UTF-8 manifest into the build process of > GNU Make when hosted on > Windows, and forces the built executable to use UTF-8 as its active code > page, solving all problems shown > above because this has a global effect in the process. All existing "A" > calls use the UTF-8 code page now > instead of the legacy one. This is the relevant Microsoft doc: > > https://learn.microsoft.com/en-us/windows/apps/design/globalizing/use-utf8-code-page > > With the patch, after building make, the above cases now work on Windows: > > ###################### > C:\Users\cargyris\temp>cat utf8Makefile.mk > hello : > @echo ﹏ > @echo ❎ > C:\Users\cargyris\temp>make -f utf8Makefile.mk > ﹏ > ❎ > > C:\Users\cargyris\temp>make -f ❎\utf8Makefile.mk > ﹏ > ❎ > > C:\Users\cargyris\temp>cd ❎ > > C:\Users\cargyris\temp\❎>make -f utf8Makefile.mk > ﹏ > ❎ > > C:\Users\cargyris\temp\❎>make -f ❎\utf8Makefile.mk > ﹏ > ❎ > ###################### > > This change might also fix other existing issues on Windows having to do with > filenames and paths, but I > can't point at something particular right now. > > Would a patch like that be considered? Yes, of course. However, we need to understand better the conditions under which the UTF-8 support in Make will be activated, and the consequences of activating it. Here are some specific questions, based on initial thinking about this: . Does this support require Make to be linked against the UCRT run-time library, or does it also work with the older MSVCRT? If Make is built with MSVC, does it have to be built with some new enough version of Studio to have the necessary run-time support for this feature, or any version will do? . Does using UTF-8 as the active page in Make mean that locale-dependent C library functions will behave as expected? For example, what happens with character classification functions such as isalpha and isdigit, and what happens with functions related to letter-case, such as tolower and stricmp -- will they perform correctly with characters in the entire Unicode range? (This might be related to the first question above.) . Did you try running Make with this manifest on older Windows systems, like Windows 8.1 or 7? It is important to make sure this manifest doesn't preclude Make from running on those older systems, even though the UTF-8 feature will then be unavailable. . When Make invokes other programs (which it does quite a lot ;-), and passes command-line arguments to it with non-ASCII characters, what will happen to those non-ASCII characters? I'm guessing that if the program also has such a manifest, it will get the UTF-8 encoded strings verbatim, but what if it doesn't have such a manifest? (The vast majority of the programs Make invokes nowadays don't have such manifests.) Will Windows convert the UTF-8 encoded strings into the system codepage, or will the program get UTF-8 regardless of whether it can or cannot handle them? If the latter, it will become impossible to use non-ASCII strings and file names with such programs even if those non-ASCII characters can be represented using the current system ANSI codepage, because most programs Make invokes on Windows don't support UTF-8. Your examples invoked only the built-in commands of cmd.exe, but what happens if you instead invoke, say, GCC, and pass it a non-ASCII file name, including a file name which cannot be represented in the current ANSI codepage? . Even if the answer to the previous question is, as I expect, that Windows will convert UTF-8 encoded strings to the current ANSI codepage, it is important to understand that with the UTF-8 active codepage enabled Make will still be unable to invoke programs with UTF-8 encoded strings if those programs don't have the same UTF-8 active codepage enabled, except if the non-ASCII characters in those strings can be represented by the current ANSI codepage. So this feature will only be complete when the programs invoked by Make are also UTF-8 capable. A specific comment on your patch: > --- a/Makefile.am > +++ b/Makefile.am > @@ -46,6 +46,8 @@ w32_SRCS = src/w32/pathstuff.c src/w32/w32os.c > src/w32/compat/dirent.c \ > src/w32/subproc/misc.c src/w32/subproc/proc.h \ > src/w32/subproc/sub_proc.c src/w32/subproc/w32err.c It is OK to alter the general Makefile.am files, but please note that Make for Windows is canonically built using the build_w32.bat batch file; building using the Unix configury stuff is an option not currently directly supported by the project (although I believe it does work). So to be effective, these changes need to make the commands in that batch file to run windres as well, and the file README.W32 should mention the manifest installation alongside the executable. Note that build_w32.bat supports both MinGW GCC and MSVC builds, in two separate parts, and so the changes should affect both parts. Thanks again for your interest in GNU Make.
