> From: Costas Argyris <costas.argy...@gmail.com>
> Date: Sat, 18 Mar 2023 16:37:20 +0000
> This is a proposed patch to enable UTF-8 support in GNU Make running on 
> Windows host.


> Today, the make process on Windows is using the legacy system code page 
> because of the "A" functions
> called in the source code.    This means that any UTF-8 input to make on 
> Windows will break.    A few
> examples follow:

Yes, this misfeature of using the system codepage is known, together
with the consequences.

> The attached patch incorporates the UTF-8 manifest into the build process of 
> GNU Make when hosted on
> Windows, and forces the built executable to use UTF-8 as its active code 
> page, solving all problems shown
> above because this has a global effect in the process.    All existing "A" 
> calls use the UTF-8 code page now
> instead of the legacy one.    This is the relevant Microsoft doc:
> https://learn.microsoft.com/en-us/windows/apps/design/globalizing/use-utf8-code-page
> With the patch, after building make, the above cases now work on Windows:
> ######################
> C:\Users\cargyris\temp>cat utf8Makefile.mk
> hello :
>         @echo ﹏
>         @echo ❎
> C:\Users\cargyris\temp>make -f utf8Makefile.mk
> ﹏
> ❎
> C:\Users\cargyris\temp>make -f ❎\utf8Makefile.mk
> ﹏
> ❎
> C:\Users\cargyris\temp>cd ❎
> C:\Users\cargyris\temp\❎>make -f utf8Makefile.mk
> ﹏
> ❎
> C:\Users\cargyris\temp\❎>make -f ❎\utf8Makefile.mk
> ﹏
> ❎
> ######################
> This change might also fix other existing issues on Windows having to do with 
> filenames and paths, but I
> can't point at something particular right now.
> Would a patch like that be considered?

Yes, of course.

However, we need to understand better the conditions under which the
UTF-8 support in Make will be activated, and the consequences of
activating it.  Here are some specific questions, based on initial
thinking about this:

  . Does this support require Make to be linked against the UCRT
    run-time library, or does it also work with the older MSVCRT?  If
    Make is built with MSVC, does it have to be built with some new
    enough version of Studio to have the necessary run-time support
    for this feature, or any version will do?

  . Does using UTF-8 as the active page in Make mean that
    locale-dependent C library functions will behave as expected?  For
    example, what happens with character classification functions such
    as isalpha and isdigit, and what happens with functions related to
    letter-case, such as tolower and stricmp -- will they perform
    correctly with characters in the entire Unicode range?  (This
    might be related to the first question above.)

  . Did you try running Make with this manifest on older Windows
    systems, like Windows 8.1 or 7?  It is important to make sure this
    manifest doesn't preclude Make from running on those older
    systems, even though the UTF-8 feature will then be unavailable.

  . When Make invokes other programs (which it does quite a lot ;-),
    and passes command-line arguments to it with non-ASCII characters,
    what will happen to those non-ASCII characters?  I'm guessing that
    if the program also has such a manifest, it will get the UTF-8
    encoded strings verbatim, but what if it doesn't have such a
    manifest?  (The vast majority of the programs Make invokes
    nowadays don't have such manifests.)  Will Windows convert the
    UTF-8 encoded strings into the system codepage, or will the
    program get UTF-8 regardless of whether it can or cannot handle
    them?  If the latter, it will become impossible to use non-ASCII
    strings and file names with such programs even if those non-ASCII
    characters can be represented using the current system ANSI
    codepage, because most programs Make invokes on Windows don't
    support UTF-8.  Your examples invoked only the built-in commands
    of cmd.exe, but what happens if you instead invoke, say, GCC, and
    pass it a non-ASCII file name, including a file name which cannot
    be represented in the current ANSI codepage?

  . Even if the answer to the previous question is, as I expect, that
    Windows will convert UTF-8 encoded strings to the current ANSI
    codepage, it is important to understand that with the UTF-8 active
    codepage enabled Make will still be unable to invoke programs with
    UTF-8 encoded strings if those programs don't have the same UTF-8
    active codepage enabled, except if the non-ASCII characters in
    those strings can be represented by the current ANSI codepage.  So
    this feature will only be complete when the programs invoked by
    Make are also UTF-8 capable.

A specific comment on your patch:

> --- a/Makefile.am
> +++ b/Makefile.am
> @@ -46,6 +46,8 @@ w32_SRCS =  src/w32/pathstuff.c src/w32/w32os.c 
> src/w32/compat/dirent.c \
>               src/w32/subproc/misc.c src/w32/subproc/proc.h \
>               src/w32/subproc/sub_proc.c src/w32/subproc/w32err.c

It is OK to alter the general Makefile.am files, but please note that
Make for Windows is canonically built using the build_w32.bat batch
file; building using the Unix configury stuff is an option not
currently directly supported by the project (although I believe it
does work).  So to be effective, these changes need to make the
commands in that batch file to run windres as well, and the file
README.W32 should mention the manifest installation alongside the
executable.  Note that build_w32.bat supports both MinGW GCC and MSVC
builds, in two separate parts, and so the changes should affect both

Thanks again for your interest in GNU Make.

Reply via email to