[Bug lto/110710] LTO linker on Windows creates an invalid Makefile
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110710 Richard Biener changed: What|Removed |Added Known to work||15.0 --- Comment #17 from Richard Biener --- This has now been fixed on trunk, it's too late for 14.1 but we can backport later if the patch doesn't cause any issues.
[Bug lto/110710] LTO linker on Windows creates an invalid Makefile
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110710 --- Comment #16 from GCC Commits --- The master branch has been updated by Richard Biener : https://gcc.gnu.org/g:a704554d2e798e2e1b74b9fea4baf3477180bd9d commit r15-108-ga704554d2e798e2e1b74b9fea4baf3477180bd9d Author: Peter Damianov Date: Sun Apr 28 16:16:12 2024 -0700 lto-wrapper: Truncate files using -truncate driver option [PR110710] This commit changes the Makefiles generated by lto-wrapper to no longer use the "mv" and "touch" shell commands. These don't exist on Windows, so when the Makefile attempts to call them, it results in errors like: The system cannot find the file specified. This problem only manifested when calling gcc from cmd.exe, and having no sh.exe present on the PATH. The Windows port of GNU Make searches the PATH for an sh.exe, and uses it if present. I have tested this in environments with and without sh.exe on the PATH and confirmed it works as expected. Signed-off-by: Peter Damianov PR lto/110710 * lto-wrapper.cc (run_gcc): Instead of truncating a processed ltrans input from the Makefile use the new -truncate option to accomplish the same.
[Bug lto/110710] LTO linker on Windows creates an invalid Makefile
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110710 --- Comment #15 from GCC Commits --- The master branch has been updated by Richard Biener : https://gcc.gnu.org/g:985b5a90f70c7376c771317c6c8c3bc5ef05e227 commit r15-107-g985b5a90f70c7376c771317c6c8c3bc5ef05e227 Author: Peter Damianov Date: Sun Apr 28 16:16:11 2024 -0700 Driver: Add new -truncate option This commit adds a new option to the driver that truncates one file after linking. Tested likeso: $ gcc hello.c -c $ du -h hello.o 4.0K hello.o $ gcc hello.o -truncate hello.o $ ./a.out Hello world $ du -h hello.o $ 0 hello.o $ gcc hello.o -truncate gcc: error: missing filename after '-truncate' The motivation for adding this is PR110710. It is used by lto-wrapper to truncate files in a shell-independent manner. Signed-off-by: Peter Damianov PR lto/110710 * common.opt (truncate): New internal option. * gcc.cc (totruncate_file): New global. (driver_handle_option): Handle -truncate . (driver::final_actions): Truncate the file indicated.
[Bug lto/110710] LTO linker on Windows creates an invalid Makefile
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110710 Eric Gallager changed: What|Removed |Added Keywords||patch CC||egallager at gcc dot gnu.org URL||https://gcc.gnu.org/piperma ||il/gcc-patches/2024-March/6 ||48427.html Status|WAITING |ASSIGNED Assignee|unassigned at gcc dot gnu.org |peter0x44 at disroot dot org --- Comment #14 from Eric Gallager --- (In reply to Peter Damianov from comment #13) > https://gcc.gnu.org/pipermail/gcc-patches/2024-March/648427.html > > Patch submitted. Couldn't figure out how to assign myself in bugzilla. There's a privilege for it; I just assigned you.
[Bug lto/110710] LTO linker on Windows creates an invalid Makefile
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110710 --- Comment #13 from peter0x44 at disroot dot org --- https://gcc.gnu.org/pipermail/gcc-patches/2024-March/648427.html Patch submitted. Couldn't figure out how to assign myself in bugzilla.
[Bug lto/110710] LTO linker on Windows creates an invalid Makefile
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110710
--- Comment #12 from peter0x44 at disroot dot org ---
diff --git a/gcc/lto-wrapper.cc b/gcc/lto-wrapper.cc
index 5186d040ce0..7119157d830 100644
--- a/gcc/lto-wrapper.cc
+++ b/gcc/lto-wrapper.cc
@@ -2024,9 +2024,7 @@ cont:
truncate them as soon as we have processed it. This
reduces temporary disk-space usage. */
if (! save_temps)
- fprintf (mstream, "\t@-touch -r \"%s\" \"%s.tem\" > /dev/null "
-"2>&1 && mv \"%s.tem\" \"%s\"\n",
-input_name, input_name, input_name, input_name);
+ fprintf (mstream, "\t@-copy /y nul \"%s\"" input_name);
}
else
{
This experimental patch works for me, it doesn't seem to matter if the
timestamp gets updated. I guess what's left to do is write code that runs "sh
-c "true"" to determine if there is an sh.exe present, and then emit the
correct fragment to truncate the file.
[Bug lto/110710] LTO linker on Windows creates an invalid Makefile
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110710 --- Comment #11 from peter0x44 at disroot dot org --- .I did some digging into why lto-wrapper.cc is emitting these commands It seems that they are not essential. /* If we are not preserving the ltrans input files then truncate them as soon as we have processed it. This reduces temporary disk-space usage. */ if (! save_temps) fprintf (mstream, "\t@-touch -r \"%s\" \"%s.tem\" > /dev/null " "2>&1 && mv \"%s.tem\" \"%s\"\n", input_name, input_name, input_name, input_name); This is the only thing in the Makefile that actually needs the posix shell commands. When emitted, it looks like this: @-touch -r "/tmp/ccprwrlE.ltrans0.o" "/tmp/ccprwrlE.ltrans0.o.tem" > /dev/null 2>&1 && mv "/tmp/ccprwrlE.ltrans0.o.tem" "/tmp/ccprwrlE.ltrans0.o" I think what this is doing is copying the timestamp of the ".o" file onto a ".o.tmp", then renaming the ".o.tmp" to be the ".o". This is a truncation because the file is not supposed to exist in advance, so touch is creating it. So, the purpose is to truncate a file without updating its modified timestamp. I checked how this could be done in cmd, and it seems it is rather deficient here. There is no direct equivalent of "touch", but some commands can update the timestamp of files to the current time. There is no way to copy a timestamp from another file without using powershell it seems, which make can't use. So, the questions I'm left with are: Does it matter if the timestamp doesn't get preserved? Does it matter if the file doesn't get truncated? And lastly, is it possible to copy the timestamp of one file to another using cmd commands?
[Bug lto/110710] LTO linker on Windows creates an invalid Makefile
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110710 --- Comment #10 from peter0x44 at disroot dot org --- > For a workaround, I would recommend busybox-w32. It's what w64devkit uses. > You can just put it on your PATH and see if it solves that issue. Just to be completely explicit, what I mean to do is: Download busybox.exe from here: https://frippery.org/busybox/ Rename it to "sh.exe", and place it on your PATH.
[Bug lto/110710] LTO linker on Windows creates an invalid Makefile
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110710 --- Comment #9 from peter0x44 at disroot dot org --- That sent twice... sorry
[Bug lto/110710] LTO linker on Windows creates an invalid Makefile
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110710 --- Comment #8 from peter0x44 at disroot dot org --- The way make works is it prefers using an sh.exe if it finds it on the PATH. https://git.savannah.gnu.org/cgit/make.git/tree/src/variable.c#n1637 (see variable.c of GNU make) Otherwise, cmd.exe is used. This is probably why many people don't notice, as most "mingw" environments do have this. But "mingw" does not imply "has an sh.exe shell". You can totally add gcc to your cmd.exe PATH and use it fine, and you can also write Makefiles with GNU Make that expect to use cmd.exe to execute their build rules. If GCC's generated temporary Makefiles expect to run under a "posix sh" always, I believe they should have on the first line: SHELL = /bin/sh For a workaround, I would recommend busybox-w32. It's what w64devkit uses. You can just put it on your PATH and see if it solves that issue. It's what w64devkit uses, and I have used `-flto=auto` with it successfully before. As for actually reviewing this issue in the generated Makefiles, there are a few approaches that can be used. One of them it to utilize a "polyglot test" to tell which shell the build rules are being executed under. Something like: SHELLTYPE := posix ifeq ($(shell echo "test", "test")) SHELLTYPE := cmd endif This works because cmd.exe's echo command writes quotes. under cmd.exe echo "test" will write: "test" and under a posix: test (without quotes) Then, the Makefile can later do something like: ifeq (posix, $(SHELLTYPE)) gcc whatever.c 2>/dev/null else gcc whatever.c 2>NUL endif Now, this is an enormous HACK, but it worked okay in some cases, for me. I implemented it in a different Makefile generating project here: https://github.com/premake/premake-core/pull/2039 But is probably a terrible idea for GCC, since I believe ifeq is a GNU Make extension. So is $(shell), afaik. I'm only presenting this option for completeness. I think the better solution for this case would be for --host=*-w64-mingw32, write out Makefiles with: SHELL = cmd and then proceed to use all the cmd.exe shell equivalents, like 2>NUL, move instead of mv, etc. I've spent quite a bit of time before thinking and exploring this particular "problem space", so it's just my analysis here. Unfortunately, I think there are no "nice" ways to solve this problem.
[Bug lto/110710] LTO linker on Windows creates an invalid Makefile
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110710 peter0x44 at disroot dot org changed: What|Removed |Added CC||peter0x44 at disroot dot org --- Comment #7 from peter0x44 at disroot dot org --- The way make works is it prefers using an sh.exe if it finds it on the PATH. https://git.savannah.gnu.org/cgit/make.git/tree/src/variable.c#n1637 (see variable.c of GNU make) Otherwise, cmd.exe is used. This is probably why many people don't notice, as most "mingw" environments do have this. But "mingw" does not imply "has an sh.exe shell". You can totally add gcc to your cmd.exe PATH and use it fine, and you can also write Makefiles with GNU Make that expect to use cmd.exe to execute their build rules. If GCC's generated temporary Makefiles expect to run under a "posix sh" always, I believe they should have on the first line: SHELL = /bin/sh For a workaround, I would recommend busybox-w32. It's what w64devkit uses. You can just put it on your PATH and see if it solves that issue. It's what w64devkit uses, and I have used `-flto=auto` with it successfully before. As for actually reviewing this issue in the generated Makefiles, there are a few approaches that can be used. One of them it to utilize a "polyglot test" to tell which shell the build rules are being executed under. Something like: SHELLTYPE := posix ifeq ($(shell echo "test", "test")) SHELLTYPE := cmd endif This works because cmd.exe's echo command writes quotes. under cmd.exe echo "test" will write: "test" and under a posix: test (without quotes) Then, the Makefile can later do something like: ifeq (posix, $(SHELLTYPE)) gcc whatever.c 2>/dev/null else gcc whatever.c 2>NUL endif Now, this is an enormous HACK, but it worked okay in some cases, for me. I implemented it in a different Makefile generating project here: https://github.com/premake/premake-core/pull/2039 But is probably a terrible idea for GCC, since I believe ifeq is a GNU Make extension. So is $(shell), afaik. I'm only presenting this option for completeness. I think the better solution for this case would be for --host=*-w64-mingw32, write out Makefiles with: SHELL = cmd and then proceed to use all the cmd.exe shell equivalents, like 2>NUL, move instead of mv, etc. I've spent quite a bit of time before thinking and exploring this particular "problem space", so it's just my analysis here. Unfortunately, I think there are no "nice" ways to solve this problem.
[Bug lto/110710] LTO linker on Windows creates an invalid Makefile
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110710 Amyspark changed: What|Removed |Added CC||amy at amyspark dot me --- Comment #6 from Amyspark --- This still happens in GCC 13.2, and breaks standalone usage of GCC outside of a MSYS2 (or similar) shell. See my report (I did it way before finding this bug, will cross link now): https://github.com/StephanTLavavej/mingw-distro/issues/102
[Bug lto/110710] LTO linker on Windows creates an invalid Makefile
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110710 --- Comment #5 from Jan Nárovec --- We are using GNU/make (which is a superset of POSIX make, I don't know, whether it makes some difference) with SHELL=cmd.exe. If GCC intends to support only POSIX shells, it is OK for us (we will condsider using WSL) and you can close this issue as invalid.
[Bug lto/110710] LTO linker on Windows creates an invalid Makefile
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110710 --- Comment #4 from Andrew Pinski --- You cannot use posix make without a posix shell. That means /dev/null support too. https://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap10.html
[Bug lto/110710] LTO linker on Windows creates an invalid Makefile
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110710 --- Comment #3 from Jan Nárovec --- The commands (mv, touch) are not the main problem, these can be indeed somehow provided, but redirecion to /dev/null cannot work at all in the windows SHELL (cmd.exe). We use gcc for a cross compilation, primary development platform is MS/Windows and the target platform is ARM Cortex-R.
[Bug lto/110710] LTO linker on Windows creates an invalid Makefile
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110710 Andrew Pinski changed: What|Removed |Added Last reconfirmed||2023-07-18 Status|UNCONFIRMED |WAITING Ever confirmed|0 |1 --- Comment #2 from Andrew Pinski --- Even make is not normally on windows. So what env are you using to build programs? Gcc assumes simple commands are working in general.
[Bug lto/110710] LTO linker on Windows creates an invalid Makefile
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110710 --- Comment #1 from Andrew Pinski --- In a mingw env, I thought most of the posix simple posix commands are there.
