Re: Generating missing depfiles by an automake based makefile
I finally went back to the top of the thread. Dmitry> Here is a rule from an automake generated makefile. Dmitry> Below is a sample bash session with gnu make which demonstrates how a Dmitry> dummy shuffle.Po makefile fails to have shuffle.o rebuilt when Dmitry> shuffle.h changes. Dmitry> $ rm src/.deps/shuffle.Po Just don't do this. Maybe it's possible in theory to deal with this scenario, but it seems to me that it's not worth putting much effort into it. Tom
Re: Generating missing depfiles by an automake based makefile
On Fri, 10 Feb 2023, Edward Welbourne wrote: Dmitry Goncharov (10 February 2023 00:24) wrote: When a depfile is missing (for any reason) the current automake makefile creates a dummy depfile. This seems misguided. Better to include $(wildcard $(DEPFILES)) This sounds like a GNU make feature. Automake-generated Makefiles do not rely on GNU make. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt
Re: Generating missing depfiles by an automake based makefile
Dmitry Goncharov (10 February 2023 00:24) wrote: > When a depfile is missing (for any reason) the current automake > makefile creates a dummy depfile. This seems misguided. Better to include $(wildcard $(DEPFILES)) instead, so there are no misleading depfiles lying around to cause make to fail to make them when needed. As long as depfiles are created as a side-effect of creating the target they describe, absence of the depfile usually implies absence of that target, so there's no need to know what it depends on because you need to recreate it anyway. A more robust version of that is, taking the example of .o files as the targets, to EXTANT_OBJS := $(wildcard $(OBJECTS)) DEPFILES := $(EXTANT_OBJS:%.o=%.dep) include $(DEPFILES) so that if a target does exist, we do insist on regenerating its depfile, even if that means regenerating the target itself. We don't know the target's dependencies, so we don't know that it is up to date, so this isn't unreasonable, for all that it might turn out to have been up to date, after all. > From that point on the user has to notice that make is no longer > tracking dependencies and their build is incorrect. The issue here is creation of dummy depfiles. Don't Do That. Eddy.
Re: Generating missing depfiles by an automake based makefile
Dmitry Goncharov wrote: On Thursday, February 9, 2023, Tom Tromey wrote: It's been a long time since I worked on automake, but the dependency tracking in automake is designed not to need to rebuild or pre-build dep files. Doing that means invoking the compiler twice, which is slow. Instead, automake computes dependencies as a side effect of compilation. The hello.Po example presented above computes depfiles as a side effect of compilation. Moreover, when hello.Po is absent that makefile compiles hello.o as a side effect of hello.Po computation. In total there is only one compilation. What is the scenario where you both end up with an empty depfile and a compilation that isn't out of date for some other reason? That seems like it shouldn't be possible. When a depfile is missing (for any reason) the current automake makefile creates a dummy depfile. From that point on the user has to notice that make is no longer tracking dependencies and their build is incorrect. I am asking if automake can be enhanced to do something similar to hello.Po example above, in those cases when make supports that. If I understand correctly, the problem here is that the depfile is both empty and current. If Automake could set the dummy depfile's mtime to some appropriate past timestamp (maybe the Makefile itself?), it would appear out-of-date immediately and therefore be remade, also rebuilding the corresponding object. A quick check of the POSIX manual finds that touch(1) accepts the '-r' option to name a reference file and can create a file. Could we simply use "touch -r Makefile $DEPFILE" to create depfiles when we need dummies? -- Jacob
Re: Generating missing depfiles by an automake based makefile
On Thursday, February 9, 2023, Tom Tromey wrote: > > > It's been a long time since I worked on automake, but the dependency > tracking in automake is designed not to need to rebuild or pre-build dep > files. Doing that means invoking the compiler twice, which is slow. > Instead, automake computes dependencies as a side effect of compilation. The hello.Po example presented above computes depfiles as a side effect of compilation. Moreover, when hello.Po is absent that makefile compiles hello.o as a side effect of hello.Po computation. In total there is only one compilation. > > What is the scenario where you both end up with an empty depfile and a > compilation that isn't out of date for some other reason? That seems > like it shouldn't be possible. > > When a depfile is missing (for any reason) the current automake makefile creates a dummy depfile. From that point on the user has to notice that make is no longer tracking dependencies and their build is incorrect. I am asking if automake can be enhanced to do something similar to hello.Po example above, in those cases when make supports that. Regards, Dmitry
Re: Generating missing depfiles by an automake based makefile
Dmitry> i am not looking forward to -include (even though -include is Dmitry> supported by bmake, gnu make and sun make). Dmitry> -include robs the user the error message should make fails to rebuild a depfile. Dmitry> i'd rather introduce rules to rebuild depfiles, as presented in the Dmitry> earlier email. It's been a long time since I worked on automake, but the dependency tracking in automake is designed not to need to rebuild or pre-build dep files. Doing that means invoking the compiler twice, which is slow. Instead, automake computes dependencies as a side effect of compilation. What is the scenario where you both end up with an empty depfile and a compilation that isn't out of date for some other reason? That seems like it shouldn't be possible. Tom
Re: Generating missing depfiles by an automake based makefile
On Thu, Feb 9, 2023 at 12:15 PM Jan Engelhardt wrote: > On Thursday 2023-02-09 22:53, Dmitry Goncharov wrote: > It would probably take a new m4 macro AM_MAKE_SILENT_INCLUDE that tests > for the availability of "-include", and if found, also somehow changes > the rest of the depfile logic to use absent-depfiles rather than > empty-depfiles. i am not looking forward to -include (even though -include is supported by bmake, gnu make and sun make). -include robs the user the error message should make fails to rebuild a depfile. i'd rather introduce rules to rebuild depfiles, as presented in the earlier email. regards, Dmitry
Re: Generating missing depfiles by an automake based makefile
On Thursday 2023-02-09 22:53, Dmitry Goncharov wrote: > >> If you try this with e.g. >> OpenBSD make, it will complain. > >That's why i asked those questions about portability. >Do i understand it correctly, that a need to support bmake forces >automake to abandon a good mechanism to rebuild depfiles? Maybe not. autoconf is big on text substitution, and there already is an AM_MAKE_INCLUDE m4 macro to test for the type of include directive. It would probably take a new m4 macro AM_MAKE_SILENT_INCLUDE that tests for the availability of "-include", and if found, also somehow changes the rest of the depfile logic to use absent-depfiles rather than empty-depfiles.
Re: Generating missing depfiles by an automake based makefile
On Thu, Feb 9, 2023 at 11:41 AM Jan Engelhardt wrote: > This is a GNU extension. Not only gnu. Also supported by sun make. > If you try this with e.g. > OpenBSD make, it will complain. That's why i asked those questions about portability. Do i understand it correctly, that a need to support bmake forces automake to abandon a good mechanism to rebuild depfiles? regards, Dmitry
Re: Generating missing depfiles by an automake based makefile
On Thursday 2023-02-09 22:33, Dmitry Goncharov wrote: > >> .Po file contents control when an .o file -- and thus also >> the .Po file itself -- is remade. >> If a .Po file has no practical content, there is no indication >> that it needs to be remade. > >Absence of the depfile is such an indication. >Here is a sample bash session which demonstrates how gnu make and sun >make are able to build a missing depfile. > >$ ls >hello.c hello.h makefile >$ cat makefile >all: hello.tsk >hello.tsk: hello.o >gcc-10 -o $@ $^ > >hello.o: hello.c hello.Po >gcc-10 -c hello.c -MD -MF hello.Po > >hello.Po: >gcc-10 -c hello.c -MD -MF hello.Po > >include hello.Po This is a GNU extension. If you try this with e.g. OpenBSD make, it will complain.
Re: Generating missing depfiles by an automake based makefile
On Wed, Feb 8, 2023 at 11:54 AM Jan Engelhardt wrote: > depfiles are created ahead of make so that the include command > in Makefiles succeeds (include-with-ignore is non-portable AFAIR). Atleast, gnu make and sun make can build included files. > depfiles are not specifically tracked; this is impossible, > due to a chicken-egg problem. Do you mean this is impossible when we have to support some specific make impl? > .Po file contents control when an .o file -- and thus also > the .Po file itself -- is remade. > If a .Po file has no practical content, there is no indication > that it needs to be remade. Absence of the depfile is such an indication. Here is a sample bash session which demonstrates how gnu make and sun make are able to build a missing depfile. $ ls hello.c hello.h makefile $ cat makefile all: hello.tsk hello.tsk: hello.o gcc-10 -o $@ $^ hello.o: hello.c hello.Po gcc-10 -c hello.c -MD -MF hello.Po hello.Po: gcc-10 -c hello.c -MD -MF hello.Po include hello.Po $ gmake gcc-10 -c hello.c -MD -MF hello.Po gcc-10 -o hello.tsk hello.o $ gmake gmake: Nothing to be done for 'all'. $ rm hello.Po $ gmake gcc-10 -c hello.c -MD -MF hello.Po gcc-10 -o hello.tsk hello.o $ gmake gmake: Nothing to be done for 'all'. $ rm hello.o hello.Po hello.tsk $ $ $ # this is sun make $ /bin/make gcc-10 -c hello.c -MD -MF hello.Po gcc-10 -o hello.tsk hello.o $ /bin/make $ rm hello.Po $ /bin/make gcc-10 -c hello.c -MD -MF hello.Po gcc-10 -o hello.tsk hello.o $ /bin/make $ In the case of gnu make the rule to build the depfile does not have to have a recipe and can be simplified to hello.Po: Which implementations of make does automake generate makefiles for? Can automake be enhanced to generate gnu make specific code to allow for depfiles to be rebuilt? Such enhancement can be conditional, that is, only when the generated makefile is supposed to be used with gnu make. Same for sun make. regards, Dmitry
Re: Generating missing depfiles by an automake based makefile
On Wednesday 2023-02-08 03:39, Dmitry Goncharov wrote: > >This rule restores a missing depfile file by creating a file with one >line '# dummy'. (Next version of automake will create an empty one). >There must have been a reason for generating such a depfile. depfiles are created ahead of make so that the include command in Makefiles succeeds (include-with-ignore is non-portable AFAIR). >However, this depfile fails dependency tracking. Once a depfile was >removed and recreated in this shape, it will no longer perform its >function. depfiles are not specifically tracked; this is impossible, due to a chicken-egg problem. .Po file contents control when an .o file -- and thus also the .Po file itself -- is remade. If a .Po file has no practical content, there is no indication that it needs to be remade.
Generating missing depfiles by an automake based makefile
Good morning. Here is a rule from an automake generated makefile. $(am__depfiles_remade): @$(MKDIR_P) $(@D) @echo '# dummy' >$@-t && $(am__mv) $@-t $@ This rule restores a missing depfile file by creating a file with one line '# dummy'. (Next version of automake will create an empty one). There must have been a reason for generating such a depfile. However, this depfile fails dependency tracking. Once a depfile was removed and recreated in this shape, it will no longer perform its function. Below is a sample bash session with gnu make which demonstrates how a dummy shuffle.Po makefile fails to have shuffle.o rebuilt when shuffle.h changes. $ make -j3 CFLAGS='-Wall -Wextra -ggdb -m64 -O0 -DMAKE_MAINTAINER_MODE=1' ./config.status --header=src/mkconfig.h Making all in lib make[1]: Entering directory '/home/dgoncharov/src/gmake/make/l64/lib' make all-recursive config.status: creating src/mkconfig.h config.status: src/mkconfig.h is unchanged make[2]: Entering directory '/home/dgoncharov/src/gmake/make/l64/lib' make[3]: Entering directory '/home/dgoncharov/src/gmake/make/l64/lib' make[3]: Nothing to be done for 'all-am'. make[3]: Leaving directory '/home/dgoncharov/src/gmake/make/l64/lib' make[2]: Leaving directory '/home/dgoncharov/src/gmake/make/l64/lib' make[1]: Leaving directory '/home/dgoncharov/src/gmake/make/l64/lib' Making all in po make[1]: Entering directory '/home/dgoncharov/src/gmake/make/l64/po' make[1]: Nothing to be done for 'all'. make[1]: Leaving directory '/home/dgoncharov/src/gmake/make/l64/po' Making all in doc make[1]: Entering directory '/home/dgoncharov/src/gmake/make/l64/doc' make[1]: Nothing to be done for 'all'. make[1]: Leaving directory '/home/dgoncharov/src/gmake/make/l64/doc' make[1]: Entering directory '/home/dgoncharov/src/gmake/make/l64' make[1]: Leaving directory '/home/dgoncharov/src/gmake/make/l64' $ ls -l src/shuffle.o -rw-r--r-- 1 dgoncharov wireshark 13K Feb 7 21:16 src/shuffle.o $ head src/.deps/shuffle.Po src/shuffle.o: ../src/shuffle.c /usr/include/stdc-predef.h \ ../src/makeint.h src/config.h ../src/../src/mkcustom.h lib/alloca.h \ /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/include/stddef.h \ ../src/gnumake.h /usr/include/sys/types.h /usr/include/features.h \ /usr/include/sys/cdefs.h /usr/include/bits/wordsize.h \ /usr/include/bits/long-double.h /usr/include/gnu/stubs.h \ /usr/include/gnu/stubs-64.h /usr/include/bits/types.h \ /usr/include/bits/timesize.h /usr/include/bits/typesizes.h \ /usr/include/bits/time64.h /usr/include/bits/types/clock_t.h \ /usr/include/bits/types/clockid_t.h /usr/include/bits/types/time_t.h \ $ grep shuffle.h src/.deps/*.Po src/.deps/file.Po: ../src/variable.h ../src/debug.h ../src/shuffle.h src/.deps/file.Po:../src/shuffle.h: src/.deps/implicit.Po: ../src/output.h ../src/commands.h ../src/shuffle.h /usr/include/assert.h src/.deps/implicit.Po:../src/shuffle.h: src/.deps/job.Po: ../src/variable.h ../src/os.h ../src/dep.h ../src/shuffle.h \ src/.deps/job.Po:../src/shuffle.h: src/.deps/main.Po: ../src/shuffle.h /usr/include/assert.h /usr/include/fcntl.h \ src/.deps/main.Po:../src/shuffle.h: src/.deps/shuffle.Po: /usr/include/glob.h ../src/shuffle.h ../src/filedef.h ../src/hash.h \ src/.deps/shuffle.Po:../src/shuffle.h: $ rm src/.deps/shuffle.Po $ make -j3 CFLAGS='-Wall -Wextra -ggdb -m64 -O0 -DMAKE_MAINTAINER_MODE=1' ./config.status --header=src/mkconfig.h Making all in lib make[1]: Entering directory '/home/dgoncharov/src/gmake/make/l64/lib' make all-recursive make[2]: Entering directory '/home/dgoncharov/src/gmake/make/l64/lib' config.status: creating src/mkconfig.h config.status: src/mkconfig.h is unchanged make[3]: Entering directory '/home/dgoncharov/src/gmake/make/l64/lib' make[3]: Nothing to be done for 'all-am'. make[3]: Leaving directory '/home/dgoncharov/src/gmake/make/l64/lib' make[2]: Leaving directory '/home/dgoncharov/src/gmake/make/l64/lib' make[1]: Leaving directory '/home/dgoncharov/src/gmake/make/l64/lib' Making all in po make[1]: Entering directory '/home/dgoncharov/src/gmake/make/l64/po' make[1]: Nothing to be done for 'all'. make[1]: Leaving directory '/home/dgoncharov/src/gmake/make/l64/po' Making all in doc make[1]: Entering directory '/home/dgoncharov/src/gmake/make/l64/doc' make[1]: Nothing to be done for 'all'. make[1]: Leaving directory '/home/dgoncharov/src/gmake/make/l64/doc' make[1]: Entering directory '/home/dgoncharov/src/gmake/make/l64' make[1]: Leaving directory '/home/dgoncharov/src/gmake/make/l64' $ head src/.deps/shuffle.Po # dummy $ touch ../src/shuffle.h $ make -j3 CFLAGS='-Wall -Wextra -ggdb -m64 -O0 -DMAKE_MAINTAINER_MODE=1' ./config.status --header=src/mkconfig.h Making all in lib make[1]: Entering directory '/home/dgoncharov/src/gmake/make/l64/lib' make all-recursive config.status: creating src/mkconfig.h config.status: src/mkconfig.h is unchanged make[2]: Entering directory '/home/dgoncharov/src/gmake/make/l64/lib' make[3]: Entering directory '/home/dgoncharov/src/gmake/make/l6
