On 06/09/18 10:10, Jan Beulich wrote: >>>> On 06.09.18 at 09:34, <jgr...@suse.com> wrote: >> On 05/09/18 16:05, Juergen Gross wrote: >>> I think we have a major problem in our build system regarding >>> automatic dependencies. >>> >>> Starting with a new tree (after git clone or make clean) I have >>> no dependency files (*.d2) anywhere: >>> >>> $ make clean >>> $ find . -name '*.d2' | wc -l >>> 0 >>> >>> Doing "make" will produce only some of them in a very limited number of >>> directories: >>> >>> $ make >>> $ find . -name '*.d2' | wc -l >>> 279 >>> $ find . -name '*.d2' | xargs -n 1 dirname | sort -u >>> ./tools/firmware/xen-dir/xen-root/xen/arch/x86 >>> ./tools/firmware/xen-dir/xen-root/xen/arch/x86/x86_64 >>> ./tools/firmware/xen-dir/xen-root/xen/common >>> ./tools/firmware/xen-dir/xen-root/xen/common/compat >>> ./xen/arch/x86 >>> ./xen/arch/x86/x86_64 >>> ./xen/common >>> ./xen/common/compat >>> >>> And only after the next "make" we have all of the *.d2 files available: >>> >>> $ make >>> $ find . -name '*.d2' | wc -l >>> 969 >>> >>> I guess the reason for that is that the *.d2 files only depend on the >>> *.d files which are built together with the *.o files. They are not >>> needed for building the product files, so they seem to be always one >>> make step behind. >>> >>> Now comes the weird part (that's why I started to look into this): >>> When I now do: >>> >>> $ touch tools/libxc/include/xenctrl.h >>> $ make >>> >>> I get build failures in tools/tests/depriv (lots of unknown types like >>> uint64_t). The .d file in the depriv directory suddenly contains only >>> 7 instead of 44 entries, while the .d2 file is still okay (it is one >>> make "older", do you remember?): >>> >>> $ wc .*.d* >>> 7 14 546 .depriv-fd-checker.d >>> 44 88 1923 .depriv-fd-checker.d2 >>> >>> And doing another make will succeed again, but now both dependency files >>> are cut down to 7 entries (sure they are: .d2 is rebuilt from .d, while >>> .d would only be rebuilt if .o is being built, which isn't happening as >>> no file it depends on has changed): >>> >>> $ make >>> $ wc .*.d* >>> 7 14 546 .depriv-fd-checker.d >>> 7 14 336 .depriv-fd-checker.d2 >>> >>> I have no immediate idea how to solve that. >> >> I've setup a little example Makefile solving the problem (just to show >> the correct dependencies, needs to be adapted for naming the .d and .d2 >> files and how to build the .d2): >> >> -->8 snip here 8<-- >> >> DEPS := tst.d2 >> >> all: tst $(DEPS) > > -include $(DEPS) already ought to have the effect of such a dependency, > since all makefiles are checked for rules of how to re-make them.
Obviously this isn't the case. Otherwise there would be .d2 files more common after doing a make. > >> %.d2: %.d >> cp $< $@ > > Such a rule already exists in ./Config.mk. Right, that was just needed in my little test Makefile. > >> %.o %.d: %.c >> gcc -MMD -o $(patsubst %.c,%.o,$<) -c $< > > Doesn't this result in gcc to be invoked twice, perhaps resulting in > corrupt .o and/or .d? I think %.d wants to depend on %.o, without > a command. No, that's perfectly fine. make will invoke the command only once, its just not clear for which target (that's the reason I need to use the $(patsubst %.c,%.o,$<) instead of $@, which might be the .o _or_ the .d file). From the make docs: Pattern rules may have more than one target. Unlike normal rules, this does not act as many different rules with the same prerequisites and recipe. If a pattern rule has multiple targets, make knows that the rule’s recipe is responsible for making all of the targets. The recipe is executed only once to make all the targets. > >> %: %.o >> gcc $< -o $@ >> >> -include $(DEPS) >> >> -->8 snip here 8<-- >> >> So the basic ideas are: >> >> - add a rule for constructing the .d files >> - let the build depend on the .d2 files > > IOW I wonder whether this really is any different from what we > do now (minus bugs/quirks in make itself, of course). And from this > as well as your original mail I still don't understand what's actually > broken with the current approach. The main problem is that the .d2 files used for determining which object files need to be (re-)built are based on the build before the last one. I'm not sure this is always the case, but at least when starting with a clean tree I need two invocations of "make" to get all .d2 files built. Juergen _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
