On Sun, 2023-06-11 at 12:29 +0000, Zoltán Turányi wrote: >> Couldn’t these be handled by creating a "namespace" concept for >> rules? One could merge 2 Makefiles by prefixing each target with the >> directory of the Makefile. This would make dir1/all different from >> dir2/all. Also in recipes variables, pattern rules could be used from >> the Makefile describing the recipe. Would this not be possible (in >> theory, at least)? >
On Jun 22, 2023, at 2:41 PM, Paul Smith <psm...@gnu.org> wrote: > Sure, but obviously it's not sufficient because some makefile rules DO > want to refer targets in other makefiles / directories. I want my > target to depend on the "libfoo.a" target from some other directory. > How does that work? If you want to create multiple makefiles and specify a "current namespace for files in a directory's makefile", and you're using GNU make, that's easily done by using this construct in the makefile fragment in each directory: DIR := $(shell pwd) Below is a sample of what this would look like. I'm using ".mk" as the extension for fragments. A real makefile should generate the dependencies, not hand-jam them, but hopefully you get the idea. If you *need* portability with BSD make as well, you could use this instead: DIR != pwd ... but if any directory includes "$" you're in trouble :-). POSIX make is very minimal & it's usually better to just say "must use GNU make". If you always run "make" from the root directory, this should work fine. It should take less than a second for the no-work case even in larger systems (e.g., 100K files or so being managed as dependencies). A user *could* run "make" in just a subdirectory as long as there are never cross-directory dependencies, using this approach. However, it will end BADLY if there were EVER cross-directory dependencies AND a user ran "make" using only the makefile in a *subdirectory*. In my experience, the cross-directory dependencies WILL eventually happen. So it's better to NEVER name them "makefile" (so users won't accidentally screw up building), and instead name these makefile fragments with a ".mk" extension. Create your makefiles in each directory, and run "make" at the top directory, where its "makefile" transitively includes the others. If it takes more than than a second to run "do-nothing" at the top dir and you have <100K files, then there's a serious problem that needs fixing. If you think you *have* to run make in a way that it can *only* see the dependencies of a subdirectory, I think you're optimizing the wrong problem. Make is *very* fast at parsing makefiles & at checking file timestamps. But if you give Make the wrong data, it will do the wrong thing. You may then try to "fix" it by overspecifying things, leading to complex & unmaintainable makefiles that will be slow & wrong. --- David A. Wheeler ~~~~~~~~~~~~~~~~~~~~~~~~~~~ === ./makefile === # Demo getting current directory for use in file references DIR := $(shell pwd) $(DIR)/result: $(DIR)/result.o $(DIR)/subdir/bar.o clean: rm -f *.o */*.o include subdir/fragment.mk === ./subdir/fragment.mk === DIR := $(shell pwd) $(DIR)/bar.o: $(DIR)/bar.c === ./subdir/bar.c === #include <stdio.h> void bar(void) { printf("In bar.\n"); } === ./result.c === #include <stdio.h> extern void bar(void); int main(void) { printf("In main.\n"); bar(); return 0; } > >> My problem is that contrary to the make wisdom of writing a single >> Makefile (to which I agree) most projects are still divided into >> parts with separate build definitions. One can debate if this is good >> or bad - for me it is a requirement to handle this case efficiently. >> (If you do not agree there is no point in discussing further.) > > I wouldn't say "requirement", especially in conjunction with > "efficiently". If people want "more efficient" they may have to be > willing to pay the up-front cost of changing their makefiles and I > think that's a defensible position. > > But you are basically saying what I said in my previous email: the only > way it makes sense to try to implement something like this in GNU Make, > is if it could be done in such a way that people DON'T have to modify > their existing makefiles. > > Because if they have to modify their makefiles anyway they might as > well just do the work to make them non-recursive in the first place. > >>> However, I think this will be extremely difficult, because makefile >>> targets are so free-form and prerequisites and targets are not >>> actually necessary files at all. >> > > Also, how can you replace these target references if they appear inside > a recipe? Make can't parse shell script content in recipes (if they > are even shell scripts at all; SHELL could be /usr/bin/perl or > something). You can't just go through all the recipe text and replace > instances of the target "foo" with "dir1/foo". > > We couldn't use a simple namespace naming convention like you suggest > of prefixing the directory because targets in the current makefile > could already be named "dir1/all" for example. So you have to find a > way to talk about namespaces which don't conflict with existing naming. > > Also what about things like make recipes that invoke recursive make, > _inside the same directory_. Automake-generated makefiles do this all > the time. If the namespace is based solely on the containing directory > is that sufficient? Often the recursive invocation of make overrides > variables on the command line, so it seems like using the same > "namespace" wouldn't work. > > And remember the premise here is that all these things have to work > WITHOUT any changes to makefiles (or Makefile.am files for automake) > themselves. > > I'm not saying I know for a fact that it couldn't be done. But it > definitely seems to me like there are lots of issues that have to be > considered and worked out, and I'm not really sure that it would really > be that much more efficient anyway. The real efficiency gain would be > from: > > (a) Not invoking multiple make processes at all... but the proposal on > the table is that we'd still invoke the make process, and it would > still parse its makefile, it would just ship the post-parsed content > back to the parent make, so that's at best equivalently efficient and > almost certainly slightly slower; and > > (b) Allowing make to have a "global view" of all the rules and targets > so it can more efficiently run things in parallel... but here we're > talking about creating multiple namespaces anyway so can we really have > these multiple namespaces to separate target AND have this global view > of targets for better parallelism at the same time? > > Probably what needs to happen to help answer these questions is for > someone (but, not me :)) to take a sample automake-enabled software > package, look at the makefiles generated by automake for that package, > and come up with a concrete proposal for how it might work and some > comments on how the result would be more efficient. >