Re: GNU make troubleshooting

[Alejandro Colomar]

On 2023-07-16 22:14, Philip Guenther wrote:
> I just put this at the very end of top makefile:
> 
> ${MAKEFILE_LIST}: ;

Hmm, that's giving me another good idea.  Luckily, I haven't pushed the
changes I had done yet.

Thanks!
Alex

> 
> Philip Guenther

-- 

GPG key fingerprint: A9348594CE31283A826FBDD8D57633D441E25BB5


OpenPGP_signature
Description: OpenPGP digital signature

Re: wildcard and globstar (was: GNU make troubleshooting)

[Alejandro Colomar]

Hi!

On 2023-07-16 18:31, Dmitry Goncharov wrote:
> On Sun, Jul 16, 2023 at 10:49 AM Alejandro Colomar  wrote:
>> I guess I'd need some kind of globstar support in GNU Make to be able
>> to use that
> 
> Do you mean you need to find .mk files in subdirectories at different
> depth levels?
> mk:=$(wildcard */*/*.mk */*.mk *.mk)

Yeah, that's a working workaround.  The issues I have with it are that
the prefix has to be repeated, so the line get overly long, and that it's
less robust (e.g., if I add a 4th level).

However, after the suggestion from Torbjörn, I've picked this workaround
to reduce the code.  It ended up being as small as this:

MK := \
$(srcdir)/Makefile \
$(wildcard $(MAKEFILEDIR)/*.mk $(MAKEFILEDIR)/*/*.mk 
$(MAKEFILEDIR)/*/*/*.mk)
include $(MK)
$(MK):: ;

I added an include guard to the main Makefile, to prevent a loop.

I'm still interested in globstar support, though.  It would make that
line shorter and more robust.  Maybe you could add an option
--glob=globstar or similar to enable support.

> 
> regards, Dmitry




On 2023-07-16 21:24, Torbjorn SVENSSON wrote:

[reordering your mail to answer to separate topics in it]

> On 2023-07-16 15:58, Alejandro Colomar wrote:

> I would have done something along these lines if I were in your shoes:
> 
> MK := $(srcdir)/Makefile
> MK += $(shell $(FIND) $(MAKEFILEDIR) -type f -name '*.mk')

> Why are you wasting time doing grep when you can have find return just 
> those files that ends with .mk?

I'm probably just wasting microseconds, so that's not a big deal.

grep(1) provides a common interface, namely regex, that I can rely on
to do the most complex stuff.

find(1) has a brain-damaged design[1], and I never remember all the
details about a given feature, so I would waste minutes reading the
manual.  That's a meaningful waste.  I decided long ago that I
wouldn't use most of find(1)'s features (I only use the ones which
can't be replaced by external commands in a pipe easily).

[1]: 

> Also, why need to sort the list when 
> it's only for make to define a rule?
> 

I admit in other cases it's more merited.  In this one, it was that I
didn't really think it much.  It still has a (very small) benefit,
which is that when debugging the Makefile, the targets of the rule
will appear sorted, which might be slightly better, and anyway,
sorting a few dozens of files won't take much either.

> Or, a more cleaner solution would have been to define the MK-array in 
> the $(srcdir)/Makefile as you have it hard coded for the include 
> statements now.
> 
> MK :=   \
> $(MAKEFILEDIR)/check/_.mk   \
> $(MAKEFILEDIR)/check/catman.mk  \
> $(MAKEFILEDIR)/build/_.mk   \
> $(MAKEFILEDIR)/build/catman.mk  \
> $(MAKEFILEDIR)/build/html.mk\
> $(MAKEFILEDIR)/build/pdf.mk \
> $(MAKEFILEDIR)/build/pre.mk \
> $(MAKEFILEDIR)/build/ps.mk  \
> $(MAKEFILEDIR)/build/src.mk \
> $(MAKEFILEDIR)/dist.mk  \
> $(MAKEFILEDIR)/install/_.mk \
> $(MAKEFILEDIR)/install/html.mk  \
> $(MAKEFILEDIR)/install/man.mk   \
> $(MAKEFILEDIR)/lint/_.mk\
> $(MAKEFILEDIR)/lint/c.mk\
> $(MAKEFILEDIR)/lint/man/_.mk\
> $(MAKEFILEDIR)/lint/man/man.mk  \
> $(MAKEFILEDIR)/lint/man/mdoc.mk \
> $(MAKEFILEDIR)/make.mk  \
> $(MAKEFILEDIR)/verbose.mk
> 
> include $(MK)

I didn't know I could include more than one file in a single include
statement.  I guess my C background didn't let me imagine that.  :)

I didn't consider that to be very clean, and so I've taken the parts
I like from your examples, which is the ability to include all files
from a variable, and set the variable with a wildcard.

Thank you both!

Cheers,
Alex

> $(srcdir)/Makefile $(MK):: ;
> 
> 
> Kind regards,
> Torbjörn
> 

-- 

GPG key fingerprint: A9348594CE31283A826FBDD8D57633D441E25BB5



OpenPGP_signature
Description: OpenPGP digital signature

Re: GNU make troubleshooting

[Philip Guenther]

I just put this at the very end of top makefile:

${MAKEFILE_LIST}: ;

Philip Guenther

On Sun, Jul 16, 2023 at 12:25 PM Torbjorn SVENSSON <
torbjorn.svens...@foss.st.com> wrote:

>
>
> On 2023-07-16 15:58, Alejandro Colomar wrote:
> > Wow!  That's a great help for debugging the Linux man-pages's makefiles,
> > which don't need to be remade.  It was very painful to me to ignore so
> > much make -d output.  I applied the following patch, and it works like
> > a charm.
> >
> > <
> https://git.kernel.org/pub/scm/docs/man-pages/man-pages.git/commit/?id=c98d237c22e9e898ae7d05ee7eac47791bd3
> >
>
> Why are you wasting time doing grep when you can have find return just
> those files that ends with .mk? Also, why need to sort the list when
> it's only for make to define a rule?
>
> I would have done something along these lines if I were in your shoes:
>
> MK := $(srcdir)/Makefile
> MK += $(shell $(FIND) $(MAKEFILEDIR) -type f -name '*.mk')
>
>
> Or, a more cleaner solution would have been to define the MK-array in
> the $(srcdir)/Makefile as you have it hard coded for the include
> statements now.
>
> MK :=   \
> $(MAKEFILEDIR)/check/_.mk   \
> $(MAKEFILEDIR)/check/catman.mk  \
> $(MAKEFILEDIR)/build/_.mk   \
> $(MAKEFILEDIR)/build/catman.mk  \
> $(MAKEFILEDIR)/build/html.mk\
> $(MAKEFILEDIR)/build/pdf.mk \
> $(MAKEFILEDIR)/build/pre.mk \
> $(MAKEFILEDIR)/build/ps.mk  \
> $(MAKEFILEDIR)/build/src.mk \
> $(MAKEFILEDIR)/dist.mk  \
> $(MAKEFILEDIR)/install/_.mk \
> $(MAKEFILEDIR)/install/html.mk  \
> $(MAKEFILEDIR)/install/man.mk   \
> $(MAKEFILEDIR)/lint/_.mk\
> $(MAKEFILEDIR)/lint/c.mk\
> $(MAKEFILEDIR)/lint/man/_.mk\
> $(MAKEFILEDIR)/lint/man/man.mk  \
> $(MAKEFILEDIR)/lint/man/mdoc.mk \
> $(MAKEFILEDIR)/make.mk  \
> $(MAKEFILEDIR)/verbose.mk
>
> include $(MK)
> $(srcdir)/Makefile $(MK):: ;
>
>
> Kind regards,
> Torbjörn
>
>

Re: GNU make troubleshooting

[Torbjorn SVENSSON]

On 2023-07-16 15:58, Alejandro Colomar wrote:

Wow!  That's a great help for debugging the Linux man-pages's makefiles,
which don't need to be remade.  It was very painful to me to ignore so
much make -d output.  I applied the following patch, and it works like
a charm.




Why are you wasting time doing grep when you can have find return just 
those files that ends with .mk? Also, why need to sort the list when 
it's only for make to define a rule?


I would have done something along these lines if I were in your shoes:

MK := $(srcdir)/Makefile
MK += $(shell $(FIND) $(MAKEFILEDIR) -type f -name '*.mk')


Or, a more cleaner solution would have been to define the MK-array in 
the $(srcdir)/Makefile as you have it hard coded for the include 
statements now.


MK :=   \
$(MAKEFILEDIR)/check/_.mk   \
$(MAKEFILEDIR)/check/catman.mk  \
$(MAKEFILEDIR)/build/_.mk   \
$(MAKEFILEDIR)/build/catman.mk  \
$(MAKEFILEDIR)/build/html.mk\
$(MAKEFILEDIR)/build/pdf.mk \
$(MAKEFILEDIR)/build/pre.mk \
$(MAKEFILEDIR)/build/ps.mk  \
$(MAKEFILEDIR)/build/src.mk \
$(MAKEFILEDIR)/dist.mk  \
$(MAKEFILEDIR)/install/_.mk \
$(MAKEFILEDIR)/install/html.mk  \
$(MAKEFILEDIR)/install/man.mk   \
$(MAKEFILEDIR)/lint/_.mk\
$(MAKEFILEDIR)/lint/c.mk\
$(MAKEFILEDIR)/lint/man/_.mk\
$(MAKEFILEDIR)/lint/man/man.mk  \
$(MAKEFILEDIR)/lint/man/mdoc.mk \
$(MAKEFILEDIR)/make.mk  \
$(MAKEFILEDIR)/verbose.mk

include $(MK)
$(srcdir)/Makefile $(MK):: ;


Kind regards,
Torbjörn

Re: wildcard and globstar (was: GNU make troubleshooting)

[Dmitry Goncharov]

On Sun, Jul 16, 2023 at 10:49 AM Alejandro Colomar  wrote:
> I guess I'd need some kind of globstar support in GNU Make to be able
> to use that

Do you mean you need to find .mk files in subdirectories at different
depth levels?
mk:=$(wildcard */*/*.mk */*.mk *.mk)

regards, Dmitry

wildcard and globstar (was: GNU make troubleshooting)

[Alejandro Colomar]

On 2023-07-16 16:17, Dmitry Goncharov wrote:
> Como estas, Alex?

Muy bien, y tu?  =)

> 
> On Sun, Jul 16, 2023 at 9:59 AM Alejandro Colomar  wrote:
>> I applied the following patch
> 
>> 
> 
> Glad you liked it.

:-)

> Rather than `find -type f...` (which is painfully slow) why don't you
> do something like
> mk:=$(wildcard $(MAKEFILEDIR)/*.mk)
> ?

I guess I'd need some kind of globstar support in GNU Make to be able
to use that (and AFAIK there's no support), since MAKEFILEDIR contains
this:

$ find
.
./lint
./lint/man
./lint/man/man.mk
./lint/man/_.mk
./lint/man/mdoc.mk
./lint/_.mk
./lint/c.mk
./version.mk
./check
./check/_.mk
./check/catman.mk
./compress.mk
./make.mk
./verbose.mk
./cmd.mk
./dist.mk
./src.mk
./build
./build/pre.mk
./build/pdf.mk
./build/groff.mk
./build/html.mk
./build/ps.mk
./build/_.mk
./build/src.mk
./build/catman.mk
./install
./install/man.mk
./install/html.mk
./install/_.mk


Now you mention it, it might be interesting to add support for it.

Cheers,
Alex

> 
> regards, Dmitry

-- 

GPG key fingerprint: A9348594CE31283A826FBDD8D57633D441E25BB5



OpenPGP_signature
Description: OpenPGP digital signature

Re: GNU make troubleshooting

[Dmitry Goncharov]

Como estas, Alex?

On Sun, Jul 16, 2023 at 9:59 AM Alejandro Colomar  wrote:
> I applied the following patch

> 

Glad you liked it.
Rather than `find -type f...` (which is painfully slow) why don't you
do something like
mk:=$(wildcard $(MAKEFILEDIR)/*.mk)
?

regards, Dmitry

Re: GNU make troubleshooting

[Alejandro Colomar]

Hi Dmitry!

On 2023-07-15 17:28, Dmitry Goncharov wrote:
[...]

> 
> $ make -d |tail +7 |head
> Reading makefiles...
> Reading makefile 'makefile'...
> Updating makefiles
>  Considering target file 'makefile'.
>   Looking for an implicit rule for 'makefile'.
>Trying pattern rule '%: %.o' with stem 'makefile'.
>Trying implicit prerequisite 'makefile.o'.
>Not found 'makefile.o'.
>Trying pattern rule '%: %.c' with stem 'makefile'.
>Trying implicit prerequisite 'makefile.c'.
> $
> 
> Upon startup make looks for a rule to update the makefile. (see How
> Makefiles Are Remade). This is what the initial portion of this debug
> output is about. If we don’t need your makefile to be remade, we can
> instruct make to skip this update process.
> 
> $ cat makefile
> all: hello
> makefile::;
> $ make -d |tail +7 |wc
> 7304101   32848
> 
> Note, we added rule
> 
> makefile::;
> 
> This addition cut the debug output from 1338 to 730 lines.

Wow!  That's a great help for debugging the Linux man-pages's makefiles,
which don't need to be remade.  It was very painful to me to ignore so
much make -d output.  I applied the following patch, and it works like
a charm.



Thanks a lot!
Alex

-- 

GPG key fingerprint: A9348594CE31283A826FBDD8D57633D441E25BB5


OpenPGP_signature
Description: OpenPGP digital signature

Re: GNU make troubleshooting

[Dmitry Goncharov]

On Sat, Jul 15, 2023 at 5:27 PM Bruno Haible  wrote:
> I believe these two user goals are so different; they belong in different
> chapters.
i believe, a good approach to big debug output is to modify the
makefile to relieve make from doing redundant work.
Once make only does necessary minimal work only, the debug output is
also necessary and minimal.
In other words, i don't see a chapter on how to avoid big debug output
without optimizing the makefile.


> I believe that 80%-90% of the developers, when they see
> 1000 lines of debug/trace output, give up understanding it after 5 seconds
> and try alternative approaches. (This is based on observing the habits
> of my developer colleagues at work.)

i agree.


> > > Can the addition of 'makefile::;' be replaced by a make option or
> > > by some (sed-based?) postprocessing?
> >
> > There is no option.
> > You can grep away most of the matching lines with 'grep -v makefile'.
>
> Then it's useful to present this as a filter in the troubleshooting
> section.

i'd rather have the manual to teach how to modify the makefile to
relieve make from doing redundant work.
i am puzzled why you prefer working around with options and filters.
Are you thinking about a scenario where you cannot modify the makefile?
Another approach for the case when you cannot modify the makefile is this

$ cat makefile
all: hello
$ make -nd |wc
   13347530   63382
$ make -E $'.DEFAULT_GOAL:=all\nmakefile::;' -nd |wc
7324117   32947

regards, Dmitry