Re: Goodbye to GNU make's "build.sh" ... ?

[Paul Smith]

On Sat, 2022-06-25 at 17:47 -0400, Paul Smith wrote:
> This leaves me with two options:
> 
>    1. Stop using gnulib, or at least sharply limit the modules we
> will include to those with trivial-enough configurations.
>    2. Abandon the build.sh script and require an existing make
> program in order to build a new version of GNU make.

What I ended up doing is importing parts of gnulib, but customizing the
imports to avoid explosions of secondary-effect modules being pulled in
and not needing a lot of extra machinery in the Makefiles.

I created a separate branch gnulib-port which contains unmodified
versions of gnulib modules, then merged them in and modified them. 
This should allow us to make use of git's merge facilities to import
newer gnulib versions more easily.

So far so good.

Re: Goodbye to GNU make's "build.sh" ... ?

[David A. Wheeler]

> On Jun 27, 2022, at 9:48 AM, Paul Smith  wrote:
> 
> As mentioned, the goal of "build.sh" is to allow systems without an
> already-existing make program to bootstrap themselves, by providing a
> way to build GNU make without using "make".

I think it's important that there be *a* way to rebuild
to support "bootstrappable builds"  
and diverse double-compiling (DDC) .
The overall goal would be to counter the "trusting trust" attack.
Being able to rebuild using another existing "make" is an obvious way to do 
that.

I'm not so picky on *how* to do that.

--- David A. Wheeler

Re: Goodbye to GNU make's "build.sh" ... ?

[Paul Smith]

On Sun, 2022-06-26 at 22:00 -0400, Dmitry Goncharov wrote:
> What was the original driving force to introduce build.sh?

As mentioned, the goal of "build.sh" is to allow systems without an
already-existing make program to bootstrap themselves, by providing a
way to build GNU make without using "make".

> i think, people use shell globbing much more often than $(wildcard)
> or glibc's glob impl.  bash's glob impl is derived from the same old
> rms impl and has the bug fixed. And this allows the situation to
> continue.

Sorry I think I lost the thread here.

Just to remind, two things: first, the glob issue is not the primary
one; even if we leave glob the same way it is right now we'll still
have the build.sh problem.

And second, today GNU make uses its built-in glob ONLY on systems which
don't provide a GNU version of glob in their libc.  So for any system
which uses GNU libc, we are using the libc version of glob and whatever
bugs it contains.

Re: Goodbye to GNU make's "build.sh" ... ?

[Dmitry Goncharov]

On Sun, Jun 26, 2022 at 5:48 PM Paul Smith  wrote:
> Seems like it's not so much that the patch was rejected; indeed they
> seem to agree there's a bug.  But they wanted more work on it and it
> sort of fell apart.  I have no idea how much effort it would be to push
> hard enough to get a change merged.


i think, people use shell globbing much more often than $(wildcard) or
glibc's glob impl.
bash's glob impl is derived from the same old rms impl and has the bug
fixed. And this allows the situation to continue.

regards, Dmitry

Re: Goodbye to GNU make's "build.sh" ... ?

[Dmitry Goncharov]

On Sunday, June 26, 2022, Paul Smith  wrote
>
>
> I would prefer to have that fixed in gnulib, than continue to maintain
> a local copy just to avoid this issue.


Then there is no choice to drop gnulib and build.sh should go.
Since you asked for opinions, my opinion is that maintaining a mini make
along with the real make can complicate the release procedure.
It is not clear that anybody would benefit from a mini make. What was the
original driving force to introduce build.sh?





> Seems like it's not so much that the patch was rejected; indeed they
> seem to agree there's a bug.  But they wanted more work on it and it
> sort of fell apart.  I have no idea how much effort it would be to push
> hard enough to get a change merged.
>
>
>
>
>

Re: Goodbye to GNU make's "build.sh" ... ?

[Paul Smith]

On Sun, 2022-06-26 at 16:57 -0400, Dmitry Goncharov wrote:
> Have you considered to to avoid glob from gnulib? Make has its one
> impl of glob which it uses on certain systems.
> We could use this impl and maintain it on all systems.

While the original issue came up WRT glob, in actuality the problem is
far deeper.  The later versions of gnulib will pull significant content
for the findprog-in module, for example, which we do rely on quite
significantly in the GNU make fastpath.

However my plan was to preserve this internal version of glob and use
it for systems where we couldn't use the gnulib version, such as on
Windows.

> Aside from dependencies, one problem with the gnulib impl of glob is
> that it has a bug on some filesystems.

I would prefer to have that fixed in gnulib, than continue to maintain
a local copy just to avoid this issue.

Seems like it's not so much that the patch was rejected; indeed they
seem to agree there's a bug.  But they wanted more work on it and it
sort of fell apart.  I have no idea how much effort it would be to push
hard enough to get a change merged.

Re: Goodbye to GNU make's "build.sh" ... ?

[Dmitry Goncharov]

On Sun, Jun 26, 2022 at 10:11 AM Paul Smith  wrote:
> This really came up because I was trying to find a way to include the
> latest gnulib globbing library.  Adding "glob" to this pulls in a HUGE
> number of prerequisites.

Have you considered to to avoid glob from gnulib? Make has its one
impl of glob which it uses on certain systems.
We could use this impl and maintain it on all systems.
Aside from dependencies, one problem with the gnulib impl of glob is
that it has a bug on some filesystems.
i submitted a patch to both glibc and gnulib.
https://sourceware.org/pipermail/libc-alpha/2017-November/088984.html
and
https://lists.gnu.org/archive/html/bug-gnulib/2019-11/msg6.html.
The code is complex enough that review requires significant effort and
the patch is not merged.
The result is that the gnulib impl, which is supposed to be preferred,
is the only one that has the bug. Code like this
$(wildcard lib/ src/)
works on any system other than gnu (again on some filesystems).

regards, Dmitry

Re: Goodbye to GNU make's "build.sh" ... ?

[Howard Chu via Bug reports and discussion for GNU make]

Herring, Daniel - 0447 - MITLL wrote:
> On Jun 26, 2022, at 3:30 PM, Eli Zaretskii  wrote:
> 
> How do people bootstrap systems nowadays?  Is it only via
> cross-compilation?
> 
> 
> Another way to bootstrap is to have a native tool chain that is less 
> capable…. For example, an embedded device may come with an old or proprietary 
> shell and compiler.

But it would still include something resembling "make" - embedded systems 
toolkits have shipped with clones of make since the 1980s.

I'm not convinced that there are any notable systems out there capable of 
supporting gmake that don't already have a make utility.
Or that telling anyone so unlucky "first bootstrap with gmake 4.3" is a bad 
answer.

-- 
  -- Howard Chu
  CTO, Symas Corp.   http://www.symas.com
  Director, Highland Sun http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/

Re: Goodbye to GNU make's "build.sh" ... ?

[Herring, Daniel - 0447 - MITLL]

On Jun 26, 2022, at 3:30 PM, Eli Zaretskii  wrote:

How do people bootstrap systems nowadays?  Is it only via
cross-compilation?


Another way to bootstrap is to have a native tool chain that is less capable…. 
For example, an embedded device may come with an old or proprietary shell and 
compiler.

— Daniel 

smime.p7s
Description: S/MIME cryptographic signature

Re: Goodbye to GNU make's "build.sh" ... ?

[Eli Zaretskii]

> From: Paul Smith 
> Cc: bug-make@gnu.org
> Date: Sun, 26 Jun 2022 15:06:36 -0400
> 
> > No, I'm talking about not using Gnulib only for stage-1 build, the
> > one that produces a Make binary capable of building the full
> > version.  I was not talking about avoiding to use Gnulib for the
> > final build whose results are installed.
> 
> This is sort of what Sven was suggesting.  It could work, but configure
> does a lot of things: not only does it generate a config.h but it also
> finds compilers, figures out what options they accept, etc.  Obviously
> we don't need to do all that for a minimal build, but still the Windows
> ecosystem is much simpler because it's so homogeneous; even then
> build_w32.bat doesn't support building with Clang for Windows, for
> example.  Generating a config.h on any random system is much harder
> than having one pre-defined for the Windows32 API.

But this part was already supported by the bootstrap process, right?
So maybe we will need 2 configure scripts: one that only configures
for build.sh (and doesn't configure Gnulib), and the other which does
everything.

> The goal of this "mini-make" must be to be able to compile automake-
> generated makefiles (since that's what's needed in order to work with a
> gnulib-enabled system).  Or at least, the Makefile.am that GNU make and
> its gnulib modules use.
> 
> Since POSIX make does support wildcard syntax (e.g., "foo: *.c" is
> legal) we would need glob/fnmatch of some sort, unless we can guarantee
> that automake won't use it, for example.

Using wildcards in Makefiles is pretty rare.  Do we have that now?

> There's some amount of disingenuity here since make is really just the
> top of the iceberg when it comes to bootstrapping: you can't run
> configure without a full libc, compiler, coreutils etc. and how can you
> build those without a make?  So either we're talking about cross-
> compilation, or else this "minimake" would need to be sufficient to
> build more than just make itself.

How do people bootstrap systems nowadays?  Is it only via
cross-compilation?

Re: Goodbye to GNU make's "build.sh" ... ?

[Paul Smith]

On Sun, 2022-06-26 at 17:39 +0300, Eli Zaretskii wrote:
> > The problem is that the process for "make-less" systems in the past
> > has been: run configure, then run build.sh.  But we won't be able
> > to use the environment created by configure in this "new" build.sh,
> > because it works in tandem with the makefiles.  The generated
> > config.h etc. will assume lots of things that simply aren't true
> > anymore, since we don't have make.
> 
> I thought of making build.sh compile and link make without any
> configure step, similar to what build_w32.bat does.
> 
> > So to me this is equivalent to my option #1, don't use gnulib at
> > all (or at least, use it incredibly sparingly).
> 
> No, I'm talking about not using Gnulib only for stage-1 build, the
> one that produces a Make binary capable of building the full
> version.  I was not talking about avoiding to use Gnulib for the
> final build whose results are installed.

This is sort of what Sven was suggesting.  It could work, but configure
does a lot of things: not only does it generate a config.h but it also
finds compilers, figures out what options they accept, etc.  Obviously
we don't need to do all that for a minimal build, but still the Windows
ecosystem is much simpler because it's so homogeneous; even then
build_w32.bat doesn't support building with Clang for Windows, for
example.  Generating a config.h on any random system is much harder
than having one pre-defined for the Windows32 API.

The goal of this "mini-make" must be to be able to compile automake-
generated makefiles (since that's what's needed in order to work with a
gnulib-enabled system).  Or at least, the Makefile.am that GNU make and
its gnulib modules use.

Since POSIX make does support wildcard syntax (e.g., "foo: *.c" is
legal) we would need glob/fnmatch of some sort, unless we can guarantee
that automake won't use it, for example.

We can hopefully disable other advanced features such as ar parsing,
job control, etc. which will certainly help simplify things.

There's some amount of disingenuity here since make is really just the
top of the iceberg when it comes to bootstrapping: you can't run
configure without a full libc, compiler, coreutils etc. and how can you
build those without a make?  So either we're talking about cross-
compilation, or else this "minimake" would need to be sufficient to
build more than just make itself.

Re: Goodbye to GNU make's "build.sh" ... ?

[Eli Zaretskii]

> From: Paul Smith 
> Date: Sun, 26 Jun 2022 10:11:12 -0400
> 
> > Can you list the Gnulib functions we currently use?
> 
> You can find them in the bootstrap.conf file:
> 
> https://git.savannah.gnu.org/cgit/make.git/tree/bootstrap.conf
> 
> See the list at the end.

OK, thanks.  The list is very short.  And I think most if not all of
the functions there can be stubbed out.  For example, strtoll can be
stubbed by a trivial wrapper around strtol, mempcpy can be easily
implemented using memcpy, and getloadavg and glob could be replaced by
functions that return trivial results, since building Make doesn't
really need those functions, does it?

> > Perhaps we could then prepare a fake-gnulib module with trivial
> > implementations of those functions, which could be used by build.sh
> > instead of the real Gnulib functions.
> 
> This seems odd to me: I mean, isn't that just replacing gnulib with
> what it already is?

No, because we only need this to build the initial binary of Make,
which then could be used to build the full version, and that full
version will use Gnulib.  So the idea is to create a small file with
trivially simple implementation for functions that might be missing,
and for the rest rely on libc -- just for the first "stage-1",
so-to-say, binary of Make.  Then "stage 2" will use this stage-1
binary to configure and build the full version which uses Gnulib.

> The point of gnulib is to provide replacements on systems that don't
> provide these.

That's not the only purpose, perhaps not only the main one.  The main
one (AFAIU) is to replace library functions with versions that are
either devoid of known bugs or are more compatible to Posix and glibc.

> > (Some Gnulib functions are replacements for those found in every C
> > library, and those could be simply ignored in the build.sh build,
> > relying on libc to do a reasonably good job.)
> 
> Yes, we could say that build.sh can only be used on systems which are
> essentially POSIX-compliant and don't need a lot of fixup.

I think it's more than that: to build Make we need much less than full
Posix compliance.

> The problem is that the process for "make-less" systems in the past has
> been: run configure, then run build.sh.  But we won't be able to use
> the environment created by configure in this "new" build.sh, because it
> works in tandem with the makefiles.  The generated config.h etc. will
> assume lots of things that simply aren't true anymore, since we don't
> have make.

I thought of making build.sh compile and link make without any
configure step, similar to what build_w32.bat does.

> So to me this is equivalent to my option #1, don't use gnulib at all
> (or at least, use it incredibly sparingly).

No, I'm talking about not using Gnulib only for stage-1 build, the one
that produces a Make binary capable of building the full version.  I
was not talking about avoiding to use Gnulib for the final build whose
results are installed.

Re: Goodbye to GNU make's "build.sh" ... ?

[Paul Smith]

On Sun, 2022-06-26 at 07:34 +0100, Sven C. Dack wrote:
> 3. Allow to build a minimal Make executable, which provides basic and
> traditional Make functionality and does not rely on gnulib, and then
> use it as a bootstrap.

It's certainly a possibility.  But there is already so much ifdef etc.
in GNU make, I'm loathe to increase it dramatically by trying to carve
out some sort of minimalist, but still sufficiently powerful, version
of make.  And, the things we get from gnulib are not really "extra
things" that we can just jettison, so much as replacements for basic
POSIX functions that not all platforms provide, or where the provided
versions are broken in some way on some platform.

If the GNU make source code was well-organized and cleanly modular this
would probably be a far more reasonable proposition but anyone who's
looked at the code can attest that this is far from the case.

Re: Goodbye to GNU make's "build.sh" ... ?

[Paul Smith]

On Sun, 2022-06-26 at 08:41 +0300, Eli Zaretskii wrote:
> It is sad that Gnulib maintainers aren't prepared to cater to a GNU
> project, and an important one such as Make.  These are special
> requirements that only a handful of GNU project could ever have, and
> for good reasons, so supporting their needs should be a no-brainer.

I suspect their position is that it IS a no-brainer, but with exactly
the opposite result: this requirement is needed by exactly one project
(GNU make, since once you have it you're all set) and so undertaking
all that work in gnulib is not resource-effective.

> > This leaves me with two options:
> > 
> >    1. Stop using gnulib, or at least sharply limit the modules we
> > will include to those with trivial-enough configurations.
> >    2. Abandon the build.sh script and require an existing make
> > program in order to build a new version of GNU make.
> 
> #2 would require that we promise to have Make 4.3 available from here
> to eternity.  Is that feasible?

Well, it doesn't have to be GNU make.  It just needs to be some make
that can use automake generated makefiles.  However, it does require
that such a make exists.

There are two different situations: first, bringing up GNU make on a
non-GNU system where you have some make, but not GNU make.  Maybe you
have BSD make or something.  Then the question is, is your existing
make capable of using automake-generated makefiles?

Second, you might be bootstrapping an entirely new system without ANY
make at all.  Now you need to get one.  I don't know whether the other
versions of make available have facilities to build without an existing
make.  Here you might need an older version of GNU make which has a
build.sh.  Of course here you likely already have a cross-build
environment so you could just cross-compile make as well.

> Can you list the Gnulib functions we currently use?

You can find them in the bootstrap.conf file:

https://git.savannah.gnu.org/cgit/make.git/tree/bootstrap.conf

See the list at the end.

The newly-added strtoll() is causing the proximate issue because it
relies on unistd.h which generates an enormous sed script in the
makefile.

This really came up because I was trying to find a way to include the
latest gnulib globbing library.  Adding "glob" to this pulls in a HUGE
number of prerequisites.  But even after undoing this I discovered
strtoull() is bad enough.

My idea was to keep the current glob/fnmatch to use on systems which
couldn't run "configure" (for example with the build_w32.bat script)
and use the gnulib glob/fnmatch for all systems that could run
configure.

But, configure is not enough: you also need make.  The current build.sh
script contains some minimal amount of magic needed to transform the
simple files we used in GNU make 4.3; see
https://git.savannah.gnu.org/cgit/make.git/tree/build.sh#n69
but making this work for the complex targets is frankly more than I
want to take on.

So then I wondered, why even both with gnulib glob?  Maybe we should
just take ownership of the ancient glob we already have and say, if you
have a system with GNU libc you get the latest, else you get this
basically functional version.

> Perhaps we could then prepare a fake-gnulib module with trivial
> implementations of those functions, which could be used by build.sh
> instead of the real Gnulib functions.

This seems odd to me: I mean, isn't that just replacing gnulib with
what it already is?  The point of gnulib is to provide replacements on
systems that don't provide these.  I don't want to create a new "micro-
gnulib" project.

> (Some Gnulib functions are replacements for those found in every C
> library, and those could be simply ignored in the build.sh build,
> relying on libc to do a reasonably good job.)

Yes, we could say that build.sh can only be used on systems which are
essentially POSIX-compliant and don't need a lot of fixup.

The problem is that the process for "make-less" systems in the past has
been: run configure, then run build.sh.  But we won't be able to use
the environment created by configure in this "new" build.sh, because it
works in tandem with the makefiles.  The generated config.h etc. will
assume lots of things that simply aren't true anymore, since we don't
have make.

So to me this is equivalent to my option #1, don't use gnulib at all
(or at least, use it incredibly sparingly).

Re: Goodbye to GNU make's "build.sh" ... ?

[Paul Smith]

On Sun, 2022-06-26 at 02:14 +0200, Henrik Carlqvist wrote:
> Would  this "old version of make" have to be GNU make?

It would need to be some version of make that is supported by automake.

As far as I'm aware, automake-generated makefiles are currently
intended to run with any POSIX-compliant version of make; I believe
that BSD make qualifies.  Certainly SunPro make from the late Jörg
Shilling would qualify, but I believe it requires a pre-existing make
to build (I haven't tried in quite a while).  And now that we've lost
him I'm not sure who will take up maintenance of that project.

Also, I've heard calls for automake maintainers to simplify the
generated makefiles by assuming GNU make and taking advantage of GNU
make features: then we really WOULD be in a quandary.  But I have no
idea how likely such a thing is to happen.

> The scary part of relying on an old version of some software is that
> old versions sometimes suffer from bit rot caused by changes in
> language standards or API changes in libraries.

Indeed.

> Maybe we would have to rely on cross compiling GNU make for new
> systems?

Another solution, to be sure.

Re: Goodbye to GNU make's "build.sh" ... ?

[David Boyce]

The obvious end-point of #2 is to add a make-4.3.tar.gz file to future make 
packages and modify build.sh to unpack and use it. This might involve renaming 
the "inner" script to "build-4.3.sh" or similar.

But I also have the same question: what are the real-world cases where build.sh 
is needed?

> On Jun 26, 2022, at 10:44 AM, Philip Guenther  wrote:
> 
> 
>> On Sat, Jun 25, 2022 at 12:48 PM Paul Smith  wrote:
> 
>> I'm trying to decide what the future is for GNU make's "build.sh"
>> bootstrapping script. As you may recall, this script is provided to
>> allow GNU make to build on systems which don't already have an instance
>> of make installed. Its goal is to build the first make binary, without
>> of course all the fancy parts of avoiding rebuilds, generating
>> dependency files, etc.
> 
> Can anyone contribute information about actual cases where build.sh was used?
> 
> I've been using GNU make since 1992 or so, on only 9 or so different  UNIX 
> platforms, but they all had _some_ form of make available, however crippled, 
> that was sufficient to bootstrap GNU make.  So, I've avoided being its target 
> audience.  Who has been and can they speak to what they would do without it?  
> "I have a native compiler, but not make" is kinda a weird place to be; those 
> bootstrapping a new OS typically cross-compile the entire toolchain until 
> they can self-host: make is just another small divot in that bumpy road and 
> using build.sh when they still have to cross-compile the rest of the 
> toolchain doesn't actually help.
> 
> 
>> This leaves me with two options:
>> 
>>1. Stop using gnulib, or at least sharply limit the modules we will
>>   include to those with trivial-enough configurations.
>>2. Abandon the build.sh script and require an existing make program
>>   in order to build a new version of GNU make.
> 
> Lacking a description of a case in the past where build.sh was actually the 
> only sane option, I would go with #2.
> 
> 
> Philip Guenther
> 

Re: Goodbye to GNU make's "build.sh" ... ?

[Philip Guenther]

On Sat, Jun 25, 2022 at 12:48 PM Paul Smith  wrote:

> I'm trying to decide what the future is for GNU make's "build.sh"
> bootstrapping script. As you may recall, this script is provided to
> allow GNU make to build on systems which don't already have an instance
> of make installed. Its goal is to build the first make binary, without
> of course all the fancy parts of avoiding rebuilds, generating
> dependency files, etc.
>

Can anyone contribute information about actual cases where build.sh was
used?

I've been using GNU make since 1992 or so, on only 9 or so different  UNIX
platforms, but they all had _some_ form of make available, however
crippled, that was sufficient to bootstrap GNU make.  So, I've avoided
being its target audience.  Who has been and can they speak to what they
would do without it?  "I have a native compiler, but not make" is kinda a
weird place to be; those bootstrapping a new OS typically cross-compile the
entire toolchain until they can self-host: make is just another small divot
in that bumpy road and using build.sh when they still have to cross-compile
the rest of the toolchain doesn't actually help.


This leaves me with two options:
>
>1. Stop using gnulib, or at least sharply limit the modules we will
>   include to those with trivial-enough configurations.
>2. Abandon the build.sh script and require an existing make program
>   in order to build a new version of GNU make.


Lacking a description of a case in the past where build.sh was actually the
only sane option, I would go with #2.


Philip Guenther

Re: Goodbye to GNU make's "build.sh" ... ?

[Sven C. Dack]

Hello,

developers causing a "Catch 22" paradox is not new. You want to avoid
such software, or at least work around it with a big margin. Sharply
limiting the use of gnulib here may not provide such a margin and cause
repeated trouble in the future. Your condition of "to those with
trivial-enough configurations" can fail with the next update to gnulib.

How about a third option:

3. Allow to build a minimal Make executable, which provides basic and
traditional Make functionality and does not rely on gnulib, and then use
it as a bootstrap.

Then make the gnulib maintainers aware of the "Catch 22" and tell them
not rely on newer GNU Make versions. Offering a minimal Make executable
next to the full GNU Make may also have uses for other projects.

-Sven


On 25/06/2022 22:47, Paul Smith wrote:

I'm trying to decide what the future is for GNU make's "build.sh"
bootstrapping script. As you may recall, this script is provided to
allow GNU make to build on systems which don't already have an instance
of make installed. Its goal is to build the first make binary, without
of course all the fancy parts of avoiding rebuilds, generating
dependency files, etc.

Unfortunately, this really cannot work if GNU make is going to rely on
gnulib, because a MANY gnulib modules require not only autoconf to
work, but also automake. That is, it's not enough to just run configure
and then you get a set of source files and header files that you can
compile by hand by just invoking the compiler. You must ALSO run make,
because the modules provide makefile recipes that invoke sed, etc. to
convert files into their final form.

I had a discussion about this with the Gnulib maintainers a while ago:

https://lists.gnu.org/archive/html/bug-gnulib/2019-09/msg00041.html

However the gnulib maintainers were disinclined to modify the practices
of the gnulib modules.

This leaves me with two options:

1. Stop using gnulib, or at least sharply limit the modules we will
   include to those with trivial-enough configurations.
2. Abandon the build.sh script and require an existing make program
   in order to build a new version of GNU make.

#1 is what I followed for GNU make 4.3, which has a limited subset of
carefully-chosen modules. However this becomes harder over time. For
example any module that needs unistd.h requires a very complex automake
rule.

If #2 is chosen, then a bootstrap process would involve first obtaining
an older version of make, such as GNU make 4.3 or lower, and building
that with its build.sh, then using the resulting make to build the
newer version.

I'm interested in thoughts about these options.

Re: Goodbye to GNU make's "build.sh" ... ?

[Eli Zaretskii]

> From: Paul Smith 
> Date: Sat, 25 Jun 2022 17:47:47 -0400
> 
> I had a discussion about this with the Gnulib maintainers a while ago:
> 
> https://lists.gnu.org/archive/html/bug-gnulib/2019-09/msg00041.html
> 
> However the gnulib maintainers were disinclined to modify the practices
> of the gnulib modules.

It is sad that Gnulib maintainers aren't prepared to cater to a GNU
project, and an important one such as Make.  These are special
requirements that only a handful of GNU project could ever have, and
for good reasons, so supporting their needs should be a no-brainer.

> This leaves me with two options:
> 
>1. Stop using gnulib, or at least sharply limit the modules we will
>   include to those with trivial-enough configurations.
>2. Abandon the build.sh script and require an existing make program
>   in order to build a new version of GNU make.
> 
> #1 is what I followed for GNU make 4.3, which has a limited subset of
> carefully-chosen modules. However this becomes harder over time. For
> example any module that needs unistd.h requires a very complex automake
> rule.
> 
> If #2 is chosen, then a bootstrap process would involve first obtaining
> an older version of make, such as GNU make 4.3 or lower, and building
> that with its build.sh, then using the resulting make to build the
> newer version.
> 
> I'm interested in thoughts about these options.

#2 would require that we promise to have Make 4.3 available from here
to eternity.  Is that feasible?

Can you list the Gnulib functions we currently use?  I mean only those
which Make calls directly, not their Gnulib dependencies.  Perhaps we
could then prepare a fake-gnulib module with trivial implementations
of those functions, which could be used by build.sh instead of the
real Gnulib functions.  (Some Gnulib functions are replacements for
those found in every C library, and those could be simply ignored in
the build.sh build, relying on libc to do a reasonably good job.)  Of
course, this would mean the need to add such stubs every time another
Gnulib function is added to Make, so it is still a maintenance burden
of sorts.  But maybe it isn't too heavy?

Re: Goodbye to GNU make's "build.sh" ... ?

[Henrik Carlqvist]

On Sat, 25 Jun 2022 15:28:52 -0800
Britton Kerin  wrote:

> On Sat, Jun 25, 2022, 1:48 PM Paul Smith  wrote:
> > If #2 is chosen, then a bootstrap process would involve first obtaining
> > an older version of make, such as GNU make 4.3 or lower, and building
> > that with its build.sh, then using the resulting make to build the
> > newer version.

> I realize it's easy to say and pretty obvious but option 2 from a user
> perspective is sort of horrific.

Would  this "old version of make" have to be GNU make? Otherwise, if the
Makefiles to build GNU make did not rely on ony GNU extensions those future
systems might be able to initially rely on some other version of make like BSD
pmake.

The scary part of relying on an old version of some software is that old
versions sometimes suffer from bit rot caused by changes in language standards
or API changes in libraries.

Maybe we would have to rely on cross compiling GNU make for new systems?

regards Henrik

Re: Goodbye to GNU make's "build.sh" ... ?

[Britton Kerin]

On Sat, Jun 25, 2022, 1:48 PM Paul Smith  wrote:

> I'm trying to decide what the future is for GNU make's "build.sh"
> bootstrapping script. As you may recall, this script is provided to
> allow GNU make to build on systems which don't already have an instance
> of make installed. Its goal is to build the first make binary, without
> of course all the fancy parts of avoiding rebuilds, generating
> dependency files, etc.
>
> Unfortunately, this really cannot work if GNU make is going to rely on
> gnulib, because a MANY gnulib modules require not only autoconf to
> work, but also automake. That is, it's not enough to just run configure
> and then you get a set of source files and header files that you can
> compile by hand by just invoking the compiler. You must ALSO run make,
> because the modules provide makefile recipes that invoke sed, etc. to
> convert files into their final form.
>
> I had a discussion about this with the Gnulib maintainers a while ago:
>
> https://lists.gnu.org/archive/html/bug-gnulib/2019-09/msg00041.html
>
> However the gnulib maintainers were disinclined to modify the practices
> of the gnulib modules.
>
> This leaves me with two options:
>
>1. Stop using gnulib, or at least sharply limit the modules we will
>   include to those with trivial-enough configurations.
>2. Abandon the build.sh script and require an existing make program
>   in order to build a new version of GNU make.
>
> #1 is what I followed for GNU make 4.3, which has a limited subset of
> carefully-chosen modules. However this becomes harder over time. For
> example any module that needs unistd.h requires a very complex automake
> rule.
>
> If #2 is chosen, then a bootstrap process would involve first obtaining
> an older version of make, such as GNU make 4.3 or lower, and building
> that with its build.sh, then using the resulting make to build the
> newer version.
>

I realize it's easy to say and pretty obvious but option 2 from a user
perspective is sort of horrific.


> I'm interested in thoughts about these options.
>
>