Re: Make 4.3.90 RC fails 3 tests on OpenBSD 7.1 i386

[Paul Smith]

Thanks for testing.

On Sun, 2022-09-25 at 17:25 -0700, Mike Gran wrote:
> I configured it using configure with CC=clang and no other options.
> 
> loadapi.mk.3 and loadapi.mk.7 both complain that testapi.so failed
> to load.  If there is an actual testapi.so being built, I can't
> find it.
> 
> output of make -f work/features/loadapi.mk.3
> 
>   work/features/loadapi.mk.3:2: File not found
>   work/features/loadapi.mk.3:2: *** testapi.so: failed to load. 
> Stop.
> 
> features/loadapi.log.3
> 
>   testapi_gmk_setup
>   testapi.so

Can you attach the contents of the "tests/work" subdirectory (for
example tar it up and send it)?  You can email it directly to me, you
don't have to send it to the mailing list.

> And shuffle.1 has the echo lines in a different order
> 
> k, i, b, c, g, j, l, a, d, e, f, all

This just means that OpenBSD uses a different random number algorithm
which is not unexpected.  I might think about implementing a simple RNG
rather than using the system RNG.

But in any event this is not important.

Re: GNU make 4.3.90 release candidate available

[Sam James]

> On 24 Sep 2022, at 18:45, Paul Smith  wrote:
> 
> On Wed, 2022-09-21 at 06:18 +0100, Sam James wrote:
>> Can I suggest forwarding to the platform-testers@ mailing list as
>> well?
> 
> I've never heard of this; is it @gnu.org?  Or somewhere else?  A full
> email address would be useful and also if you have any information on
> this (a website or whatever) that would be good.
> 

Eli beat me to explaining - sorry for being vague! You got it exactly,
just platform-test...@gnu.org as a mailing list.

> Cheers!
> 

best,
sam



signature.asc
Description: Message signed with OpenPGP

Make 4.3.90 RC fails 3 tests on OpenBSD 7.1 i386

[Mike Gran]

Hi all-
I just happened to have an OpenBSD 7.1 i386 VM open for testing
so I gave the new Make release candidate a test.

I configured it using configure with CC=clang and no other options.

loadapi.mk.3 and loadapi.mk.7 both complain that testapi.so failed
to load.  If there is an actual testapi.so being built, I can't
find it.

output of make -f work/features/loadapi.mk.3

  work/features/loadapi.mk.3:2: File not found
  work/features/loadapi.mk.3:2: *** testapi.so: failed to load.  Stop.

features/loadapi.log.3

  testapi_gmk_setup
  testapi.so

And shuffle.1 has the echo lines in a different order

k, i, b, c, g, j, l, a, d, e, f, all

I know almost nothing about OpenBSD, or make's test suite, but, I
could poke it a little, if I knew what to look for.

Regards,
Mike Gran

[bug #61154] (w32/sub_proc.c) make_command_line should also quote full_exec_path.

[Paul D. Smith]

Follow-up Comment #2, bug #61154 (project make):

@eliz do you have any thoughts about this?


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[bug #63098] make-4.3.90 regression of unexpected dependencies in pattern rules with multiple targets

[Paul D. Smith]

Update of bug #63098 (project make):

  Status:None => Not A Bug  
 Open/Closed:Open => Closed 

___

Follow-up Comment #5:

I'm going to close this for now.  We will see how things go throughout the
rest of the testing to see if this is a serious problem or not.

I verified that the docs don't make any explicit guarantees about this, and so
I updated them to be more clear about the behavior and I also added a note to
the NEWS about this change.


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[bug #63100] Crash in a loadable plugin

[Paul D. Smith]

Update of bug #63100 (project make):

  Status:None => Fixed  
 Open/Closed:Open => Closed 
   Fixed Release:None => SCM
   Triage Status:None => Small Effort   

___

Follow-up Comment #2:

Thanks Dmitry.


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[bug #31002] make picks wrong pattern rule

[Paul D. Smith]

Update of bug #31002 (project make):

  Status:None => Wont Fix   
 Open/Closed:Open => Closed 

___

Follow-up Comment #9:

The new behavior has been standard for 12+ years and most people seem to
prefer it; I'm not going to revert this change.


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[bug #63070] posix_spawn fails to run a child process.

[Paul D. Smith]

Update of bug #63070 (project make):

  Status:None => Fixed  
 Open/Closed:Open => Closed 
   Fixed Release:None => SCM
   Triage Status:None => Small Effort   

___

Follow-up Comment #15:

OK I fixed the configure.ac checking program.  Hopefully it works properly
now.


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[bug #55532] Missing prerequisites are not included in $? if .SECONDARY is present

[Paul D. Smith]

Update of bug #55532 (project make):

  Status:None => Not A Bug  
 Open/Closed:Open => Closed 

___

Follow-up Comment #5:

I've changed my mind: I don't think this is a bug.

The docs for .SECONDARY say:
> [.SECONDARY targets] are treated as intermediate files
which means they are not rebuilt when they are missing.

The docs for $? say it contains:
> The names of all the prerequisites that are newer than the target
In other words, the prerequisites that caused the target to be rebuilt.  If
you don't rebuild the prerequisite, then it's not in $?.  If it's
intermediate, and it has no recipe, then it doesn't cause the target to be
rebuilt (in your example, for instance, if you touch foo so that it's newer
than baz, then the recipe will not be run even though bar is missing).


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

Re: 4.3.90 release candidate segfaults on linux and solaris

[Paul Smith]

On Sun, 2022-09-25 at 13:03 -0400, Paul Smith wrote:
> The odd thing is that make ALREADY preserves the environment pointer
> and restores it, but it does so after child_execute_job() is called
> (after removing some ifdefs):

Oh, well, clearly child_execute_job() is called from other places, that
don't do this.  Doh!

OK will fix, thanks for the assistance all.

Re: 4.3.90 release candidate segfaults on linux and solaris

[Paul Smith]

On Sun, 2022-09-25 at 10:25 -0400, Dmitry Goncharov wrote:
> However, the this does this reproduce on my sun.

Sorry Dmitry but I didn't understand that sentence...?

The odd thing is that make ALREADY preserves the environment pointer
and restores it, but it does so after child_execute_job() is called
(after removing some ifdefs):

{
  /* Fork the child process.  */

  char **parent_environ;

run_local:
  block_sigs ();

  child->remote = 0;

  parent_environ = environ;

  jobserver_pre_child (flags & COMMANDS_RECURSE);

  child->pid = child_execute_job ((struct childbase *)child,
  child->good_stdin, argv);

  environ = parent_environ; /* Restore value child may have clobbered.  */
  jobserver_post_child (flags & COMMANDS_RECURSE);
}

I suppose this may not be sufficient to fix the problem?

Re: 4.3.90 release candidate segfaults on linux and solaris

[Martin Dorey]

https://docs.oracle.com/cd/E88353_01/html/E37841/execvpe-2.html alleges to 
document Oracle Solaris 11.4 and appears to support execvpe.  Yes, it's listed 
in Linux pages as a GNU extension and it's not on the Open Group page for the 
exec family, but symmetry makes it quite an obvious extension.  Perhaps that 
could be detected by a configure test.

The Linux page for vfork, but not the Open Group one, says that vfork suspend 
the caller until the exec is done, so the caller could restore environ after.  
I don't immediately see that it could hurt and should fix it for Linux.

Or there's copying the environment somewhere else.  Huh, even just the environ 
pointer.


From: Bug-make  on behalf of 
Martin Dorey 
Sent: Sunday, September 25, 2022 07:09
To: psm...@gnu.org ; Denis Excoffier 

Cc: bug-make@gnu.org 
Subject: Re: 4.3.90 release candidate segfaults on linux and solaris

execvpe does indeed fix it for me.  Undoing the fix again, doing make clean -k 
and coming out of the chroot to configure --disable-posix-spawn for my 64 bit 
root environment (with libc 2.24 from Debian Stretch on Linux 4.9.0-19-amd64), 
I can reproduce the problem again with this makefile.  I'd be surprised if that 
doesn't crash for Paul too because I suggest that the kernel version isn't 
actually important either.


From: Paul Smith 
Sent: Sunday, September 25, 2022 06:23
To: Martin Dorey ; Denis Excoffier 

Cc: bug-make@gnu.org 
Subject: Re: 4.3.90 release candidate segfaults on linux and solaris

* EXTERNAL EMAIL *

On Sun, 2022-09-25 at 06:02 +, Martin Dorey wrote:
> And vfork is where that happens.  If I’ve followed the thicket of
> #ifdef correctly and understood the vfork man page, then this is
> illegal when using vfork:
>
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmirror%2Fmake%2Fblob%2Fmaster%2Fsrc%2Fjob.c%23L2556data=05%7C01%7CMartin.Dorey%40hitachivantara.com%7C82c25851389d4e1c12cb08da9ef91def%7C18791e1761594f52a8d4de814ca8284a%7C0%7C0%7C637997090043171790%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=v9hASQYmLkyN49Vpvk4DGrT0roD482%2FC5uIyc3V2DLQ%3Dreserved=0

It's long past time to rewrite this entire part of job.c but I don't
want to do that before the release.

I did try this with a 32bit Ubuntu 14.04 docker container image which
uses a similar libc version, but contrary to my previous assertion
(sorry Denis!) if it's an issue with vfork then it IS related to the
kernel version and so my docker containers won't be of any use.

This is not an actual fix (won't work on Solaris obviously since
execvpe() is a GNU extension) but can you try this change to see if it
works on your system Martin?  At least we can verify that this is the
issue:

diff --git a/src/job.c b/src/job.c
index d12a9138..0a4ddae8 100644
--- a/src/job.c
+++ b/src/job.c
@@ -2553,8 +2553,7 @@ exec_command (char **argv, char **envp)

 # else
   /* Run the program.  */
-  environ = envp;
-  execvp (argv[0], argv);
+  execvpe (argv[0], argv, envp);

 # endif /* !__EMX__ */

Re: 4.3.90 release candidate segfaults on linux and solaris

[Dmitry Goncharov]

On Sun, Sep 25, 2022 at 2:09 AM Martin Dorey
 wrote:
> And vfork is where that happens.  If I’ve followed the thicket of #ifdef 
> correctly and understood the vfork man page, then this is illegal when using 
> vfork:
>
> https://github.com/mirror/make/blob/master/src/job.c#L2556


Thanks, Martin.
This is indeed the culprit.
I was able to find a linux where this reproduces. However, the this
does this reproduce on my sun. Denis, can you please try this patch on
your sun?

regards, Dmitry

Here is a patch

index d12a9138..1ed71f0a 100644
--- a/src/job.c
+++ b/src/job.c
@@ -2286,6 +2286,8 @@ child_execute_job (struct childbase *child, int
good_stdin, char **argv)
   posix_spawnattr_t attr;
   posix_spawn_file_actions_t fa;
   short flags = 0;
+#else
+  char **parent_environ = environ;
 #endif

   /* Divert child output if we want to capture it.  */
@@ -2301,7 +2303,12 @@ child_execute_job (struct childbase *child, int
good_stdin, char **argv)

   pid = vfork();
   if (pid != 0)
+{
+  /* The child sets environ to child->environment before calling execvp.
+   * Restore it here.  */
+  environ = parent_environ;
   return pid;
+}

   /* We are the child.  */
   unblock_all_sigs ();
@@ -2552,7 +2559,8 @@ exec_command (char **argv, char **envp)
 errno = ENOEXEC;

 # else
-  /* Run the program.  */
+  /* Run the program.
+   * The parent has to restore environ.  */
   environ = envp;
   execvp (argv[0], argv);

Re: GNU make 4.3.90 release candidate available

[Eli Zaretskii]

> From: Paul Smith 
> Cc: bug-make@gnu.org
> Date: Sun, 25 Sep 2022 08:28:50 -0400
> 
> On Sun, 2022-09-25 at 13:30 +0300, Eli Zaretskii wrote:
> > > From: Paul Smith 
> > > Cc: bug-make@gnu.org
> > > Date: Sat, 24 Sep 2022 16:44:44 -0400
> > > 
> > > On Sat, 2022-09-24 at 16:41 -0400, Paul Smith wrote:
> > > > makeint.h already has HAVE_MAKEINT_H
> > > 
> > > Doh, I meant HAVE_INTTYPES_H of course.
> > 
> > Is the below OK?
> 
> Works for me.  Thanks Eli.

Thanks, installed together with the other 2 patches that avoid
compilation warnings in the MS-Windows build.

Re: 4.3.90 release candidate segfaults on linux and solaris

[Martin Dorey]

execvpe does indeed fix it for me.  Undoing the fix again, doing make clean -k 
and coming out of the chroot to configure --disable-posix-spawn for my 64 bit 
root environment (with libc 2.24 from Debian Stretch on Linux 4.9.0-19-amd64), 
I can reproduce the problem again with this makefile.  I'd be surprised if that 
doesn't crash for Paul too because I suggest that the kernel version isn't 
actually important either.


From: Paul Smith 
Sent: Sunday, September 25, 2022 06:23
To: Martin Dorey ; Denis Excoffier 

Cc: bug-make@gnu.org 
Subject: Re: 4.3.90 release candidate segfaults on linux and solaris

* EXTERNAL EMAIL *

On Sun, 2022-09-25 at 06:02 +, Martin Dorey wrote:
> And vfork is where that happens.  If I’ve followed the thicket of
> #ifdef correctly and understood the vfork man page, then this is
> illegal when using vfork:
>
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmirror%2Fmake%2Fblob%2Fmaster%2Fsrc%2Fjob.c%23L2556data=05%7C01%7CMartin.Dorey%40hitachivantara.com%7C82c25851389d4e1c12cb08da9ef91def%7C18791e1761594f52a8d4de814ca8284a%7C0%7C0%7C637997090043171790%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=v9hASQYmLkyN49Vpvk4DGrT0roD482%2FC5uIyc3V2DLQ%3Dreserved=0

It's long past time to rewrite this entire part of job.c but I don't
want to do that before the release.

I did try this with a 32bit Ubuntu 14.04 docker container image which
uses a similar libc version, but contrary to my previous assertion
(sorry Denis!) if it's an issue with vfork then it IS related to the
kernel version and so my docker containers won't be of any use.

This is not an actual fix (won't work on Solaris obviously since
execvpe() is a GNU extension) but can you try this change to see if it
works on your system Martin?  At least we can verify that this is the
issue:

diff --git a/src/job.c b/src/job.c
index d12a9138..0a4ddae8 100644
--- a/src/job.c
+++ b/src/job.c
@@ -2553,8 +2553,7 @@ exec_command (char **argv, char **envp)

 # else
   /* Run the program.  */
-  environ = envp;
-  execvp (argv[0], argv);
+  execvpe (argv[0], argv, envp);

 # endif /* !__EMX__ */

Re: 4.3.90 release candidate segfaults on linux and solaris

[Paul Smith]

On Sun, 2022-09-25 at 06:02 +, Martin Dorey wrote:
> And vfork is where that happens.  If I’ve followed the thicket of
> #ifdef correctly and understood the vfork man page, then this is
> illegal when using vfork:
> 
> https://github.com/mirror/make/blob/master/src/job.c#L2556

It's long past time to rewrite this entire part of job.c but I don't
want to do that before the release.

I did try this with a 32bit Ubuntu 14.04 docker container image which
uses a similar libc version, but contrary to my previous assertion
(sorry Denis!) if it's an issue with vfork then it IS related to the
kernel version and so my docker containers won't be of any use.

This is not an actual fix (won't work on Solaris obviously since
execvpe() is a GNU extension) but can you try this change to see if it
works on your system Martin?  At least we can verify that this is the
issue:

diff --git a/src/job.c b/src/job.c
index d12a9138..0a4ddae8 100644
--- a/src/job.c
+++ b/src/job.c
@@ -2553,8 +2553,7 @@ exec_command (char **argv, char **envp)

 # else
   /* Run the program.  */
-  environ = envp;
-  execvp (argv[0], argv);
+  execvpe (argv[0], argv, envp);

 # endif /* !__EMX__ */

Re: GNU make 4.3.90 release candidate available

[Paul Smith]

On Sun, 2022-09-25 at 13:30 +0300, Eli Zaretskii wrote:
> > From: Paul Smith 
> > Cc: bug-make@gnu.org
> > Date: Sat, 24 Sep 2022 16:44:44 -0400
> > 
> > On Sat, 2022-09-24 at 16:41 -0400, Paul Smith wrote:
> > > makeint.h already has HAVE_MAKEINT_H
> > 
> > Doh, I meant HAVE_INTTYPES_H of course.
> 
> Is the below OK?

Works for me.  Thanks Eli.

Re: GNU make 4.3.90 release candidate available

[Eli Zaretskii]

> From: Paul Smith 
> Cc: bug-make@gnu.org
> Date: Sat, 24 Sep 2022 16:44:44 -0400
> 
> On Sat, 2022-09-24 at 16:41 -0400, Paul Smith wrote:
> > makeint.h already has HAVE_MAKEINT_H
> 
> Doh, I meant HAVE_INTTYPES_H of course.

Is the below OK?

--- src/makeint.h~0 2022-09-18 22:06:17.0 +0300
+++ src/makeint.h   2022-09-25 13:24:52.53775 +0300
@@ -290,6 +290,14 @@ char *strerror (int errnum);
 
 #if HAVE_INTTYPES_H
 # include 
+#else
+# ifndef PRId64
+#  ifdef WINDOWS32
+#   define PRId64 "I64d"
+#  else
+#   define PRId64 "lld"
+#  endif
+# endif
 #endif
 #if HAVE_STDINT_H
 # include 

--- src/function.c~02022-09-18 22:06:17.0 +0300
+++ src/function.c  2022-09-25 13:26:45.88150 +0300
@@ -825,11 +825,11 @@ func_wordlist (char *o, char **argv, con
 
   if (start < 1)
 ON (fatal, *expanding_var,
-"invalid first argument to 'wordlist' function: '%lld'", start);
+"invalid first argument to 'wordlist' function: '%" PRId64 "'", start);
 
   if (stop < 0)
 ON (fatal, *expanding_var,
-"invalid second argument to 'wordlist' function: '%lld'", stop);
+"invalid second argument to 'wordlist' function: '%" PRId64 "'", stop);
 
   count = stop - start + 1;
 

Re: 4.3.90 release candidate segfaults on linux and solaris

[Martin Dorey]

I have a 32 bit Debian Squeeze (so 2011 era, with gcc-4.4 - too early for asan 
- and libc 2.11, so two Debian releases earlier than Denis’s) chroot lying 
around.  Unsurprisingly, his reproducer crashes for me too, configured with no 
options and make run with no arguments.  gdb’s reporting line 119 rather than 
118 in expand.c.  valgrind annoyingly stops it crashing.

I had trouble compiling rule.c because johnny-come-lately mempcpy on this 
vintage setup seems to be provided as a macro and that doesn't play well with 
the STRING_SIZE_TUPLE macro, because, until the latter is expanded, mempcpy 
appears to only have two arguments, when it requires three.  #undef mempcpy in 
rule.c got me over that.  I know we depend on C99 now, seemingly without a 
storm of protest, but this is failing to compile on a setup a decade newer than 
that.

For me, it’s the variable known in the make source as “environ”, which isn't 
what my gdb calls environ but what it calls ‘environ@@GLIBC_2.0' which has been 
corrupted, by the second call to recursively_expand_for_file, not, far as I 
sampled, the environment variables it points to.  gdb alleged to let me set a 
hardware watch point on it but it didn't fire.  It seems to have been corrupted 
to point into the middle of an environment variable.  So what should be an 
array of character pointers is an array of characters.  Interpreting those as 
character pointers promptly doesn't go so well.  It’s already changed, for me, 
by the call to free_childbase.

…

And vfork is where that happens.  If I’ve followed the thicket of #ifdef 
correctly and understood the vfork man page, then this is illegal when using 
vfork:

https://github.com/mirror/make/blob/master/src/job.c#L2556

(I’d use the savannah git web thing but I don’t see line number links.)

The corruption happening in another process would explain the hardware watch 
point not catching it.  The “blame“ says that dodgy assignment is 28 years old 
but, as was recently topical again, we have stopped using posix_spawn instead 
of vfork with old libc versions and this code for copying the environment 
definitions of recursively expanded variables for $(shell) invocations is even 
newer.  Perhaps Make didn’t otherwise use the environ variable that it’s 
illegally corrupted for the last three decades.


From: Bug-make  on behalf of 
Paul Smith 
Sent: Saturday, September 24, 2022 14:17
To: Denis Excoffier 
Cc: bug-make@gnu.org 
Subject: Re: 4.3.90 release candidate segfaults on linux and solaris

* EXTERNAL EMAIL *

On Sat, 2022-09-24 at 13:06 -0400, Paul Smith wrote:
> > The crashes in solaris and linux are so similar, and go away also
> > so similarly, that i would primarily think about size of types.
> > Under cygwin and MacOS all sizes (pointers, long int, size_t,
> > SIZE_MAX, time_t) are 8, while under (this) solaris and (this)
> > linux all sizes are 4. Only size of int is 4 on all.
>
> Aha!  You're using a 32bit Linux.  That is indeed helpful
> information, thanks.  I will check it.

I installed a 32bit docker container and tried with that but still no
failure.  I will keep poking at it; maybe I can find a much older
container.