[bug #63157] Unlink temporary files.

[Paul D. Smith]

Follow-up Comment #21, bug #63157 (project make):

The only places leftover files may happen is (a) on Windows, only when (b) you
kill GNU make with ^C or similar.

Any other leftover files should not happen.  Please provide details of the
names of the files that are left over and which circumstances you see them in.
 We do have tests that verify that temp files are removed so maybe those tests
are not complete.


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[bug #63157] Unlink temporary files.

[Dmitry Goncharov]

Follow-up Comment #22, bug #63157 (project make):

Frank, can you please tell us how to reproduce?


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

Re: GNU make 4.3.91 on Debian 9.1

[Paul Smith]

On Wed, 2022-10-19 at 05:05 +, Martin Dorey wrote:
> >  Is there any possibility of seeing the output
> 
> Certainly, but I still don't want to spam the list to death with the
> whole thing and I didn't spot anything interesting.

I meant in general.  For example, for the failures that appear to be
timeouts sometimes the test suite will generate some information about
that, to stdout/stderr.

Maybe there's some make target or script that, if it exists, the
platform testing scripts will run on failure to generate the tar file?

With an error I'd like to get various other files such as the
src/config.h file, the config.status file if it exists, plus the
test/work directory and also the output of the configure/build/test
run.

I'm happy to add something to GNU make to collect "useful stuff" if
there's a standard way that testers would like it.

Maybe just augmenting the check target to do more?

Re: GNU make 4.3.91 on Debian 9.1

[Bruno Haible]

Hi Paul,

> For example, for the failures that appear to be
> timeouts sometimes the test suite will generate some information about
> that, to stdout/stderr.
> 
> Maybe there's some make target or script that, if it exists, the
> platform testing scripts will run on failure to generate the tar file?

Some testers (like Nelson Beebe) have more evolved "platform testing scripts"
than others. I use some "platform testing routine", that involves looking at
the "make check" output and guess what info would be useful to send back.

> With an error I'd like to get various other files such as the
> src/config.h file, the config.status file if it exists, plus the
> test/work directory and also the output of the configure/build/test
> run.

It would be useful to formalize this, so that the platform testers' feedback
becomes more useful to you. For example, Nelson's summary today [1] shows
that he thought it would be useful to report the part of the "make check"
output that contains the failures. I thought — seeing the line
  3 Tests in 3 Categories Failed (See .diff* files in work dir for details)
— that it would be good to do
cd tests; tar cvfz $platform-failures.tar.gz `find work -name '*.diff*'`
and send that. It turns out that both are insufficient for your analysis.
If you tell us what to return as part of the feedback, we can accommodate it.

Many GNU packages create a 'test-suite.log' file, that contains the list of
failed or skipped tests, plus the log file (stdout+stderr) for each. This
makes it simple: As a tester, I simply attach this test-suite.log (or the
parts I judge relevant).

> the config.status file if it exists

Yes, the config.status file is useful to attach, especially when the
compilation failed, but probably also when the compilation succeeded and
some tests failed.

> the src/config.h file

The src/config.h is redundant IMO, since it is derived from config.status.

> plus the test/work directory

All of it? Or a selected part of it?

> and also the output of the configure/build/test run.

Nelson and I do keep these outputs, as part of our build scripts. If you
want it, we can send it. We only need to know what to send.

> I'm happy to add something to GNU make to collect "useful stuff" if
> there's a standard way that testers would like it.
> 
> Maybe just augmenting the check target to do more?

Yes. Instead of printing
  See .diff* files in work dir for details
it could
  - generate a summary file with details (like test-suite.log, described
above), and/or
  - generate tarball with useful info to send, and/or
  - (if you prefer receiving mails with several attachments instead of a
tarball attachment) print a list of files to send.
And ideally, also print the email address for the reports. I sent my reports
to bug-make@; Nelson sent his to platform-testers@.

Assuming what you want is a tarball with
  config.status
  `find tests/work -name '*.diff*'`
  `find tests/work -name '*.diff*' | sed -e 's/\.diff/.log/'`
that's what I'm sending now.

Bruno

[1] https://lists.gnu.org/archive/html/platform-testers/2022-10/msg5.html


debian91-failures.tar.gz
Description: application/compressed-tar

Re: features/temp_stdin FAILED (was: GNU make 4.3.91 release candidate available)

[Dmitry Goncharov]

On Wed, Oct 19, 2022 at 2:33 AM Frank Heckenbach
 wrote:
> features/temp_stdin . FAILED (6/7 passed)
> [...]
> % cat tests/work/features/temp_stdin.diff.5
> *** work/features/temp_stdin.base.5 Wed Oct 19 06:05:22 2022
> --- work/features/temp_stdin.log.5  Wed Oct 19 06:05:22 2022
> ***
> *** 0 
> --- 1 
> + make: *** [work/features/temp_stdin.mk.5:5: bye.mk] Terminated
>
> The test failure doesn't occur every time, only about 1 in 6 times.


This test has a recipe to send sigterm to make. So, in your testing,
occasionally this output
"make: *** [work/features/temp_stdin.mk.5:5: bye.mk] Terminated" ends
up in the log file, but
most of the time it does not. Looks like there is a race here between
sigterm and make writing to the log file.


regards, Dmitry

Re: GNU make 4.3.91 on Debian 9.1

[Martin Dorey]

> make light of the one that isn't (far as I see) a regression

I looked into it further and concluded that failure is indeed expected on 
Debian 9, where it's a harmless documentation of the legacy $(wildcard) 
behavior with dangling symlinks there.  I've posted an explanation of that to 
https://savannah.gnu.org/bugs/index.php?52018#comment6 in the hope that a web 
search will lead me from symptom to cause more quickly next time.

Having built more times now, I’m intermittently seeing Bruno’s temp_stdin 
failure too but I see Frank and Dmitry have gotten further with that under 
separate cover.  That means that the only one left to be understood in this 
thread is Bruno's:

*** work/features/output-sync.base.15 Wed Oct 19 01:39:11 2022
--- work/features/output-sync.log.15  Wed Oct 19 01:39:11 2022
***
*** 0 
--- 1 
+ make: *** [work/features/output-sync.mk.15:3: all] Terminated

I haven't seen that one, though it bears quite a resemblance to the temp_stdin 
symptom.


From: Martin Dorey 
Sent: Tuesday, October 18, 2022 22:05
To: Bruno Haible ; bug-make@gnu.org ; 
psm...@gnu.org 
Subject: Re: GNU make 4.3.91 on Debian 9.1

>> On Debian 9.1.0 / x86 (a machine with glibc 2.24)
...
>> 3 tests fail

For me on Debian 9.13 amd64 (a machine with glibc 2.24) it remains just the one 
failure, the one I alluded to in https://savannah.gnu.org/bugs/?63016, so it's 
not very new.  Indeed, output below suggests it's several years and releases 
old.

> Is there any possibility of seeing the output

Certainly, but I still don't want to spam the list to death with the whole 
thing and I didn't spot anything interesting.

make  check-local
make[2]: Entering directory '/home/martind/download/make'
cd tests && perl -w ./run_make_tests.pl -srcdir /home/martind/download/make 
-make ../make
Name "main::testee_version" used only once: possible typo at 
./run_make_tests.pl line 598.
Name "main::testee" used only once: possible typo at ./run_make_tests.pl line 
465.
Name "main::tmpfilesuffix" used only once: possible typo at ./run_make_tests.pl 
line 467.
Name "main::profile" used only once: possible typo at ./run_make_tests.pl line 
432.
Name "main::testname" used only once: possible typo at ./run_make_tests.pl line 
715.
Name "main::vos" used only once: possible typo at ./run_make_tests.pl line 432.
Name "main::osname" used only once: possible typo at ./run_make_tests.pl line 
569.
Name "CORE::GLOBAL::rmdir" used only once: possible typo at ./run_make_tests.pl 
line 96.
Name "main::workdir" used only once: possible typo at ./run_make_tests.pl line 
254.
Name "main::cwddir" used only once: possible typo at ./run_make_tests.pl line 
42.
---
  Running tests for GNU Make on linux
GNU Make 4.3.91
---

Clearing work...
Finding tests...

features/archives ... ok (12 passed)
features/comments ... ok (1 passed)
features/conditionals ... ok (5 passed)
...
functions/warning ... ok (5 passed)
functions/wildcard .. FAILED (9/10 passed)
functions/word .. ok (18 passed)
...
variables/special ... ok (6 passed)
variables/undefine .. ok (5 passed)
vms/library . N/A

1 Test in 1 Category Failed (See .diff* files in work dir for details) :-(

make[2]: *** [Makefile:1533: check-regression] Error 1
make[2]: Leaving directory '/home/martind/download/make'
make[1]: *** [Makefile:1268: check-am] Error 2
make[1]: Leaving directory '/home/martind/download/make'
make: *** [Makefile:970: check-recursive] Error 1
martind@sirius:~/download/make$ find tests/work/ | grep diff
tests/work/functions/wildcard.diff.9
martind@sirius:~/download/make$

... the contents of which was as per Bruno.

Debian make-3.81 (on x86 aka ia32 aka 32 bit) said:

(ia32)martind@sirius:~/tmp/wildcard-9$ echo 'all: ; @echo $(wildcard __ldir)' | 
/usr/bin/make -f /dev/stdin
__ldir
(ia32)martind@sirius:~/tmp/wildcard-9$ ls -l
total 0
lrwxrwxrwx 1 martind software 6 Oct 19 04:47 __ldir -> __rdir
(ia32)martind@sirius:~/tmp/wildcard-9$

But Debian make-4.0 in the same directory said:

martind@zoot:~/tmp/wildcard-9$ echo 'all: ; @echo $(wildcard __ldir)' | 
/usr/bin/make -f /dev/stdin

martind@zoot:~/tmp/wildcard-9$

... up to the latest git:

martind@sirius:~/tmp/wildcard-9$ echo 'all: ; @echo $(wildcard __ldir)' | 
~/download/make/make -f /dev/stdin

martind@sirius:~/tmp/wildcard-9$

martind@sirius:~/tmp/wildcard-9$ echo 'all: ; @echo $(wildcard __ldir)' | 
strace -f 

[bug #52018] suggestion: test case for glob with dangling symlink

[Martin Dorey]

Follow-up Comment #6, bug #52018 (project make):

If I've understood this correctly, then the upshot of the new test,
https://git.savannah.gnu.org/cgit/make.git/commit/?id=f9ba22e02924e0a869a6e40ef4a70ee884d1ec46,
is that we should expect a "failure", which is really just documenting legacy
behavior, that looks like:

martind@sirius:~/download/make-4.3.91$ make check
...
functions/wildcard .. FAILED (9/10
passed)
...
martind@sirius:~/download/make-4.3.91$ cat
tests/work/functions/wildcard.diff.9
*** work/functions/wildcard.base.9  Wed Oct 19 09:03:36 2022
--- work/functions/wildcard.log.9   Wed Oct 19 09:03:36 2022
***
*** 1 
! __ldir
--- 1 
! 
martind@sirius:~/download/make-4.3.91$ 

... from make binaries built using glibc glob on systems with glibc older than
https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=ccf970c7a77e86f4f5ef8ecc5e637114b1c0136a
so older than 2.27.  So Debian Stretch aka 9, with glibc 2.24, fails, where
Debian Buster aka 10, with glibc 2.28, passes, meaning that it exhibits the
new behavior, where $(wildcard) matches even dangling symlinks.


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[bug #63157] Unlink temporary files.

[Martin Dorey]

Follow-up Comment #23, bug #63157 (project make):

Frank wrote [

I now found a way to reproduce it quite easily on my system.
As I suspected, it seems to be SIGPIPE:

% rm -f /tmp/G*
% cat Makefile
all: ; sleep 1
% make -j2 -O 2>&1 | :
% ls -l /tmp/G*
prw--- 1 frank frank 0 19. Okt 22:42 /tmp/GMfifo370973
-rw--- 1 frank frank 0 19. Okt 22:42 /tmp/GmFkJIuq
%


].  That looked like just the ticket but sadly it doesn't reproduce it for me,
on amd64 Debian Stretch.


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[bug #63157] Unlink temporary files.

[Frank Heckenbach]

Follow-up Comment #24, bug #63157 (project make):


[comment #23 comment #23:]
> Frank wrote [
> 
> I now found a way to reproduce it quite easily on my system.
> As I suspected, it seems to be SIGPIPE:
> 
> % rm -f /tmp/G*
> % cat Makefile
> all: ; sleep 1
> % make -j2 -O 2>&1 | :
> % ls -l /tmp/G*
> prw--- 1 frank frank 0 19. Okt 22:42 /tmp/GMfifo370973
> -rw--- 1 frank frank 0 19. Okt 22:42 /tmp/GmFkJIuq
> %
> 
> 
> ].  That looked like just the ticket but sadly it doesn't reproduce it for
me, on amd64 Debian Stretch.

I just reproduced it on a Linux 4.9.0-5-amd64 #1 SMP Debian 4.9.65-3+deb9u2
(2018-01-04) x86_64 GNU/Linux machine. So it doesn't seem to be the version.

Anyway, strace confirms that make is killed by SIGPIPE, so if you catch it (or
ignore it and then handle or ignore stdout/stderr write errors), it should
work.


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

Re: features/temp_stdin FAILED (was: GNU make 4.3.91 release candidate available)

[Dmitry Goncharov]

On Wed, Oct 19, 2022 at 4:25 PM Frank Heckenbach
 wrote:

> Indeed, adding a sleep after kill seems to ensure the message is
> always written to the log file.

Thanks, Frank.
The same fix is also needed in output-sync. I opened
https://savannah.gnu.org/bugs/index.php?63243 and attached a patch.
regards, Dmitry

[bug #63243] Fix a race between sigterm and make writing its error message.

[Dmitry Goncharov]

Additional Item Attachment, bug #63243 (project make):

File name: sv63243.diff   Size:2 KB




___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[bug #63241] implement "exit" in makefile parsing

[anonymous]

URL:
  

 Summary: implement "exit" in makefile parsing
 Project: make
   Submitter: None
   Submitted: Wed 19 Oct 2022 04:43:52 PM UTC
Severity: 3 - Normal
  Item Group: Enhancement
  Status: None
 Privacy: Public
 Assigned to: None
 Open/Closed: Open
 Discussion Lock: Any
   Component Version: SCM
Operating System: None
   Fixed Release: None
   Triage Status: None


___

Follow-up Comments:


---
Date: Wed 19 Oct 2022 04:43:52 PM UTC By: Anonymous
I’ll start this by making an analogy with scripting languages. A Python
script might have logic like:

while :
if :
# do thing A
else: 
# do thing B

Indenting can go to unlimited depths so most languages offer a short-circuit
which makes things more readable:

while :
if :
# do thing A
continue

# do thing B

Coming back to GNU make, I have quite a few complex (many hundreds of lines)
makefiles which require logic like this (greatly simplified):

ifneq ($(filter %clean clobber,$(MAKECMDGOALS)),)
  
else
  
endif

This makes it harder to read, because all the “real” logic is a subclause
(or 2, or 3) deep. Tracking if-else-endif across large files is difficult.
I’m wondering if the make parser would be capable of something like this:

ifneq ($(filter %clean clobber,$(MAKECMDGOALS)),)
  
  exit
endif


Where the “exit” keyword simply says to stop parsing. Perhaps there’s a
better name, but the basic idea is to implement short-circuit parsing
analogously to the shell example. Doable?








___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[bug #52018] suggestion: test case for glob with dangling symlink

[Bruno Haible]

Follow-up Comment #7, bug #52018 (project make):

> So Debian Stretch aka 9, with glibc 2.24, fails, where Debian Buster aka 10,
with glibc 2.28, passes, meaning that it exhibits the new behavior, where
$(wildcard) matches even dangling symlinks.

It would be possible for 'make' to not have this test failure, that is, to
exhibit the new behaviour of $(wildcard) even on older glibc systems. The way
to do so would be to use the Gnulib module 'glob', which is documented to fix
this problem: https://www.gnu.org/software/gnulib/manual/html_node/glob.html

Currently GNU make uses only part of the Gnulib module 'glob': It has the .c
file, but uses a module 'make-glob' instead of 'glob'. On Debian 9.1, this
module produces:
- during configuration the message "checking if system libc has GNU glob...
yes"
- in config.status the substitution USE_SYSTEM_GLOB ← yes.

That's why we see this test fail on glibc < 2.27 (and likely also AIX 7.2,
Solaris 11.4) systems.

But it's not all so simple to use full Gnulib modules in GNU make, see
gl/README...


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[bug #63243] Fix a race between sigterm and make writing its error message.

[Dmitry Goncharov]

URL:
  

 Summary:  Fix a race between sigterm and make writing its
error message.
 Project: make
   Submitter: dgoncharov
   Submitted: Thu 20 Oct 2022 12:35:26 AM UTC
Severity: 3 - Normal
  Item Group: Bug
  Status: None
 Privacy: Public
 Assigned to: None
 Open/Closed: Open
 Discussion Lock: Any
   Component Version: SCM
Operating System: POSIX-Based
   Fixed Release: None
   Triage Status: None


___

Follow-up Comments:


---
Date: Thu 20 Oct 2022 12:35:26 AM UTC By: Dmitry Goncharov 
.







___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

Re: features/temp_stdin FAILED (was: GNU make 4.3.91 release candidate available)

[Frank Heckenbach]

Dmitry Goncharov wrote:

> On Wed, Oct 19, 2022 at 2:33 AM Frank Heckenbach
>  wrote:
> > features/temp_stdin . FAILED (6/7 
> > passed)
> > [...]
> > % cat tests/work/features/temp_stdin.diff.5
> > *** work/features/temp_stdin.base.5 Wed Oct 19 06:05:22 2022
> > --- work/features/temp_stdin.log.5  Wed Oct 19 06:05:22 2022
> > ***
> > *** 0 
> > --- 1 
> > + make: *** [work/features/temp_stdin.mk.5:5: bye.mk] Terminated
> >
> > The test failure doesn't occur every time, only about 1 in 6 times.
> 
> 
> This test has a recipe to send sigterm to make. So, in your testing,
> occasionally this output
> "make: *** [work/features/temp_stdin.mk.5:5: bye.mk] Terminated" ends
> up in the log file, but
> most of the time it does not. Looks like there is a race here between
> sigterm and make writing to the log file.

Indeed, adding a sleep after kill seems to ensure the message is
always written to the log file.

I put a very long sleep here; it doesn't really matter since it's
killed together with make, but there may be a more elegant way.

--- make-4.3.91/tests/scripts/features/temp_stdin
+++ make-4.3.91/tests/scripts/features/temp_stdin
@@ -69,9 +69,9 @@
 include bye.mk
 pid:=$(shell echo $$PPID)
 all:;
-bye.mk: force; @kill -TERM $(pid)
+bye.mk: force; @kill -TERM $(pid); sleep 1000
 force:
-!, '-f-', "", POSIX::SIGTERM);
+!, '-f-', "make: \*\*\* \[work/features/temp_stdin.mk.5:5: bye.mk\] 
Terminated", POSIX::SIGTERM);
 }
 unlink($fout);
 

[bug #63243] Fix a race between sigterm and make writing its error message.

[Dmitry Goncharov]

Follow-up Comment #1, bug #63243 (project make):

Tests in temp_stdin and output-sync send sigterm to kill make. There is a race
here between sigterm and make writing to the log file. This races causes the
tests to fail occasionally.
The original bug report and a patch provided by Frank Heckenbach
here
https://lists.gnu.org/archive/html/bug-make/2022-10/msg00150.html.


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

Re: [bug #63157] Unlink temporary files.

[Frank Heckenbach]

Paul D. Smith wrote:

> Follow-up Comment #21, bug #63157 (project make):
> 
> The only places leftover files may happen is (a) on Windows, only when (b) you
> kill GNU make with ^C or similar.
> 
> Any other leftover files should not happen.  Please provide details of the
> names of the files that are left over and which circumstances you see them in.
>  We do have tests that verify that temp files are removed so maybe those tests
> are not complete.

I now found a way to reproduce it quite easily on my system.
As I suspected, it seems to be SIGPIPE:

% rm -f /tmp/G*
% cat Makefile
all: ; sleep 1
% make -j2 -O 2>&1 | :
% ls -l /tmp/G*
prw--- 1 frank frank 0 19. Okt 22:42 /tmp/GMfifo370973
-rw--- 1 frank frank 0 19. Okt 22:42 /tmp/GmFkJIuq
%

Re: [bug #62936] Confusing description of chained rules in the manual

[Tim Murphy]

On Sat, 15 Oct 2022 at 23:57, Paul D. Smith  wrote:

> Follow-up Comment #1, bug #62936 (project make):
>
> I don't know why you say "you cannot update a non-existing file": make does
> this all the time!  When you start make in a clean directory there are no
> object files, for example, and so those files are non-existing but yet of
> course make will still update them.
>
> By "update" the make manual means "run the recipe for that target".  It
> doesn't mean "modify an existing file or fail if the file doesn't already
> exist".  This term is used everywhere in the manual, to mean that same
> thing.
>

Outside of make "update" implies existence - we use CRUD as a term for
example
not RUD.  Why redefine the language? Actually for databases the word
"upsert" has
been invented to explain a roughly analagous situation is going on -
perhaps that is
an option? e.g. "upcreate"?


> I guess I don't understand the issue well enough to suggest a better
> formulation either.  Maybe if you can say what "case" you mean when you say
> "how Make deals with this case" then it would become more clear where we
> might
> enhance the document to be less confusing.
>
> As for "leave well enough alone" that's a very common idiom; for example:
>
> https://dictionary.cambridge.org/us/dictionary/english/leave-well-enough-alone
>
> According to Cambridge:
"leave well enough alone"
*to *allow 
something
to stay  as it is
because doing more might make things worse


Is make really "trying not to make things worse?" .. As if some job is done
well enough and redoing it might make it worse?  I'm a native English
speaker and I think this phrase is not an accurate description of what's
happening.

Just to give you an example:
>> The first difference is what happens if the intermediate file does
>> not exist. If an ordinary file b does not exist, and make considers a
>> target that depends on b, it invariably creates b and then upcreates
>> the target from b. But if b is an intermediate file, then make does
>> not bother upcreting b, or the ultimate target, unless some
>> prerequisite of b is newer than that target or
>> there is some other reason to update that target.

Regards,

Tim

features/temp_stdin FAILED (was: GNU make 4.3.91 release candidate available)

[Frank Heckenbach]

% uname -a
Linux mars 5.10.0-19-amd64 #1 SMP Debian 5.10.149-1 (2022-10-17) x86_64 
GNU/Linux
% tar xf make-4.3.91.tar.lz
% cd make-4.3.91
% ./configure --prefix=/usr
% make -j
% make check
[...]
features/temp_stdin . FAILED (6/7 passed)
[...]
% cat tests/work/features/temp_stdin.diff.5
*** work/features/temp_stdin.base.5 Wed Oct 19 06:05:22 2022
--- work/features/temp_stdin.log.5  Wed Oct 19 06:05:22 2022
***
*** 0 
--- 1 
+ make: *** [work/features/temp_stdin.mk.5:5: bye.mk] Terminated

The test failure doesn't occur every time, only about 1 in 6 times.