[bug #63650] Performance regression with EXPORT_ALL_VARIABLES enabled

[Paul D. Smith]

Follow-up Comment #6, bug #63650 (project make):

I guess we might be saved by a weird behavior of GNU Make that I've long
considered changing because it confuses people: when we are about to expand a
recipe we expand all lines in the recipe at once, before we invoke the first
line.  This means that subsequent recipe lines can't observe changes to the
environment based on changes in the system that were created by prior recipe
lines, anyway.

In order for the behavior to be observed it would have to use *$(file ...)* in
one expansion followed by a *$(wildcard ...)*.  We could certainly make
expansion of *$(file ...)* be another operation that would invalidate the
environment.

I kind of hate to add anything that increases reliance on this unpleasant
(IMO) behavior of how recipes are expanded, however.


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[bug #63650] Performance regression with EXPORT_ALL_VARIABLES enabled

[Paul D. Smith]

Update of bug #63650 (project make):

  Item Group: Bug => Enhancement

___

Follow-up Comment #5:

What about this idea: we would create a new environment when needed with all
variables expanded, as in current GNU Make 4.4.  Then instead of re-computing
it every time we want to run another command, we would only recompute it if
some make operation happened that might change it.  For example if we entered
a new rule (due to target-specific variables) or we modified a make variable
value, that would invalidate the environment.  If not, we could just re-use
the existing environment.

Running a shell command in itself, without setting a make variable, cannot
change the environment of a subsequent shell command.  At least, I can't think
of a way it could happen.

In that case, the example makefile below with the rule:

slow:
echo ${AAA}
echo ${AAB}
echo ${AAC}
echo ${AAD}
echo ${AAE}
echo ${AAF}

would still invoke the shell more often in order to compute the environment
for the first *echo* line above, but then, since none of these lines alter any
state in make itself, we could re-use that same environment.

But, actually maybe that's not good enough.  The value of the variable could
refer to functions like *$(wildcard ...)*, which would depend on the state of
the filesystem and so expand differently in the second line of the recipe than
they did for the first line of the recipe, if we didn't recompute the
environment.

Hm.  I'm going to convert this to an enhancement request.  I can't see a way
to fix it right now.


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[bug #63650] Performance regression with EXPORT_ALL_VARIABLES enabled

[Paul D. Smith]

Follow-up Comment #4, bug #63650 (project make):

I don't think this is a bug.  Or, at least I don't think we can fix it.

In previous versions of GNU Make we did not add make variables to the
environment of the shell function regardless of their export status.  So for
example this makefile:

export FOO = bar
BAR = $(shell echo $$FOO)
all: ; @echo BAR=$(BAR)

would print *BAR=* in older versions of GNU Make.  In GNU Make 4.4 it will
print *BAR=bar*.


This was a problem for many reasons and was unexpected.  In GNU Make 4.4 we
changed this; from the NEWS file:

* WARNING: Backward-incompatibility!
  Previously makefile variables marked as export were not exported to
commands
  started by the $(shell ...) function.  Now, all exported variables are
  exported to $(shell ...).  If this leads to recursion during expansion,
then
  for backward-compatibility the value from the original environment is used.
  To detect this change search for 'shell-export' in the .FEATURES variable.


In your situation this means that every time make wants to invoke a shell
using the $(shell ...) function it must expand all the makefile variables (the
same way that it would when invoking a recipe).  If makefile variables take a
long time to expand (for example, they require invoking a lot of other shell
functions) you get this sort of pathological behavior.

I'm not sure I see any way to address this, other than writing more reasonable
makefiles.  For example if you use this instead:

AAA := $(shell echo 1)
AAB := $(shell echo 2)
AAC := $(shell echo 3)
AAD := $(shell echo 4)
AAE := $(shell echo 5)
AAF := $(shell echo 6)

then you won't see this problem because variables will only ever be expanded
one time.


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

Re: [PATCH] Check .exe as well when a target does not exist on OS/2

[Paul Smith]

On Fri, 2023-01-13 at 22:27 +0900, KO Myung-Hun wrote:
> This pattern is usually used on UNIX. However, on OS/2, gcc creates
> foo.exe not foo when an extension is not present, and Make check foo
> only. Therefore Make tries to build foo whenever called.

I don't think I like this change.  I understand its usefulness but in
general make never tries to manipulate the target names like this.  If
users want to create a file named "foo.exe" they should use that as the
target name.

I get that it's super-annoying that when you ask GCC to build a file
named "foo" via "-o $@" it will actually create a file named "foo.exe"
instead, but I think that's something that makefile authors will have
to deal with, rather than make.

What do you do in situations where there are targets for BOTH "foo" and
"foo.exe" in the makefile?  Then when you want to build "foo" it may
decide that it's up to date, because it sees the "foo.exe" file
instead.

[bug #63650] Performance regression with EXPORT_ALL_VARIABLES enabled

[Paul D. Smith]

Follow-up Comment #3, bug #63650 (project make):

I think they just mean they also tested Git HEAD (which is the commit
mentioned) and see the same behavior as with the GNU Make 4.4 release.


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

Re: [PATCH] Check .exe as well when a target does not exist on OS/2

[Eli Zaretskii]

> From: KO Myung-Hun 
> Date: Fri, 13 Jan 2023 22:27:43 +0900
> 
> For example,
> 
> foo: foo.c
> gcc $@ $<
> 
> This pattern is usually used on UNIX. However, on OS/2, gcc creates
> foo.exe not foo when an extension is not present, and Make check foo
> only. Therefore Make tries to build foo whenever called.

Please describe the use case in detail.  This situation exists on
other platforms, not just of OS/2, and we don't do anything like that
for those other targets, AFAIK.  Instead, the Makefile should use
$(EXEEXT) or somesuch to account for the issue.  I don't see why OS/2
should be handled differently.  But maybe I'm missing something.

In general, settling for 'foo.exe' when the target is 'foo' can easily
cause false positives, so such a change should IMO not be introduced
without a serious discussion of the possible downsides and
regressions.

[bug #63650] Performance regression with EXPORT_ALL_VARIABLES enabled

[Eli Zaretskii]

Follow-up Comment #2, bug #63650 (project make):

??? Do you mean to say that commit f51fc130 caused this regression?  If so, I
don't understand: that commit changed code that is only used on MS-Windows,
whereas the OP is on Arch Linux...



___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[PATCH] Check .exe as well when a target does not exist on OS/2

[KO Myung-Hun]

For example,

foo: foo.c
gcc $@ $<

This pattern is usually used on UNIX. However, on OS/2, gcc creates
foo.exe not foo when an extension is not present, and Make check foo
only. Therefore Make tries to build foo whenever called.

* src/remake.c (f_mtime) [EMX]: Check a target again by appending .exe.
---
 src/remake.c | 29 +
 1 file changed, 29 insertions(+)

diff --git a/src/remake.c b/src/remake.c
index 62b3d791..99285498 100644
--- a/src/remake.c
+++ b/src/remake.c
@@ -1409,6 +1409,12 @@ f_mtime (struct file *file, int search)
   else
 #endif
 {
+#ifdef __EMX__
+  const char *saved_name = file->name;
+  size_t fname_len;
+
+try_again_with_dot_exe:
+#endif
   mtime = name_mtime (file->name);
 
   if (mtime == NONEXISTENT_MTIME && search && !file->ignore_vpath)
@@ -1452,6 +1458,29 @@ f_mtime (struct file *file, int search)
 mtime = name_mtime (name);
 }
 }
+#ifdef __EMX__
+  if (mtime == NONEXISTENT_MTIME)
+{
+  if (stricmp (_getext2 (file->name), ".exe"))
+{
+  fname_len = strlen (file->name);
+
+  file->name = alloca (fname_len + 4/*.exe*/ + 1);
+  memcpy ((char *)file->name, saved_name, fname_len);
+  memcpy ((char *)file->name + fname_len, ".exe", 4 + 1);
+
+  goto try_again_with_dot_exe;
+}
+}
+
+  fname_len = strlen (saved_name);
+
+  /* If found in VPATH, use it. Otherwise restore file->name. */
+  if (strlen (file->name) == fname_len + 4
+  && !strnicmp (file->name, saved_name, fname_len)
+  && !stricmp (file->name + fname_len, ".exe"))
+file->name = saved_name;
+#endif
 }
 
   /* Files can have bogus timestamps that nothing newly made will be
-- 
2.30.0

[bug #63650] Performance regression with EXPORT_ALL_VARIABLES enabled

[anonymous]

Follow-up Comment #1, bug #63650 (project make):

Update: reproduces when building from git. Commit sha f51fc130, message:
[SV 63638] Fix processing PATH on MS-Windows


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

[bug #63650] Performance regression with EXPORT_ALL_VARIABLES enabled

[anonymous]

URL:
  

 Summary: Performance regression with EXPORT_ALL_VARIABLES
enabled 
 Project: make
   Submitter: None
   Submitted: Fri 13 Jan 2023 12:33:13 PM UTC
Severity: 3 - Normal
  Item Group: Bug
  Status: None
 Privacy: Public
 Assigned to: None
 Open/Closed: Open
 Discussion Lock: Any
   Component Version: 4.4
Operating System: POSIX-Based
   Fixed Release: None
   Triage Status: None


___

Follow-up Comments:


---
Date: Fri 13 Jan 2023 12:33:13 PM UTC By: Anonymous
On fresh Arch Linux make takes 3.5s for the following makefile.


.EXPORT_ALL_VARIABLES:

AAA = $(shell echo 1)
AAB = $(shell echo 2)
AAC = $(shell echo 3)
AAD = $(shell echo 4)
AAE = $(shell echo 5)
AAF = $(shell echo 6)

slow:
echo ${AAA}
echo ${AAB}
echo ${AAC}
echo ${AAD}
echo ${AAE}
echo ${AAF}


Version: GNU Make 4.4
This reproduces with make compiled from sources from ftp.gnu.org.
The issue is not present in make 4.3.

make -d prints many lines like

Makefile:4: not recursively expanding AAB to export to shell function
Makefile:5: not recursively expanding AAC to export to shell function
Makefile:6: not recursively expanding AAD to export to shell function









___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/