Re: [bug #60538] Environment variable overrides explicit assignment

[Shareef Jalloq]

OK, just to clarify sorry, the issue here is that MAKEFLAGS are propagated
down to any sub-shells from my main command line?  So I can work around
this by just clearing MAKEFLAGS in my call to subprocess.run() when I pass
in the env.  Is that the only way to run a "clean" make command from a
sub-shell?

On Thu, May 6, 2021 at 5:08 PM Shareef Jalloq  wrote:

> Ah ok, thanks all.  I had no idea that command line args were treated
> differently to environment variables.  I'll read up on overrides.
>
> Cheers.
>
> On Thu, May 6, 2021 at 5:01 PM Paul D. Smith 
> wrote:
>
>> Update of bug #60538 (project make):
>>
>>   Status:None => Not A Bug
>>
>>  Open/Closed:Open => Closed
>>
>>
>> ___
>>
>> Follow-up Comment #5:
>>
>> As Martin says, if the variable is set ON THE COMMAND LINE, it will
>> absolutely
>> take precedence over the setting in a makefile variable (unless overridden
>> with override).
>>
>> But, you said you has set this IN THE ENVIRONMENT which is very different;
>> values set in the environment should not take precedence over values set
>> in
>> the makefile (as long as -e is not set).
>>
>> So, you'll have to investigate why/how this variable assignment is
>> appearing
>> in your command line (in MAKEFLAGS).
>>
>> I'm going to close this; you can continue to post here but better would
>> be to
>> discuss on the mailing list, until/unless we discover a bug.
>>
>> Cheers!
>>
>> ___
>>
>> Reply to this item at:
>>
>>   
>>
>> ___
>>   Message sent via Savannah
>>   https://savannah.gnu.org/
>>
>>

Re: [bug #60538] Environment variable overrides explicit assignment

[Shareef Jalloq]

Ah ok, thanks all.  I had no idea that command line args were treated
differently to environment variables.  I'll read up on overrides.

Cheers.

On Thu, May 6, 2021 at 5:01 PM Paul D. Smith 
wrote:

> Update of bug #60538 (project make):
>
>   Status:None => Not A Bug
>
>  Open/Closed:Open => Closed
>
>
> ___
>
> Follow-up Comment #5:
>
> As Martin says, if the variable is set ON THE COMMAND LINE, it will
> absolutely
> take precedence over the setting in a makefile variable (unless overridden
> with override).
>
> But, you said you has set this IN THE ENVIRONMENT which is very different;
> values set in the environment should not take precedence over values set in
> the makefile (as long as -e is not set).
>
> So, you'll have to investigate why/how this variable assignment is
> appearing
> in your command line (in MAKEFLAGS).
>
> I'm going to close this; you can continue to post here but better would be
> to
> discuss on the mailing list, until/unless we discover a bug.
>
> Cheers!
>
> ___
>
> Reply to this item at:
>
>   
>
> ___
>   Message sent via Savannah
>   https://savannah.gnu.org/
>
>

Re: [bug #60538] Environment variable overrides explicit assignment

[Martin Dorey]

> The manual doesn't actually say that MAKEOVERRIDES is exported like MAKEFLAGS 
> but I assume it is.

I was going to just say that MAKEFLAGS would be expanded before being exported, 
and indeed it is, but, having tried it, the results aren't quite what I would 
have expected, even having just skimmed the section on MAKEOVERRIDES that Paul 
just cited.

$ make TESTNAME=badger 2>&1 | grep MAKE
MAKEFLAGS= -- TESTNAME=badger
... eliding mentions of MAKE in my environment already...
MAKEOVERRIDES=${-*-command-variables-*-}
MAKELEVEL=1
$

So MAKEOVERRIDES is exported, though perhaps it doesn't need to be, but isn't 
expanded before being exported.


From: David Boyce 
Sent: Thursday, May 6, 2021 09:48
To: Paul D. Smith 
Cc: Shareef Jalloq ; Martin Dorey 
; Boris Kolpackov ; 
bug-make 
Subject: Re: [bug #60538] Environment variable overrides explicit assignment

* EXTERNAL EMAIL *
Most likely he thinks/thought that since there are two different make processes 
with a python process in between the command line override wouldn't leak to the 
child make. But (from the manual):

> Likewise variables defined on the command line are passed to the sub-make 
> through MAKEFLAGS. (5.7.3)

> The special variable MAKEFLAGS is always exported (unless you unexport it). 
> (5.7.2)

The manual doesn't actually say that MAKEOVERRIDES is exported like MAKEFLAGS 
but I assume it is.

On Thu, May 6, 2021 at 9:07 AM Paul D. Smith 
mailto:invalid.nore...@gnu.org>> wrote:
Update of bug #60538 (project make):

  Status:None => Not A Bug
 Open/Closed:Open => Closed

___

Follow-up Comment #5:

As Martin says, if the variable is set ON THE COMMAND LINE, it will absolutely
take precedence over the setting in a makefile variable (unless overridden
with override).

But, you said you has set this IN THE ENVIRONMENT which is very different;
values set in the environment should not take precedence over values set in
the makefile (as long as -e is not set).

So, you'll have to investigate why/how this variable assignment is appearing
in your command line (in MAKEFLAGS).

I'm going to close this; you can continue to post here but better would be to
discuss on the mailing list, until/unless we discover a bug.

Cheers!

___

Reply to this item at:

  
>

___
  Message sent via Savannah
  
https://savannah.gnu.org/

Re: [bug #60538] Environment variable overrides explicit assignment

[Paul Smith]

On Thu, 2021-05-06 at 17:11 +, Martin Dorey wrote:
> So MAKEOVERRIDES is exported, though perhaps it doesn't need to be,
> but isn't expanded before being exported.

Oh duh!!  I didn't provide any overrides.

You're right, it is exported.  OK I guess I'll have to go back and look
at the code but it will not be today.

Re: [bug #60538] Environment variable overrides explicit assignment

[Paul Smith]

On Thu, 2021-05-06 at 17:11 +, Martin Dorey wrote:
> > The manual doesn't actually say that MAKEOVERRIDES is exported like
> MAKEFLAGS but I assume it is.
> 
> I was going to just say that MAKEFLAGS would be expanded before being
> exported, and indeed it is, but, having tried it, the results aren't
> quite what I would have expected, even having just skimmed the
> section on MAKEOVERRIDES that Paul just cited.
> 
> $ make TESTNAME=badger 2>&1 | grep MAKE
> MAKEFLAGS= -- TESTNAME=badger
> ... eliding mentions of MAKE in my environment already...
> MAKEOVERRIDES=${-*-command-variables-*-}
> MAKELEVEL=1
> $ 
> 
> So MAKEOVERRIDES is exported, though perhaps it doesn't need to be,
> but isn't expanded before being exported.

Without seeing your makefile I can't say what you're seeing but I don't
see it exported:

  $ echo 'all:;@echo "mo=$$MAKEOVERRIDES"' | make -f-
  mo=

Re: [bug #60538] Environment variable overrides explicit assignment

[Paul Smith]

On Thu, 2021-05-06 at 09:48 -0700, David Boyce wrote:
> The manual doesn't actually say that MAKEOVERRIDES is exported like
> MAKEFLAGS but I assume it is.

No, it's not.

The value of MAKEOVERRIDES is used within the makefile itself.  Its
value is added to the MAKEFLAGS variable when invoking a sub-make.

The only environment variable make uses to communicate between one make
and its sub-make is via MAKEFLAGS.

Re: [bug #60538] Environment variable overrides explicit assignment

[Paul Smith]

On Thu, 2021-05-06 at 17:27 +0100, Shareef Jalloq wrote:
> OK, just to clarify sorry, the issue here is that MAKEFLAGS are
> propagated down to any sub-shells from my main command line?  So I
> can work around this by just clearing MAKEFLAGS in my call to
> subprocess.run() when I pass in the env.  Is that the only way to run
> a "clean" make command from a sub-shell?

Sorry, I don't understand the question.

In general you should not modify MAKEFLAGS, unless you have a very good
reason.

If someone invokes "make FOO=bar" then that setting of FOO will
override any setting of FOO in the makefile (unless overridden).

That command line setting will also be passed down to any sub-makes
that are invoked, just like most other make options are passed down.
 This is generally what you want.

If there's a reason you DON'T want a specific command-line variable
override to be passed down then the safest way to handle that is by
changing the MAKEOVERRIDES variable.

All the details are in the manual as Martin mentioned:

https://www.gnu.org/software/make/manual/html_node/Options_002fRecursion.html

If, after reading that, you still have problems or questions let us
know.

Cheers!

Re: [bug #60538] Environment variable overrides explicit assignment

[David Boyce]

Most likely he thinks/thought that since there are two different make
processes with a python process in between the command line override
wouldn't leak to the child make. But (from the manual):

> Likewise variables defined on the command line are passed to the sub-make
through MAKEFLAGS. (5.7.3)

> The special variable MAKEFLAGS is always exported (unless you unexport
it). (5.7.2)

The manual doesn't actually say that MAKEOVERRIDES is exported like
MAKEFLAGS but I assume it is.

On Thu, May 6, 2021 at 9:07 AM Paul D. Smith 
wrote:

> Update of bug #60538 (project make):
>
>   Status:None => Not A Bug
>
>  Open/Closed:Open => Closed
>
>
> ___
>
> Follow-up Comment #5:
>
> As Martin says, if the variable is set ON THE COMMAND LINE, it will
> absolutely
> take precedence over the setting in a makefile variable (unless overridden
> with override).
>
> But, you said you has set this IN THE ENVIRONMENT which is very different;
> values set in the environment should not take precedence over values set in
> the makefile (as long as -e is not set).
>
> So, you'll have to investigate why/how this variable assignment is
> appearing
> in your command line (in MAKEFLAGS).
>
> I'm going to close this; you can continue to post here but better would be
> to
> discuss on the mailing list, until/unless we discover a bug.
>
> Cheers!
>
> ___
>
> Reply to this item at:
>
>   
>
> ___
>   Message sent via Savannah
>   https://savannah.gnu.org/
>
>
>

[bug #60538] Environment variable overrides explicit assignment

[Paul D. Smith]

Update of bug #60538 (project make):

  Status:None => Not A Bug  
 Open/Closed:Open => Closed 

___

Follow-up Comment #5:

As Martin says, if the variable is set ON THE COMMAND LINE, it will absolutely
take precedence over the setting in a makefile variable (unless overridden
with override).

But, you said you has set this IN THE ENVIRONMENT which is very different;
values set in the environment should not take precedence over values set in
the makefile (as long as -e is not set).

So, you'll have to investigate why/how this variable assignment is appearing
in your command line (in MAKEFLAGS).

I'm going to close this; you can continue to post here but better would be to
discuss on the mailing list, until/unless we discover a bug.

Cheers!

___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

[bug #60538] Environment variable overrides explicit assignment

[Martin Dorey]

Follow-up Comment #4, bug #60538 (project make):

Looks like it's working as designed to me.  If I do:

$ make -n -p -- TESTNAME=test_mtp_remap

... like in your original post:

> make blah TESTNAME=mytest1

... then I see that:

MAKEFLAGS = np -- $(MAKEOVERRIDES)

... where:

MAKEOVERRIDES = ${-*-command-variables-*-}

... and:

-*-command-variables-*- := TESTNAME=test_mtp_remap

https://www.gnu.org/software/make/manual/html_node/Override-Directive.html
explains what you would need to do to override such a command line assignment.

___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

[bug #60538] Environment variable overrides explicit assignment

[Shareef Jalloq]

Follow-up Comment #3, bug #60538 (project make):

Ah, sorry, should have thought about a mailing list.  I can move it there if
this doesn't help:

w -- TESTNAME=test_mtp_remap

Not modifying MAKEFLAGS as far as I know and using -e explicitly doesn't have
any effect on the variable.

___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

[bug #60538] Environment variable overrides explicit assignment

[Paul D. Smith]

Follow-up Comment #2, bug #60538 (project make):

Sounds like this discussion would be better on the bug-make@gnu.org or
help-m...@gnu.org mailing lists, at least until we determine there's a real
bug.

The only way this can happen that I know of (barring bugs) is by adding the
*-e* make command line option which causes environment variables to override
make variables.

Can you print the $(MAKEFLAGS) variable in your makefile to see if it's set? 
Are you modifying MAKEFLAGS anywhere in your setup, that might be done
incorrectly so make thinks you want to set the -e option?

___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

[bug #60538] Environment variable overrides explicit assignment

[Shareef Jalloq]

Follow-up Comment #1, bug #60538 (project make):

I've just tried running the make command using *env -i* and that prevents the
make variable from being overridden but obviously I lose the environment
required for the test.

env -i make 

___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

[bug #60538] Environment variable overrides explicit assignment

[Shareef Jalloq]

URL:
  

 Summary: Environment variable overrides explicit assignment
 Project: make
Submitted by: sjalloq
Submitted on: Thu 06 May 2021 02:24:37 PM UTC
Severity: 3 - Normal
  Item Group: Bug
  Status: None
 Privacy: Public
 Assigned to: None
 Open/Closed: Open
 Discussion Lock: Any
   Component Version: 4.3
Operating System: POSIX-Based
   Fixed Release: None
   Triage Status: None

___

Details:

I'm not sure what is going on here as I'm not able to create a testcase that
highlights the same failure.  This issue only happens when calling Make from a
Python subprocess running inside a Cocotb test (a simulation framework for RTL
simulation).

I have a pretty simple Makefile where I set a variable *TESTNAME* using an
explicit assignment:

TESTNAME=mytest2

and what I'm seeing is that variable being overridden from a matching
environment variable.

I'm calling Make to run a simulation (make blah TESTNAME=mytest1) and from
within that simulation, the Python environment makes a second call to Make
using subprocess in order to build a dependency.  That dependency
(TESTNAME=mytest2) isn't being built because the TESTNAME is overridden to be
mytest1.

I've built a number of testcases to check this and can't replicate the
failure.

Also, if I use any other environment variable name aside from TESTNAME, it
works as expected.  So is there something special about that variable name? 
It's not reserved or something?

What mechanism could there be that would allow an explicit assignment be
overridden like this?

Thanks.




___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/