Re: exported vs command line variables

[Edward Welbourne]

On Fri, 2016-09-23 at 13:24 +, Edward Welbourne wrote:
>> This does seem like an unsound choice, for the reasons David gave,
Paul Smith replied:
> Perhaps but as discussed, it's been that way in GNU make (and other
> versions of make) forever and is required by the POSIX standard.  So
> we'll just have to live with it.

Indeed; some unsound choices get tenure,

 Eddy.
___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

Re: exported vs command line variables

[Paul Smith]

On Fri, 2016-09-23 at 16:09 -0700, David Boyce wrote:
> "Macros defined by the MAKEFLAGS environment variable and macros
> defined in the makefile(s) shall not be added to the environment of
> make if they are not already in its environment."

That text is not in the currently published version of the standard
(2008), so it's not normative (yet).  It's a change in behavior from the
current version.  I'm not sure I even have access to the draft.

However, I'm open to making this change as it doesn't seem like much of
an issue from a backwards-compatibility standpoint even though, as
Philip points out, the behavior of make when MAKEFLAGS is set in the
makefile is not specified.

On Fri, 2016-09-23 at 13:24 +, Edward Welbourne wrote:
> This does seem like an unsound choice, for the reasons David gave,

Perhaps but as discussed, it's been that way in GNU make (and other
versions of make) forever and is required by the POSIX standard.  So
we'll just have to live with it.

___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

Re: exported vs command line variables

[Philip Guenther]

On Fri, Sep 23, 2016 at 4:09 PM, David Boyce  wrote:
> In that case there does seem to be at least one bug here. In reference to
> the sentence:
>
> "Macros defined by the MAKEFLAGS environment variable and macros defined in
> the makefile(s) shall not be added to the environment of make if they are
> not already in its environment."
>
> And building on Eddy Welbourne's example:
>
> % cat Makefile
> WURGLE = 11
> MAKEFLAGS = WURGLE=$(WURGLE)

You need to read all the applicable bits in the standard before
calling behavior incorrect.  In particular, the Macros section has
this line earlier in it:
The result of setting MAKEFLAGS in the Makefile is unspecified.

So if you want POSIX-like behavior, don't do that; if you do that,
you're at the whims of GNU make's choices.


Philip Guenther

___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

Re: exported vs command line variables

[David Boyce]

In that case there does seem to be at least one bug here. In reference to
the sentence:

"Macros defined by the MAKEFLAGS environment variable and macros defined in
the makefile(s) shall not be added to the environment of make if they are
not already in its environment."

And building on Eddy Welbourne's example:

% cat Makefile
WURGLE = 11
MAKEFLAGS = WURGLE=$(WURGLE)

all:; env | grep ^WUR ||:

% make
env | grep ^WUR ||:
WURGLE=11

It appears that macros added to MAKEFLAGS do end up in the environment.

BTW another workaround for this behavior, in addition to inserting
non-alphanumeric characters into variable names, is to use "unexport FOO"
in the receiving makefile. I'd been experimenting with the MAKEFLAGS
technique too which is how I discovered the non-POSIX behavior above.

David

On Fri, Sep 23, 2016 at 2:52 PM, Philip Guenther  wrote:

> On Fri, Sep 23, 2016 at 11:23 AM, Paul Smith  wrote:
> > On Fri, 2016-09-23 at 05:52 -0700, David Boyce wrote:
> >> I still think that's a good plan, except that now I find out it ends
> >> up exporting anyway. I'm sure it's too late to change now but is
> >> there a reason for this (unfortunate IMHO) behavior?
> >
> > I can't tell you: this behavior has existed since the very first import
> > of GNU make code into source code control, back in 1991.
>
> ...and in System III in 1980:
> http://minnie.tuhs.org/cgi-bin/utree.pl?file=SysIII/usr/src/cmd/make
>
> Looks like BSD picked it up in 4.4 around 1993 with the switch to pmake:
> http://minnie.tuhs.org/cgi-bin/utree.pl?file=4.4BSD/usr/src/usr.bin/make
>
>
> POSIX specifies this behavior in the "Macros" subsection of the
> EXTENDED DESCRIPTION section for 'make'.  About a page and half in is
> this paragraph:
> Before the makefile(s) are read, all of the make utility command
> line macro definitions (except the
> MAKEFLAGS macro or the SHELL macro) shall be added to the
> environment of make. Other
> +   implementation-defined variables may also be added to the
> environment of make. Macros
> +  defined by the MAKEFLAGS environment variable and macros defined in
> the makefile(s) shall
> +   not be added to the environment of make if they are not already in
> its environment. With the
> +   exception of SHELL (see below), it is unspecified whether macros
> defined in these ways update
> the value of an environment variable that already exists in the
> environment of make.
>
> (That's from the 2013 draft update; the '+' lines were modified from
> the original 2008 version of the standard, but that doesn't affect the
> first sentence.)
>
>
> Philip Guenther
>
___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

Re: exported vs command line variables

[Philip Guenther]

On Fri, Sep 23, 2016 at 11:23 AM, Paul Smith  wrote:
> On Fri, 2016-09-23 at 05:52 -0700, David Boyce wrote:
>> I still think that's a good plan, except that now I find out it ends
>> up exporting anyway. I'm sure it's too late to change now but is
>> there a reason for this (unfortunate IMHO) behavior?
>
> I can't tell you: this behavior has existed since the very first import
> of GNU make code into source code control, back in 1991.

...and in System III in 1980:
http://minnie.tuhs.org/cgi-bin/utree.pl?file=SysIII/usr/src/cmd/make

Looks like BSD picked it up in 4.4 around 1993 with the switch to pmake:
http://minnie.tuhs.org/cgi-bin/utree.pl?file=4.4BSD/usr/src/usr.bin/make


POSIX specifies this behavior in the "Macros" subsection of the
EXTENDED DESCRIPTION section for 'make'.  About a page and half in is
this paragraph:
Before the makefile(s) are read, all of the make utility command
line macro definitions (except the
MAKEFLAGS macro or the SHELL macro) shall be added to the
environment of make. Other
+   implementation-defined variables may also be added to the
environment of make. Macros
+  defined by the MAKEFLAGS environment variable and macros defined in
the makefile(s) shall
+   not be added to the environment of make if they are not already in
its environment. With the
+   exception of SHELL (see below), it is unspecified whether macros
defined in these ways update
the value of an environment variable that already exists in the
environment of make.

(That's from the 2013 draft update; the '+' lines were modified from
the original 2008 version of the standard, but that doesn't affect the
first sentence.)


Philip Guenther

___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

Re: exported vs command line variables

[Edward Welbourne]

David Boyce observed:
> I was unpleasantly surprised to learn while following this discussion
> that "Except by explicit request, make exports a variable only if it
> is either defined in the environment initially _or set on the command
> line_".

... and I was surprised to find (by experiment with


default:
/usr/bin/env

 invoked via make WURGLE=8, finding this indeed reported by env)
that this really does mean export *to the environment*, where I had
supposed it was exported only to subordinate make invocations (by virtue
of being part of MAKEFLAGS).

This does seem like an unsound choice, for the reasons David gave,

Eddy.

___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

Re: exported vs command line variables

[David Boyce]

Changing the subject a bit, I was unpleasantly surprised to learn while
following this discussion that "Except by explicit request, make exports a
variable only if it is either defined in the environment initially _or set
on the command line_". For years I've noticed a tendency for users to mark
variables for export indiscriminately in order to get them to some child
make. Sort of a "big hammer" approach, which I dislike for the same reason
global variables etc are deprecated: once every child process (not just
every child make) can see a variable, there's no way to know who depends on
it and the result is both harder to debug and harder to clean up. I
generally advise people that rather than:

export CFLAGS
...
$(MAKE) -C subdir ...

They should pass required variables on the command line:

$(MAKE) -C subdir CFLAGS="$(CFLAGS)"

to keep scoping under control. I still think that's a good plan, except
that now I find out it ends up exporting anyway. I'm sure it's too late to
change now but is there a reason for this (unfortunate IMHO) behavior?

The only workaround I can think of would be to rely on non-alphanumeric
names like $(C-FLAGS) but that's a long way from elegant or obvious.

-David



On Tue, Sep 20, 2016 at 11:24 AM, Paul Smith  wrote:

> On Tue, 2016-09-20 at 15:28 +, Pietro wrote:
> > Hi,
> >
> > I have noticed that there is a difference between the two scenarios
> > listed below:
> >
> > i) make  CC=arm-linux-gnueabihf-gcc CPPFLAGS=[..]
> >
> > ii) export CPPFLAGS=[..] [RET]
> > make CC=arm-linux-gnueabihf-gcc
> >
> > I was told that the difference is that in the second case Make
> > will append stuff to the CPPFLAGS variable exported while in the
> > first case the command line argument will override the settings
> > contained in the makefile itself.
>
> That is definitely not true.  Whether or not the values here replace or
> append to any value of CPPFLAGS in the makefile depends entirely on how
> the variable is set in the makefile.
>
> GNU make has a hierarchy of precedence when it comes to how variables
> are set.  The weakest setting is values that are built-in to make.
>  Next highest is values taken from the environment.  Those are
> overridden by values set in the makefile itself.  And finally values
> set on the command line have the highest precedence.
>
> So if you have a makefile like this:
>
>   CPPFLAGS = -Wall
>
>   all: ; echo 'CPPFLAGS=$(CPPFLAGS)'
>
> Then if you run:
>
>   $ CPPFLAGS=-Werror make
>   CPPFLAGS=-Wall
>
> and the makefile setting overrides the environment setting.  But:
>
>   $ make CPPFLAGS=-Werror
>   CPPFLAGS=-Werror
>
> command line settings take precedence over makefile settings.  In
> neither case will the value in the environment or command line be
> appended to the value in the makefile: it always replaces the value.
>
> There are ways you can manipulate the priority, using the -e command
> line option and the override specifier on variable assignment (see the
> GNU make manual for this).
>
> > The following example, literally pasted from another thread could
> > clarify the matter :
> >
> > var := $(shell echo "echo hi" >say_hi.sh; chmod +x say_hi.sh;
> > say_hi.sh)
> > all: ; @echo $(var)
>
> This actually just adds a lot of confusion and doesn't clarify it at
> all, because (a) you're using PATH environment variable here which is
> "special", at least nominally and (b) you're using $(shell ...) not
> recipes, and the behavior of the shell function with respect to the
> environment is different from that of recipes.  This example seems to
> me to be completely different than what you asked about above, such
> that you can't really compare them.
>
> > I have posted a question at the general make mailing list and I was
> > told that this might be a bug, I would love to have a clarification -
> > is it a bug ?
>
> I think this is more a matter of you wanted to do something and you
> followed a path and got stuck, so you asked about what got you stuck.
>
> I think you should back up and ask about what you are really trying to
> do in the first place, with an example, because likely the path you
> followed is not the one you want so discussing where you're stuck won't
> really help.
>
> ___
> Bug-make mailing list
> Bug-make@gnu.org
> https://lists.gnu.org/mailman/listinfo/bug-make
>
___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

Re: exported vs command line variables

[Pietro]

Paul Smith  writes:

> On Tue, 2016-09-20 at 15:28 +, Pietro wrote:
>> Hi,
>> 
>> I have noticed that there is a difference between the two scenarios
>> listed below:
>> 
>> i) make  CC=arm-linux-gnueabihf-gcc CPPFLAGS=[..]
>> 
>> ii) export CPPFLAGS=[..] [RET]
>> make CC=arm-linux-gnueabihf-gcc
>> 
>> I was told that the difference is that in the second case Make
>> will append stuff to the CPPFLAGS variable exported while in the
>> first case the command line argument will override the settings
>> contained in the makefile itself.
>
> That is definitely not true.  Whether or not the values here replace or
> append to any value of CPPFLAGS in the makefile depends entirely on how
> the variable is set in the makefile.
>
> GNU make has a hierarchy of precedence when it comes to how variables
> are set.  The weakest setting is values that are built-in to make.
>  Next highest is values taken from the environment.  Those are
> overridden by values set in the makefile itself.  And finally values
> set on the command line have the highest precedence.
>
> So if you have a makefile like this:
>
>   CPPFLAGS = -Wall
>
>   all: ; echo 'CPPFLAGS=$(CPPFLAGS)'
>
> Then if you run:
>
>   $ CPPFLAGS=-Werror make
>   CPPFLAGS=-Wall
>
> and the makefile setting overrides the environment setting.  But:
>
>   $ make CPPFLAGS=-Werror
>   CPPFLAGS=-Werror
>
> command line settings take precedence over makefile settings.  In
> neither case will the value in the environment or command line be
> appended to the value in the makefile: it always replaces the value.
>
> There are ways you can manipulate the priority, using the -e command
> line option and the override specifier on variable assignment (see the
> GNU make manual for this).  
>
>> The following example, literally pasted from another thread could
>> clarify the matter :
>> 
>> var := $(shell echo "echo hi" >say_hi.sh; chmod +x say_hi.sh;
>> say_hi.sh)
>> all: ; @echo $(var)
>
> This actually just adds a lot of confusion and doesn't clarify it at
> all, because (a) you're using PATH environment variable here which is
> "special", at least nominally and (b) you're using $(shell ...) not
> recipes, and the behavior of the shell function with respect to the
> environment is different from that of recipes.  This example seems to
> me to be completely different than what you asked about above, such
> that you can't really compare them.
>
>> I have posted a question at the general make mailing list and I was
>> told that this might be a bug, I would love to have a clarification -
>> is it a bug ?
>
> I think this is more a matter of you wanted to do something and you
> followed a path and got stuck, so you asked about what got you stuck.
>
> I think you should back up and ask about what you are really trying to
> do in the first place, with an example, because likely the path you
> followed is not the one you want so discussing where you're stuck won't
> really help.
>
> ___
> Bug-make mailing list
> Bug-make@gnu.org
> https://lists.gnu.org/mailman/listinfo/bug-make
Thanks a lot, really.

I was trying to cross-compile a library and reading a forum about
the library itself I have read this sentence:

--
Try this instead:

CPPFLAGS=-I/home/rafael/mobi/ArmLibs/include \ 
LDFLAGS=-L/home/rafael/mobi/ArmLibs/lib \
make HAS_PKG_CONFIG=false CC=arm-xilinx-linux-gnueabi-gcc
CXX=arm-xilinx-linux-gnueabi-g++ \
LD=arm-xilinx-linux-gnueabi-gcc LDXX=arm-xilinx-linux-gnueabi-g++ \
AR=arm-xilinx-linux-gnueabi-ar

*CPPFLAGS and LDFLAGS must be defined as environment variables instead of
command line arguments, as we're adding to them.*

-

At the beginning I thought there was something under the hood taking
place I was not aware of, but in fact I now realize my understanding of
the hierarchy of variable was ... poor: so given the rules you have
explained me that is the result of how that *specific* makefile is
written.

When asking the same question on the general mailing list I was
suggested that might be a bug that is why I posted a question here.

There is not bug whatsoever :-)
Thanks,
P.




___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

Re: exported vs command line variables

[Paul Smith]

On Tue, 2016-09-20 at 15:28 +, Pietro wrote:
> Hi,
> 
> I have noticed that there is a difference between the two scenarios
> listed below:
> 
> i) make  CC=arm-linux-gnueabihf-gcc CPPFLAGS=[..]
> 
> ii) export CPPFLAGS=[..] [RET]
> make CC=arm-linux-gnueabihf-gcc
> 
> I was told that the difference is that in the second case Make
> will append stuff to the CPPFLAGS variable exported while in the
> first case the command line argument will override the settings
> contained in the makefile itself.

That is definitely not true.  Whether or not the values here replace or
append to any value of CPPFLAGS in the makefile depends entirely on how
the variable is set in the makefile.

GNU make has a hierarchy of precedence when it comes to how variables
are set.  The weakest setting is values that are built-in to make.
 Next highest is values taken from the environment.  Those are
overridden by values set in the makefile itself.  And finally values
set on the command line have the highest precedence.

So if you have a makefile like this:

  CPPFLAGS = -Wall

  all: ; echo 'CPPFLAGS=$(CPPFLAGS)'

Then if you run:

  $ CPPFLAGS=-Werror make
  CPPFLAGS=-Wall

and the makefile setting overrides the environment setting.  But:

  $ make CPPFLAGS=-Werror
  CPPFLAGS=-Werror

command line settings take precedence over makefile settings.  In
neither case will the value in the environment or command line be
appended to the value in the makefile: it always replaces the value.

There are ways you can manipulate the priority, using the -e command
line option and the override specifier on variable assignment (see the
GNU make manual for this).  

> The following example, literally pasted from another thread could
> clarify the matter :
> 
> var := $(shell echo "echo hi" >say_hi.sh; chmod +x say_hi.sh;
> say_hi.sh)
> all: ; @echo $(var)

This actually just adds a lot of confusion and doesn't clarify it at
all, because (a) you're using PATH environment variable here which is
"special", at least nominally and (b) you're using $(shell ...) not
recipes, and the behavior of the shell function with respect to the
environment is different from that of recipes.  This example seems to
me to be completely different than what you asked about above, such
that you can't really compare them.

> I have posted a question at the general make mailing list and I was
> told that this might be a bug, I would love to have a clarification -
> is it a bug ?

I think this is more a matter of you wanted to do something and you
followed a path and got stuck, so you asked about what got you stuck.

I think you should back up and ask about what you are really trying to
do in the first place, with an example, because likely the path you
followed is not the one you want so discussing where you're stuck won't
really help.

___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

exported vs command line variables

[Pietro]

Hi,

I have noticed that there is a difference between the two scenarios
listed below:

i) make  CC=arm-linux-gnueabihf-gcc CPPFLAGS=[..]

ii) export CPPFLAGS=[..] [RET]
make CC=arm-linux-gnueabihf-gcc

I was told that the difference is that in the second case Make
will append stuff to the CPPFLAGS variable exported while in the first
case the command line argument will override the settings contained
in the makefile itself.

The following example, literally pasted from another thread could
clarify the matter :

var := $(shell echo "echo hi" >say_hi.sh; chmod +x say_hi.sh; say_hi.sh)
all: ; @echo $(var)

Calling make with the PATH environment variable either way yields:

$ make PATH=$PATH:.
/bin/sh: 1: say_hi.sh: not found

$ PATH=$PATH:. make
hi

Is this working as intended ? What is the difference between the two ?

I have posted a question at the general make mailing list and I was told
that this might be a bug, I would love to have a clarification - is it a bug ?


Thanks
P.


___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make