Re: On overriding make variables from the environment...
SZEDER Gábor writes: > So, then it's either 'config.mak', or passing a 'CC=$CC' argument to > _all_ make commands, including those that are not supposed to build > anything, but only run the tests. I find the latter aesthetically not > particularly pleasing. The config.mak file is available for individual builder-testers to customize their build, and in the context of this discussion, I think the CI builder is just one particular individual who happens to be non human. If it is easy to throw suitable settings from within the CI configuration .yaml files into config.mak, I'd think that is exactly how the mechanism was invented to be used, so...
Re: On overriding make variables from the environment...
Hi Gábor,
On Wed, 17 Oct 2018, SZEDER Gábor wrote:
> On Tue, Oct 16, 2018 at 03:40:01PM -0700, Jonathan Nieder wrote:
> > SZEDER Gábor wrote:
> > > On Tue, Oct 16, 2018 at 02:54:56PM -0700, Jonathan Nieder wrote:
> > >> SZEDER Gábor wrote:
> >
> > >>> Our Makefile has lines like these:
> > >>>
> > >>> CFLAGS = -g -O2 -Wall
> > >>> CC = cc
> > >>> AR = ar
> > >>> SPATCH = spatch
> > [...]
> > >>> I'm not sure what to do. I'm fine with updating our 'ci/' scripts to
> > >>> explicitly respect CC in the environment (either by running 'make
> > >>> CC=$CC' or by writing $CC into 'config.mak'). Or I could update our
> > >>> Makefile to use '?=' for specific variables, but:
> > >>
> > >> That's a good question. I don't have a strong opinion myself, so I
> > >> tend to trust larger projects like Linux to have thought this through
> > >> more, and they use 'CC = cc' as well.
> > >
> > > I don't think Linux is a good example to follow in this case, see e.g.
> > > 6d62c983f7 (Makefile: Change the default compiler from "gcc" to "cc",
> > > 2011-12-20) (in short: Git is supposed to be buildable with compilers
> > > other than GCC as well, while Linux not really, so they can hardcode
> > > 'CC = gcc').
> >
> > Nowadays Linux builds with clang as well. People also have other
> > reasons to override the CC setting (cross-compiling, etc) and to
> > override other settings like CFLAGS.
> >
> > > Also, the projects I have on hand usually have 'CC = gcc' hardcoded in
> > > their Makefiles, too, but those Makefiles were generated by their
> > > ./configure script (which in turn by ./autogen.sh...), and those tend
> > > to write CC from the environment into the generated Makefiles.
> >
> > Yes, I think that's what makes travis's setup normally work. It makes
> > sense to me since ./configure is expected to have more implicit or
> > automatic behavior. For "make", I kind of like that it doesn't depend
> > on environment variables that I am not thinking about when debugging a
> > reported build problem.
> >
> > When building Git without autoconf, the corresponding place for
> > settings is config.mak. Would it make sense for the ci scripts to
> > write a config.mak file?
>
> A first I though it doesn't, but it turns out it acually does.
>
> 'ci/run-build-and-tests.sh' basically runs:
>
> make
> make test
>
> I naively put a 'CC=$CC' argument at the end of the first command,
> thinking it should Just Work... but then that second 'make test' got
> all clever on me, said "* new build flags", and then proceeded to
> rebuild everything with the default 'cc'. (With the additional
> complication, that on Travis CI we actually run 'make --quiet test',
> which rebuilds everything, well, quietly... so the rebuild itself is
> not even visible in the build logs.)
>
> So, then it's either 'config.mak', or passing a 'CC=$CC' argument to
> _all_ make commands, including those that are not supposed to build
> anything, but only run the tests. I find the latter aesthetically not
> particularly pleasing.
How about using `MAKEFLAGS`? I ran a quick test:
MAKEFLAGS='CC=blub' make -C .. git.o
make: Entering directory '/usr/src/git/wip'
* new build flags
CC git.o
/bin/sh: blub: command not found
In other words, you could add something like this to the ci/ script:
MAKEFLAGS="${MAKEFLAGS:+$MAKEFLAGS }CC=$CC"
export MAKEFLAGS
Ciao,
Dscho
Re: On overriding make variables from the environment...
On Tue, Oct 16, 2018 at 03:40:01PM -0700, Jonathan Nieder wrote: > SZEDER Gábor wrote: > > On Tue, Oct 16, 2018 at 02:54:56PM -0700, Jonathan Nieder wrote: > >> SZEDER Gábor wrote: > > >>> Our Makefile has lines like these: > >>> > >>> CFLAGS = -g -O2 -Wall > >>> CC = cc > >>> AR = ar > >>> SPATCH = spatch > [...] > >>> I'm not sure what to do. I'm fine with updating our 'ci/' scripts to > >>> explicitly respect CC in the environment (either by running 'make > >>> CC=$CC' or by writing $CC into 'config.mak'). Or I could update our > >>> Makefile to use '?=' for specific variables, but: > >> > >> That's a good question. I don't have a strong opinion myself, so I > >> tend to trust larger projects like Linux to have thought this through > >> more, and they use 'CC = cc' as well. > > > > I don't think Linux is a good example to follow in this case, see e.g. > > 6d62c983f7 (Makefile: Change the default compiler from "gcc" to "cc", > > 2011-12-20) (in short: Git is supposed to be buildable with compilers > > other than GCC as well, while Linux not really, so they can hardcode > > 'CC = gcc'). > > Nowadays Linux builds with clang as well. People also have other > reasons to override the CC setting (cross-compiling, etc) and to > override other settings like CFLAGS. > > > Also, the projects I have on hand usually have 'CC = gcc' hardcoded in > > their Makefiles, too, but those Makefiles were generated by their > > ./configure script (which in turn by ./autogen.sh...), and those tend > > to write CC from the environment into the generated Makefiles. > > Yes, I think that's what makes travis's setup normally work. It makes > sense to me since ./configure is expected to have more implicit or > automatic behavior. For "make", I kind of like that it doesn't depend > on environment variables that I am not thinking about when debugging a > reported build problem. > > When building Git without autoconf, the corresponding place for > settings is config.mak. Would it make sense for the ci scripts to > write a config.mak file? A first I though it doesn't, but it turns out it acually does. 'ci/run-build-and-tests.sh' basically runs: make make test I naively put a 'CC=$CC' argument at the end of the first command, thinking it should Just Work... but then that second 'make test' got all clever on me, said "* new build flags", and then proceeded to rebuild everything with the default 'cc'. (With the additional complication, that on Travis CI we actually run 'make --quiet test', which rebuilds everything, well, quietly... so the rebuild itself is not even visible in the build logs.) So, then it's either 'config.mak', or passing a 'CC=$CC' argument to _all_ make commands, including those that are not supposed to build anything, but only run the tests. I find the latter aesthetically not particularly pleasing.
Re: On overriding make variables from the environment...
SZEDER Gábor wrote: > On Tue, Oct 16, 2018 at 02:54:56PM -0700, Jonathan Nieder wrote: >> SZEDER Gábor wrote: >>> Our Makefile has lines like these: >>> >>> CFLAGS = -g -O2 -Wall >>> CC = cc >>> AR = ar >>> SPATCH = spatch [...] >>> I'm not sure what to do. I'm fine with updating our 'ci/' scripts to >>> explicitly respect CC in the environment (either by running 'make >>> CC=$CC' or by writing $CC into 'config.mak'). Or I could update our >>> Makefile to use '?=' for specific variables, but: >> >> That's a good question. I don't have a strong opinion myself, so I >> tend to trust larger projects like Linux to have thought this through >> more, and they use 'CC = cc' as well. > > I don't think Linux is a good example to follow in this case, see e.g. > 6d62c983f7 (Makefile: Change the default compiler from "gcc" to "cc", > 2011-12-20) (in short: Git is supposed to be buildable with compilers > other than GCC as well, while Linux not really, so they can hardcode > 'CC = gcc'). Nowadays Linux builds with clang as well. People also have other reasons to override the CC setting (cross-compiling, etc) and to override other settings like CFLAGS. > Also, the projects I have on hand usually have 'CC = gcc' hardcoded in > their Makefiles, too, but those Makefiles were generated by their > ./configure script (which in turn by ./autogen.sh...), and those tend > to write CC from the environment into the generated Makefiles. Yes, I think that's what makes travis's setup normally work. It makes sense to me since ./configure is expected to have more implicit or automatic behavior. For "make", I kind of like that it doesn't depend on environment variables that I am not thinking about when debugging a reported build problem. When building Git without autoconf, the corresponding place for settings is config.mak. Would it make sense for the ci scripts to write a config.mak file? Thanks, Jonathan
Re: On overriding make variables from the environment...
On Tue, Oct 16, 2018 at 02:54:56PM -0700, Jonathan Nieder wrote: > SZEDER Gábor wrote: > > Our Makefile has lines like these: > > > > CFLAGS = -g -O2 -Wall > > CC = cc > > AR = ar > > SPATCH = spatch > > > > Note the use of '=', not '?='. > [...] > > I'm not sure what to do. I'm fine with updating our 'ci/' scripts to > > explicitly respect CC in the environment (either by running 'make > > CC=$CC' or by writing $CC into 'config.mak'). Or I could update our > > Makefile to use '?=' for specific variables, but: > > That's a good question. I don't have a strong opinion myself, so I > tend to trust larger projects like Linux to have thought this through > more, and they use 'CC = cc' as well. I don't think Linux is a good example to follow in this case, see e.g. 6d62c983f7 (Makefile: Change the default compiler from "gcc" to "cc", 2011-12-20) (in short: Git is supposed to be buildable with compilers other than GCC as well, while Linux not really, so they can hardcode 'CC = gcc'). Also, the projects I have on hand usually have 'CC = gcc' hardcoded in their Makefiles, too, but those Makefiles were generated by their ./configure script (which in turn by ./autogen.sh...), and those tend to write CC from the environment into the generated Makefiles.
Re: On overriding make variables from the environment...
Hi,
SZEDER Gábor wrote:
> Our Makefile has lines like these:
>
> CFLAGS = -g -O2 -Wall
> CC = cc
> AR = ar
> SPATCH = spatch
>
> Note the use of '=', not '?='.
[...]
> I'm not sure what to do. I'm fine with updating our 'ci/' scripts to
> explicitly respect CC in the environment (either by running 'make
> CC=$CC' or by writing $CC into 'config.mak'). Or I could update our
> Makefile to use '?=' for specific variables, but:
That's a good question. I don't have a strong opinion myself, so I
tend to trust larger projects like Linux to have thought this through
more, and they use 'CC = cc' as well. So I'd lean toward the updating
'ci/' scripts approach, to do something like
make ${CC:+"CC=$CC"} ...
(or the equivalent multi-line construction).
That also has the bonus of being explicit.
Just my two cents,
Jonathan
On overriding make variables from the environment...
Our Makefile has lines like these: CFLAGS = -g -O2 -Wall CC = cc AR = ar SPATCH = spatch Note the use of '=', not '?='. This means that these variables can be overridden from the command line, i.e. 'make CC=gcc-X.Y' will build with that particular GCC version, but not from the environment, i.e. 'CC=gcc-X.Y make' will still build with 'cc'. This can be confusing for developers who come from other projects where they used to run 'CC=whatever make'. And our build jobs on Travis CI are badly affected by this. We have dedicated build jobs to build Git with GCC and Clang both on Linux and OSX from the very beginning (522354d70f (Add Travis CI support, 2015-11-27)). But guess how Travis CI specifies which compiler to use! With 'export CC=gcc' and 'export CC=clang', respectively. Consequently, our Clang Linux build job has always used gcc, because that's where 'cc' points at on Linux by default, while the GCC OSX build job has always used Clang. Oh, well. Furthermore, see 37fa4b3c78 (travis-ci: run gcc-8 on linux-gcc jobs, 2018-05-19), where Duy added an 'export CC=gcc-8' in an attempt to use a more modern compiler, but this had no effect either. I'm not sure what to do. I'm fine with updating our 'ci/' scripts to explicitly respect CC in the environment (either by running 'make CC=$CC' or by writing $CC into 'config.mak'). Or I could update our Makefile to use '?=' for specific variables, but: - I'm afraid to break somebody's setup relying on the current behavior and CC having different values in the environment and in 'config.mak'. - Where to stop, IOW which variables should be set with '?='? CFLAGS, LDFLAGS, CC, AR, ..., SPATCH, SPATCH_FLAGS? Dunno. We already have 'STRIP ?= strip' and there are variables that are checked explicitly (e.g. 'DEVELOPER=y make' works). Note also that prior to b05701c5b4 (Make CFLAGS overridable from make command line., 2005-08-06) our Makefile used 'CC?=gcc' as well.
