Re: Lintian warning: hardening-no-fortify-functions & version numbering

[Lars Wirzenius]

On Sun, Jun 24, 2012 at 11:29:56AM +0100, Lars Wirzenius wrote:
> On Sat, Jun 23, 2012 at 03:27:07PM -0700, Russ Allbery wrote:
> > CFLAGS is an old, long-standing make thing.  CPPFLAGS is newer and I
> > believe was introduced by Autoconf
> 
> I don't know the history of CPPFLAGS. It's possible it was introduced by
> Autoconf. However, it is now embedded into the implicit rules of GNU Make,
> so every Makefile that doesn't override the rule for compiling a .c file
> to a .o file uses CPPFLAGS.

I peeked around the Internet a bit, in the hope that adding some more
fact into this pointless discussion will make it stop.

CPPFLAGS was used by the GNU coding standards at least since January 11,
1993, which is when the document was imported into CVS. I failed to find
an older copy, but I'm fairly sure the coding standards are older than
that.

GNU Make documents CPPINFO in Sep 18, 1991. That's about when Autoconf
started (and also approximately when the first release of Linux happened;
heady times).

I think Autoconf is a red herring, whether CPPFLAGS originated there
or not. GNU Make supports it, and has done so for _decades_.

-- 
I wrote a book: http://gtdfh.branchable.com/


signature.asc
Description: Digital signature

Re: Lintian warning: hardening-no-fortify-functions & version numbering

[Dmitrijs Ledkovs]

On 27/06/12 14:20, Ben Hutchings wrote:
> On Wed, 2012-06-27 at 14:09 +0300, Serge wrote:
>> 2012/6/25 Ben Hutchings wrote:
>>
 BTW, it's interesting that Fedora/CentOS use -Wp,-D_FORTIFY_SOURCE=2
 and they use it in CFLAGS/CXXFLAGS.
>>>
>>> Presumably as a workaround for build systems that do not respect
>>> CPPFLAGS.
>>
>> I actually noticed that because it's "-Wp,-D...", not "-D...". But I guess
>> you're right, it's in CFLAGS because many build systems support CFLAGS,
>> but only autotools support CPPFLAGS.
>>
>>> GNU make's implicit rules use CPPFLAGS.  If other build systems or
>>> overriden rules don't use it, it's a bug.  This can of course be
>>> worked around in debian/rules.
>>
>> Well, such argument can be applied to any build system. For example: Cmake
>> uses CMAKE_C_FLAGS, but GNU's make does not use it. It's a bug.
> 
> GNU make is the standard build sequencing tool for the GNU system (i.e.
> for Debian).  CMake and the others probably ought to follow the platform
> conventions.
> 
> [...]

Actually CMake *does* honour CFLAGS and copies them into CMAKE_C_FLAGS,
it doesn't do this for CPPFLAGS though.

Look at the other cmake packages how hardening flags are handled there.
Something like copying the set CPPFLAGS into CXXFlags or something.

>> Talking just about autotools:
>> * CPPFLAGS without CFLAGS are used only by ./configure script
>> * CPPFLAGS without CFLAGS are used only for some conftests
>> * -D_FORTIFY_SOURCE=2 means nothing for those tests
>> * -D_FORTIFY_SOURCE=2 does nothing at all without -O2
>> So even for autotools there's no reason to keep -D_FORTIFY_SOURCE=2 in
>> a CPPFLAGS variable. It can be easily dropped.
> [...]
> 
> I do take the point that it's not obviously useful to separate out
> CPPFLAGS.
> 
> Ben.
> 


-- 
Regards,
Dmitrijs.



signature.asc
Description: OpenPGP digital signature

Re: Lintian warning: hardening-no-fortify-functions & version numbering

[Ben Hutchings]

On Wed, 2012-06-27 at 14:09 +0300, Serge wrote:
> 2012/6/25 Ben Hutchings wrote:
> 
> >> BTW, it's interesting that Fedora/CentOS use -Wp,-D_FORTIFY_SOURCE=2
> >> and they use it in CFLAGS/CXXFLAGS.
> >
> > Presumably as a workaround for build systems that do not respect
> > CPPFLAGS.
> 
> I actually noticed that because it's "-Wp,-D...", not "-D...". But I guess
> you're right, it's in CFLAGS because many build systems support CFLAGS,
> but only autotools support CPPFLAGS.
> 
> > GNU make's implicit rules use CPPFLAGS.  If other build systems or
> > overriden rules don't use it, it's a bug.  This can of course be
> > worked around in debian/rules.
> 
> Well, such argument can be applied to any build system. For example: Cmake
> uses CMAKE_C_FLAGS, but GNU's make does not use it. It's a bug.

GNU make is the standard build sequencing tool for the GNU system (i.e.
for Debian).  CMake and the others probably ought to follow the platform
conventions.

[...]
> Talking just about autotools:
> * CPPFLAGS without CFLAGS are used only by ./configure script
> * CPPFLAGS without CFLAGS are used only for some conftests
> * -D_FORTIFY_SOURCE=2 means nothing for those tests
> * -D_FORTIFY_SOURCE=2 does nothing at all without -O2
> So even for autotools there's no reason to keep -D_FORTIFY_SOURCE=2 in
> a CPPFLAGS variable. It can be easily dropped.
[...]

I do take the point that it's not obviously useful to separate out
CPPFLAGS.

Ben.

-- 
Ben Hutchings
Lowery's Law:
 If it jams, force it. If it breaks, it needed replacing anyway.


signature.asc
Description: This is a digitally signed message part

Re: Lintian warning: hardening-no-fortify-functions & version numbering

[Serge]

2012/6/25 Ben Hutchings wrote:

>> BTW, it's interesting that Fedora/CentOS use -Wp,-D_FORTIFY_SOURCE=2
>> and they use it in CFLAGS/CXXFLAGS.
>
> Presumably as a workaround for build systems that do not respect
> CPPFLAGS.

I actually noticed that because it's "-Wp,-D...", not "-D...". But I guess
you're right, it's in CFLAGS because many build systems support CFLAGS,
but only autotools support CPPFLAGS.

> GNU make's implicit rules use CPPFLAGS.  If other build systems or
> overriden rules don't use it, it's a bug.  This can of course be
> worked around in debian/rules.

Well, such argument can be applied to any build system. For example: Cmake
uses CMAKE_C_FLAGS, but GNU's make does not use it. It's a bug. GNU make
must use CMAKE_C_FLAGS. And dpkg must set it too. Even more, dpkg should
set only CMAKE_C_FLAGS, CMAKE_CXX_FLAGS and CMAKE_SHARED_LINKER_FLAGS.
Yes, this may create some problems for people packaging autotools, but
this can of course be worked around in debian/rules.

Come on. :) How many build systems except autotools are using CPPFLAGS?
cmake, qmake, imake, nmake? It looks completely autotools-specific to me.
And forcing it is no better than forcing qmake DEFINES or CMAKE_C_FLAGS.
It just adds more work to packagers, but makes nothing better.

Talking just about autotools:
* CPPFLAGS without CFLAGS are used only by ./configure script
* CPPFLAGS without CFLAGS are used only for some conftests
* -D_FORTIFY_SOURCE=2 means nothing for those tests
* -D_FORTIFY_SOURCE=2 does nothing at all without -O2
So even for autotools there's no reason to keep -D_FORTIFY_SOURCE=2 in
a CPPFLAGS variable. It can be easily dropped.

Also dropping CPPFLAGS would also allow:
1. Use same rules for cmake and autotools packages.
2. Make every rules file a few lines shorter, thus a bit faster to build
3. Make "Hardening" wiki page shorter
4. Reduce the number of workarounds by one
5. Reduce the number of things people need to think about when packaging,
making packaging a little bit easier.

And no disadvantages on the other hand. :)

If it's not fixed in dpkg, as you said, it can be worked around in
debian/rules. I'm just trying to avoid adding unnecessary workarounds
to debian packages.

-- 
  Serge


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/CAOVenEqf8dXxr5-W7Rd==QoJq6n_gwaFqxFt0=bpoqf1yr5...@mail.gmail.com

Re: Lintian warning: hardening-no-fortify-functions & version numbering

[Bernhard R. Link]

* Ben Hutchings  [120625 19:21]:
> GNU make's implicit rules use CPPFLAGS.  If other build systems or
> overriden rules don't use it, it's a bug.  This can of course be
> worked around in debian/rules.

I'd not call it a bug. It's just some stranger behavior. Not more strange
than not using make but some python or shell script to do the actual
compilation or any other of the many strange things found in software
packages.

Bernhard R. Link


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20120625184310.gd...@client.brlink.eu

Re: Lintian warning: hardening-no-fortify-functions & version numbering

[Ben Hutchings]

On Mon, Jun 25, 2012 at 06:55:53PM +0300, Serge wrote:
> 2012/6/24 Guillem Jover wrote:
> 
> >> Why? Just to have it autotools-compatible? If I was writing a custom
> >> build system I would be thinking about using -Wp option, since that's
> >> exactly why it's there for — to pass some options to the preprocessor
> >> (or, being honest, I would ignore CPPFLAGS unless I use the preprocessor).
> >
> > That would be wrong as -Wp bypassed the compiler driver.
> 
> Yes, that's the point! If user wanted to pass parameters to the compiler,
> he would used CFLAGS/CXXFLAGS. And since he used CPPFLAGS I assume that he
> wanted to pass them to the preprocessor, not compiler. So when invoking
> gcc build system must prepend CPPFLAGS with -Wp.
 
This does not match historical practice.

> BTW, it's interesting that Fedora/CentOS use -Wp,-D_FORTIFY_SOURCE=2
> and they use it in CFLAGS/CXXFLAGS.

Presumably as a workaround for build systems that do not respect
CPPFLAGS.

[...]
> > No, as per above, and there's no workaround here, just different build
> > system conventions.
> 
> Yes. There're different system conventions. But still dpkg-buildflags
> should do something. Currently it sets: CFLAGS, CXXFLAGS, FFLAGS, LDFLAGS
> and CPPFLAGS. Why it sets just these? Why doesn't it set, for example
> CMAKE_C_FLAGS, or QMAKE_CXXFLAGS instead? It's because those are the most
> popular flags, right? All of them are supported by most build systems.
> All of them except CPPFLAGS.
[...]

GNU make's implicit rules use CPPFLAGS.  If other build systems or
overriden rules don't use it, it's a bug.  This can of course be
worked around in debian/rules.

Ben.

-- 
Ben Hutchings
We get into the habit of living before acquiring the habit of thinking.
  - Albert Camus


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20120625172135.gt2...@decadent.org.uk

Re: Lintian warning: hardening-no-fortify-functions & version numbering

[Serge]

2012/6/24 Guillem Jover wrote:

>> Why? Just to have it autotools-compatible? If I was writing a custom
>> build system I would be thinking about using -Wp option, since that's
>> exactly why it's there for — to pass some options to the preprocessor
>> (or, being honest, I would ignore CPPFLAGS unless I use the preprocessor).
>
> That would be wrong as -Wp bypassed the compiler driver.

Yes, that's the point! If user wanted to pass parameters to the compiler,
he would used CFLAGS/CXXFLAGS. And since he used CPPFLAGS I assume that he
wanted to pass them to the preprocessor, not compiler. So when invoking
gcc build system must prepend CPPFLAGS with -Wp.

BTW, it's interesting that Fedora/CentOS use -Wp,-D_FORTIFY_SOURCE=2
and they use it in CFLAGS/CXXFLAGS.

> No one forces you to pass CPPFLAGS to the build system

Ah, no, I did not mean that. I just mean that not every build system
support CPPFLAGS, and even those who support it, do it in different
ways. So if "dpkg" wants to be easy to use with many build systems,
it should stick to CFLAGS/CXXFLAGS, and avoid using CPPFLAGS if that
is possible.

> you can always do something like:
>   make CFLAGS="$(CPPFLAGS) $(CFLAGS)"

It's not just I "can". As long as dpkg uses CPPFLAGS I must do that.
And the "must" is there only because of dpkg-buildflags. I would not
have to do that if dpkg was not setting CPPFLAGS.

> having to split CPPFLAGS after the fact for build systems that do
> support it would be extremely annoying.

I agree. It may not be a good idea in general. But in our case the
-D_FORTIFY_SOURCE=2 option does not have to be split, so it's not
a problem. After all CPPFLAGS without CFLAGS are only used by
./configure tests, and this option does nothing for them.

> _FORTIFY_SOURCE *is* a preprocesssor macro, you just need to grep for
> it in /user/include/ [...] try including  for example and
> setting an optimization level > 0 in the preprocessor call.

Right, my bad. When I checked that on my machine I did:
$ cpp test.cpp | md5sum
$ cpp -D_FORTIFY_SOURCE=2 test.cpp | md5sum
$ g++ -O2 -c test.cpp && md5sum test.o
$ g++ -O2 -D_FORTIFY_SOURCE=2 -c test.cpp && md5sum test.o
But running `cpp -O2 ...` actually makes the difference. In my defence
I can only say that this option is not documented in `man cpp`.

Looks like "-O2" is a preprocessor option too.

> No, as per above, and there's no workaround here, just different build
> system conventions.

Yes. There're different system conventions. But still dpkg-buildflags
should do something. Currently it sets: CFLAGS, CXXFLAGS, FFLAGS, LDFLAGS
and CPPFLAGS. Why it sets just these? Why doesn't it set, for example
CMAKE_C_FLAGS, or QMAKE_CXXFLAGS instead? It's because those are the most
popular flags, right? All of them are supported by most build systems.
All of them except CPPFLAGS.

Currently autotools packages must write rules in one way, while packages
of other build systems must do it in another. Why? Why dpkg prefers one
build system to the others? It's really trivial to make dpkg compatible
with almost all the build systems around by just dropping CPPFLAGS. This
will make packaging easier. Why not do that?

>> PS: Bugs must be fixed, not documented, IMO.
>
> There's no bug here.

There's a workaround. It means there's a bug being workarounded.
Since it's about dpkg and cmake, the bug is either in dpkg, or cmake
or somewhere between them. Since it happens for other build systems,
I tend to think it's in dpkg.

I'm not telling that nobody must ever use CPPFLAGS. As Russ Allbery wrote
it's necessary to use it if you have std-includes in an unusual location.
What I suggest is to not use CPPFLAGS when it's not needed.

To make it shorter. Dpkg-buildflags must set -D_FORTIFY_SOURCE=2 somewhere.
There're options:
1. Put -D_FORTIFY_SOURCE=2 to CPPFLAGS (autotools)
  advantages: none?
  disadvantages: makes [cq]make packaging a little bit harder
2. Put -D_FORTIFY_SOURCE=2 to CFLAGS/CXXFLAGS
  advantages: makes any packaging a little bit easier
  disadvantages: none?
3. Put _FORTIFY_SOURCE=2 to DEFINES (qmake)
  advantages: none?
  disadvantages: makes autotools packaging a little bit harder
4. Put it to any other variable XXX
  advantages: none?
  disadvantages: makes any packaging a little bit harder

I'm for #2. :)

The "it belongs to CPPFLAGS" thing is neither advantage nor disadvantage,
because it also belongs to CMAKE_C_FLAGS, CMAKE_CXX_FLAGS, QMAKE_CXXFLAGS
or even qmake's DEFINES option, and probably a lot of other variables of
different build systems around. It does not mean that dpkg-buildflags
should set them all.

Also if you consider "it belongs to CPPFLAGS" an important argument,
you should move "-O2" to CPPFLAGS too, because -D_FORTIFY_SOURCE=2
is a preprocessor option only together with -O2, it does nothing
without it (see above). :)

Yes, "it belongs to CPPFLAGS" is a good argument when you're trying to
convince someone, but it's not really an advantage, it does not make
anything better.

PS: thank

Re: Lintian warning: hardening-no-fortify-functions & version numbering

[Bernhard R. Link]

* Russ Allbery  [120624 23:13]:
> When integrating with a build system that uses only one variable for
> compilation flags, just pass the concatenation of CFLAGS and CPPFLAGS into
> it.  This is trivially done in debian/rules without modifying the upstream
> source.

Build systems not doing CPPFLAGS often also are incorrect in other
aspects. Like using CFLAGS where CXXFLAGS are meant. Or having a
differently named LDFLAGS. Or not having LDFLAGS at all and needing them
in a variable called CFLAGS, too. Or only passing LDFLAGS at link time so
needed CFLAGS or CXXFLAGS concatenated into LDFLAGS or or or.
It's all mostly trivially done but there are quite a number of variants.
Anything not having CPPFLAGS is usually simply so non-standard that
a human has to look at it anyway and no automatic approach will help.

Bernhard R. Link


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20120625083304.ga3...@client.brlink.eu

Re: Lintian warning: hardening-no-fortify-functions & version numbering

[Russ Allbery]

"Bernhard R. Link"  writes:
> * Lars Wirzenius  [120624 12:30]:
>> On Sat, Jun 23, 2012 at 03:27:07PM -0700, Russ Allbery wrote:

>>> CFLAGS is an old, long-standing make thing.  CPPFLAGS is newer and I
>>> believe was introduced by Autoconf

>> I don't know the history of CPPFLAGS. It's possible it was introduced by
>> Autoconf. However, it is now embedded into the implicit rules of GNU Make,
>> so every Makefile that doesn't override the rule for compiling a .c file
>> to a .o file uses CPPFLAGS.

> Where "now" means since 1990.

Yeah.  I think CFLAGS may be another 10-15 years older, but I haven't done
the research.  Anyway, this is the way that all Autoconf and Automake
software works, which gets quite a lot of free software, and while I'm not
sure whether any possible preprocessor flags we might want to add in the
future would affect cpp-only tests, I wouldn't want to rule it out.

When integrating with a build system that uses only one variable for
compilation flags, just pass the concatenation of CFLAGS and CPPFLAGS into
it.  This is trivially done in debian/rules without modifying the upstream
source.

-- 
Russ Allbery (r...@debian.org)   


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87txy0l8g5@windlord.stanford.edu

Re: Lintian warning: hardening-no-fortify-functions & version numbering

[Bernhard R. Link]

* Lars Wirzenius  [120624 12:30]:
> On Sat, Jun 23, 2012 at 03:27:07PM -0700, Russ Allbery wrote:
> > CFLAGS is an old, long-standing make thing.  CPPFLAGS is newer and I
> > believe was introduced by Autoconf
> 
> I don't know the history of CPPFLAGS. It's possible it was introduced by
> Autoconf. However, it is now embedded into the implicit rules of GNU Make,
> so every Makefile that doesn't override the rule for compiling a .c file
> to a .o file uses CPPFLAGS.

Where "now" means since 1990.

Bernhard R. Link


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20120624203026.ga8...@client.brlink.eu

Re: Lintian warning: hardening-no-fortify-functions & version numbering

[Lars Wirzenius]

On Sat, Jun 23, 2012 at 03:27:07PM -0700, Russ Allbery wrote:
> CFLAGS is an old, long-standing make thing.  CPPFLAGS is newer and I
> believe was introduced by Autoconf

I don't know the history of CPPFLAGS. It's possible it was introduced by
Autoconf. However, it is now embedded into the implicit rules of GNU Make,
so every Makefile that doesn't override the rule for compiling a .c file
to a .o file uses CPPFLAGS.

-- 
I wrote a book on personal productivity: http://gtdfh.branchable.com/


signature.asc
Description: Digital signature

Re: Lintian warning: hardening-no-fortify-functions & version numbering

[Guillem Jover]

On Sun, 2012-06-24 at 11:07:05 +0300, Serge wrote:
> 2012/6/24 Russ Allbery wrote:
> >> If you do, then... How should they do that? I.e. if I specify:
> >>   CPPFLAGS="blablabla hehehe hohoho"
> >> How should build system run gcc? Like that?
> >>   gcc blablabla hehehe hohoho -c -o test.o test.c
> >
> > Yes.
> 
> Why? Just to have it autotools-compatible? If I was writing a custom
> build system I would be thinking about using -Wp option, since that's
> exactly why it's there for — to pass some options to the preprocessor
> (or, being honest, I would ignore CPPFLAGS unless I use the preprocessor).

That would be wrong as -Wp bypassed the compiler driver. And when
calling gcc you are implicitly using the preprocessor anyway. How you
directly call gcc (or cc for that matter) does not have anything to do
with autotools, the latter only standardizes a way to pass flags to
those calls though, and the compiler is suppoed to be called in the
way Russ pointed out.

> >> Is there a standard common for all the build systems? Or some kind of
> >> recommendation? Or it's just a coincidence that all the build systems
> >> use same environment variable CFLAGS?
> >
> > CFLAGS is an old, long-standing make thing.  CPPFLAGS is newer and I
> > believe was introduced by Autoconf, since Autoconf has historically wanted
> > to run tests that only use the preprocessor and not the compiler (for
> > checking header files, for example), at which point it needed some way of
> > distinguishing between the flags.  GNU software and other software that
> > closely follows the Autoconf and Automake manuals uses a similar
> > separation, and it's important to follow that separation when passing
> > flags to configure to handle tests where just the preprocessor is run.
> > (If, for example, you put the -I flags in CFLAGS and not CPPFLAGS, you may
> > find that Autoconf probes won't find header files that are on non-standard
> > -I paths.)
> 
> You mean, `cpp` is (was?) used only for tests done by ./configure?
> I was not thinking about that. Thank you for pointing that out. I just
> checked a few random autotools-based programs, and indeed they use
> $CPP variable (they actually use `gcc -E`, not `cpp` for it).

The fact that it's called cpp or «gcc -E» should not matter and it's an
implementation detail, both should be performing the same task. If you
need to test for some preprocessed stuff, then there's no point in
compiling anything, instead just preprocessing should be enough.

> > Since the Autoconf/Automake variable naming is the closest we have to a
> > de facto standard (and is *extremely* widespread in free software), the
> > variable names for the dpkg build infrastructure followed the same
> > conventions.

> It depends on whether dpkg is trying to force standards or be compatible
> with them.
> 
> It's bad there's no some common standard or RFC about that. Since CPPFLAGS
> is used by ./configure only, and ./configure script is autotools-specific,
> it makes CPPFLAGS autotools-specific as well, and there's no reason to
> demand from other build systems to use it. :(

> And cmake is not the only one. At least mplayer's Makefile ignores
> CPPFLAGS (`make CPPFLAGS=-Isomedir` will work for autotools, but not
> for mplayer). There may be others...

No one forces you to pass CPPFLAGS to the build system, or for those
build systems to support it, you can always do something like:

  make CFLAGS="$(CPPFLAGS) $(CFLAGS)"

for example, but having to split CPPFLAGS after the fact for build
systems that do support it would be extremely annoying.

> Another point is that the option we're talking about (-D_FORTIFY_SOURCE=2)
> isn't actually a preprocessor option. Preprocessor option is some option,
> that is supposed to affect preprocessor output. But both:
>   cpp test.c
>   cpp -D_FORTIFY_SOURCE=2 test.c
> will produce exactly same output. While these:
>   gcc -c test.c
>   gcc -D_FORTIFY_SOURCE=2 -c test.c
> will give different outputs. So despite it *looks* like a preprocessor
> option (-D...), it's not used by preprocessor, it's a compiler option,
> and belongs to CFLAGS/CXXFLAGS.

_FORTIFY_SOURCE *is* a preprocesssor macro, you just need to grep for
it in /user/include/, and just to make sure I just checked the gcc
sources and there's no relevant reference to that macro, which the
compiler driver could still be checking for as pointed out above (re
-Wp), so there's something else going on in your compiler call (maybe
implicit optimization settings, an alias, etc?). It also obviously
depends on the preprocessed input, try including  for
example and setting an optimization level > 0 in the preprocessor call.

> So the suggestion is:
> if dpkg aims to be compatible with different build systems:
> move -D_FORTIFY_SOURCE=2 from autotools-specific CPPFLAGS to commonly
> used CFLAGS/CXXFLAGS where it belongs (since it's not a preprocessor
> option). This will fix the problem with `cmake` not using preprocessor
> and not checking for CPPFLAGS,

Re: Lintian warning: hardening-no-fortify-functions & version numbering

[Serge]

2012/6/24 Russ Allbery wrote:

>> If you do, then... How should they do that? I.e. if I specify:
>>   CPPFLAGS="blablabla hehehe hohoho"
>> How should build system run gcc? Like that?
>>   gcc blablabla hehehe hohoho -c -o test.o test.c
>
> Yes.

Why? Just to have it autotools-compatible? If I was writing a custom
build system I would be thinking about using -Wp option, since that's
exactly why it's there for — to pass some options to the preprocessor
(or, being honest, I would ignore CPPFLAGS unless I use the preprocessor).

>> Is there a standard common for all the build systems? Or some kind of
>> recommendation? Or it's just a coincidence that all the build systems
>> use same environment variable CFLAGS?
>
> CFLAGS is an old, long-standing make thing.  CPPFLAGS is newer and I
> believe was introduced by Autoconf, since Autoconf has historically wanted
> to run tests that only use the preprocessor and not the compiler (for
> checking header files, for example), at which point it needed some way of
> distinguishing between the flags.  GNU software and other software that
> closely follows the Autoconf and Automake manuals uses a similar
> separation, and it's important to follow that separation when passing
> flags to configure to handle tests where just the preprocessor is run.
> (If, for example, you put the -I flags in CFLAGS and not CPPFLAGS, you may
> find that Autoconf probes won't find header files that are on non-standard
> -I paths.)

You mean, `cpp` is (was?) used only for tests done by ./configure?
I was not thinking about that. Thank you for pointing that out. I just
checked a few random autotools-based programs, and indeed they use
$CPP variable (they actually use `gcc -E`, not `cpp` for it).

> Since the Autoconf/Automake variable naming is the closest we have to a
> de facto standard (and is *extremely* widespread in free software), the
> variable names for the dpkg build infrastructure followed the same
> conventions.

It depends on whether dpkg is trying to force standards or be compatible
with them.

It's bad there's no some common standard or RFC about that. Since CPPFLAGS
is used by ./configure only, and ./configure script is autotools-specific,
it makes CPPFLAGS autotools-specific as well, and there's no reason to
demand from other build systems to use it. :(

And cmake is not the only one. At least mplayer's Makefile ignores
CPPFLAGS (`make CPPFLAGS=-Isomedir` will work for autotools, but not
for mplayer). There may be others...

Another point is that the option we're talking about (-D_FORTIFY_SOURCE=2)
isn't actually a preprocessor option. Preprocessor option is some option,
that is supposed to affect preprocessor output. But both:
  cpp test.c
  cpp -D_FORTIFY_SOURCE=2 test.c
will produce exactly same output. While these:
  gcc -c test.c
  gcc -D_FORTIFY_SOURCE=2 -c test.c
will give different outputs. So despite it *looks* like a preprocessor
option (-D...), it's not used by preprocessor, it's a compiler option,
and belongs to CFLAGS/CXXFLAGS.

So the suggestion is:
if dpkg aims to be compatible with different build systems:
move -D_FORTIFY_SOURCE=2 from autotools-specific CPPFLAGS to commonly
used CFLAGS/CXXFLAGS where it belongs (since it's not a preprocessor
option). This will fix the problem with `cmake` not using preprocessor
and not checking for CPPFLAGS, make dpkg more compatible with existing
build systems, and make packaging easier with no workarounds.

Then we'll be able to remove a "cmake workaround" from the wiki page:
http://wiki.debian.org/Hardening#Notes_for_packages_using_CMake

What do you think? Should this get filled as a bugreport?

PS: Bugs must be fixed, not documented, IMO.

PPS: crossposted to debian-dpkg

-- 
  Serge


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/CAOVenEotnBYauasDUn_Uv_7441kRwhfw3nT=xm7s_gm4bu0...@mail.gmail.com

Re: Lintian warning: hardening-no-fortify-functions & version numbering

[Russ Allbery]

Serge  writes:

> Or you mean that they should run `gcc`/`g++` with CPPFLAGS?

Yes.

> If you do, then... How should they do that? I.e. if I specify:
>   CPPFLAGS="blablabla hehehe hohoho"
> How should build system run gcc? Like that?
>   gcc blablabla hehehe hohoho -c test.o test.c

Yes.

> Is there a standard common for all the build systems? Or some kind of
> recommendation? Or it's just a coincidence that all the build systems
> use same environment variable CFLAGS?

CFLAGS is an old, long-standing make thing.  CPPFLAGS is newer and I
believe was introduced by Autoconf, since Autoconf has historically wanted
to run tests that only use the preprocessor and not the compiler (for
checking header files, for example), at which point it needed some way of
distinguishing between the flags.  GNU software and other software that
closely follows the Autoconf and Automake manuals uses a similar
separation, and it's important to follow that separation when passing
flags to configure to handle tests where just the preprocessor is run.
(If, for example, you put the -I flags in CFLAGS and not CPPFLAGS, you may
find that Autoconf probes won't find header files that are on non-standard
-I paths.)

Since the Autoconf/Automake variable naming is the closest we have to a de
facto standard (and is *extremely* widespread in free software), the
variable names for the dpkg build infrastructure followed the same
conventions.

-- 
Russ Allbery (r...@debian.org)   


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87bok9vf44@windlord.stanford.edu

Re: Lintian warning: hardening-no-fortify-functions & version numbering

[Serge]

2012/6/23 Adam Borowski wrote:

>> Correct me if I'm wrong, but IIRC the CPPFLAGS have nothing to do with C++.
>> They're for `cpp` tool which is "The C PreProcessor" (check `man cpp`).
>> So as far as I understand cmake (and every other build system) MUST ignore
>> the CPPFLAGS, right?
>
> They SHOULD include CPPFLAGS.

You mean that they should run `cpp` even if they don't need it?

Or you mean that they should run `gcc`/`g++` with CPPFLAGS?
If you do, then... How should they do that? I.e. if I specify:
  CPPFLAGS="blablabla hehehe hohoho"
How should build system run gcc? Like that?
  gcc blablabla hehehe hohoho -c test.o test.c
or like that?:
  gcc -Wp,blablabla -Wp,hehehe -Wp,hohoho -c test.o test.c
or like that?:
  gcc -Wp,blablabla,hehehe,hohoho -c test.o test.c
or like that?:
  gcc -Wp,"blablabla hehehe hohoho" -c test.o test.c

It's not that obvious, after all. :)

>> Is there a bug somewhere causing CPPFLAGS to be used by g++? Is that
>> a typo on wiki? Or am I missing something?
>
> Both gcc and g++ preprocess the source first.

Ok, I'll put the question in other way. As _I_ understand the meaning of
CFLAGS, CXXFLAGS and CPPFLAGS it's actually very easy:
* CFLAGS go to gcc (or e.g. clang)
* CXXFLAGS go to g++ (or e.g. clang++)
* CPPFLAGS go to cpp (have no idea what would that be for clang/llvm)
That's all. And that's why I never used CPPFLAGS myself. If I'm wrong
(very probably) can you, please, correct me and point to some
documentation/standard/etc. explaining how CPPFLAGS should be used by
the build system?

Is there a standard common for all the build systems? Or some kind of
recommendation? Or it's just a coincidence that all the build systems
use same environment variable CFLAGS?

-- 
  Serge


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/CAOVenErfO1WK1+md7ANhDbg+KJuwyMxu+0RZEpP=xQaW=qf...@mail.gmail.com

Re: Lintian warning: hardening-no-fortify-functions & version numbering

[Russ Allbery]

Serge  writes:
> 2012/6/23 Andrey Rahmatullin wrote:

>> Do you say that cpp(1) is not used in the build process of C and C++
>> software?

> Yes, unless you actually call `cpp` directly by your build scripts somewhy.
> Gcc uses internal preprocessor by default.

Same thing in different packaging.

Reading the "Present Output Variables" section in the Autoconf manual
about CFLAGS and CPPFLAGS would probably be informative.  If you follow
the Autoconf variable conventions, CPPFLAGS must be passed to every
compiler invocation unless you're specifically telling the compiler *not*
to preprocess the input source (highly unusual).

-- 
Russ Allbery (r...@debian.org)   


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87lijdvgr0@windlord.stanford.edu

Re: Lintian warning: hardening-no-fortify-functions & version numbering

[Serge]

2012/6/23 Andrey Rahmatullin wrote:

>> They're for `cpp` tool which is "The C PreProcessor" (check `man cpp`).
>> So as far as I understand cmake (and every other build system) MUST
>> ignore the CPPFLAGS, right?
>
> Do you say that cpp(1) is not used in the build process of C and C++
> software?

Yes, unless you actually call `cpp` directly by your build scripts somewhy.
Gcc uses internal preprocessor by default.
(I just checked `strace -f g++ test.cpp` to make sure)

-- 
  Serge


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/caovenerrwibpitsqes1hi2qa0akc+mdaqyetr8edrhnohmu...@mail.gmail.com

Re: Lintian warning: hardening-no-fortify-functions & version numbering

[Adam Borowski]

On Sat, Jun 23, 2012 at 11:08:50PM +0300, Serge wrote:
> 2012/6/19 José Luis Segura Lucas wrote:
> > (http://wiki.debian.org/Hardening#Notes_for_packages_using_CMake),
> > referred by lintian-info too. Using it I only need to define "export
> > DEB_BUILD_HARDENING=1" on my debian/rules and it adds the CPPFLAGS to
> > CFLAGS and CXXFLAGS (Cmake ignores CPPFLAGS).
> 
> Correct me if I'm wrong, but IIRC the CPPFLAGS have nothing to do with C++.
> They're for `cpp` tool which is "The C PreProcessor" (check `man cpp`).
> So as far as I understand cmake (and every other build system) MUST ignore
> the CPPFLAGS, right?

They SHOULD include CPPFLAGS.

> The CFLAGS should be used by `gcc` and CXXFLAGS should to go to `g++`.
> 
> Is there a bug somewhere causing CPPFLAGS to be used by g++? Is that a typo
> on wiki? Or am I missing something?

Both gcc and g++ preprocess the source first.

-- 
I was born an ugly, dumb and work-loving child, then an evil midwife
replaced me in the crib.


signature.asc
Description: Digital signature

Re: Lintian warning: hardening-no-fortify-functions & version numbering

[Andrey Rahmatullin]

On Sat, Jun 23, 2012 at 11:08:50PM +0300, Serge wrote:
> > (http://wiki.debian.org/Hardening#Notes_for_packages_using_CMake),
> > referred by lintian-info too. Using it I only need to define "export
> > DEB_BUILD_HARDENING=1" on my debian/rules and it adds the CPPFLAGS to
> > CFLAGS and CXXFLAGS (Cmake ignores CPPFLAGS).
> 
> Correct me if I'm wrong, but IIRC the CPPFLAGS have nothing to do with C++.
Nobody said they do.

> They're for `cpp` tool which is "The C PreProcessor" (check `man cpp`).
> So as far as I understand cmake (and every other build system) MUST ignore
> the CPPFLAGS, right?
Do you say that cpp(1) is not used in the build process of C and C++
software?

-- 
WBR, wRAR


signature.asc
Description: Digital signature

Re: Lintian warning: hardening-no-fortify-functions & version numbering

[Serge]

2012/6/19 José Luis Segura Lucas wrote:

> (http://wiki.debian.org/Hardening#Notes_for_packages_using_CMake),
> referred by lintian-info too. Using it I only need to define "export
> DEB_BUILD_HARDENING=1" on my debian/rules and it adds the CPPFLAGS to
> CFLAGS and CXXFLAGS (Cmake ignores CPPFLAGS).

Correct me if I'm wrong, but IIRC the CPPFLAGS have nothing to do with C++.
They're for `cpp` tool which is "The C PreProcessor" (check `man cpp`).
So as far as I understand cmake (and every other build system) MUST ignore
the CPPFLAGS, right?

The CFLAGS should be used by `gcc` and CXXFLAGS should to go to `g++`.

Is there a bug somewhere causing CPPFLAGS to be used by g++? Is that a typo
on wiki? Or am I missing something?

-- 
  Serge


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/caoveneodggx0lqly5p9kcjp5ywmqf+x6ubuyy1yf_eqyjdq...@mail.gmail.com

Re: Lintian warning: hardening-no-fortify-functions & version numbering

[Antti-Juhani Kaijanaho]

On Tue, Jun 19, 2012 at 04:04:31PM +0200, José Luis Segura Lucas wrote:
> repository but not still in a numbered version, so, I tried to use the
> latest known version and add a ~TIMESTAMPgit... to the minor version
> number, but debuild warns me about the version 0.1.0~2012..git-1 is
> less than 0.1.0.

Use A~B only if this version should come before A - that is, for example, if
you expect the next upstream release to be A.  In your case, use A+B so the
version comes after A in version order.

> The latest thing is that I have seen several packages with ~TIMESTAMP
> (screen, by example): they add a alpha-numeric string after the "git"
> word... what does it mean?

It's the beginning of the hash git uses to uniquely identify the version.
You can see the full hash in, for example, git log, at the beginning of each
patch:

commit 2ff04fd5a95f36497bc8e8c6e44d70c474384ec2
Author: Antti-Juhani Kaijanaho 
Date:   Mon Jun 18 23:26:59 2012 +0300

Add install-sh

Signed-off-by: Antti-Juhani Kaijanaho 

In this case, the hash is the long string after the word "commit"
on the first line, and I would use for example 2ff04 in the version number if I
packaged this version as a git snapshot.

> Where can I found some information about
> packaging directly from VCS?

git-buildpackage has excellent docs.  You don't have to use the tools but they
help.

-- 
Antti-Juhani Kaijanaho, Jyväskylä, Finland
http://antti-juhani.kaijanaho.fi/newblog/
http://www.flickr.com/photos/antti-juhani/



signature.asc
Description: Digital signature

Re: Lintian warning: hardening-no-fortify-functions & version numbering

[Andrey Rahmatullin]

On Tue, Jun 19, 2012 at 05:04:46PM +0200, José Luis Segura Lucas wrote:
> > I see several solutions there, and the hardening-wrapper one is in my
> > opinion the worst one: it adds a build dependency and it uses own set of
> > configuration variables, not compatible with dpkg-buildflags ones.
> Yes, it adds a build-dependency... but it still use dpkg-buildflags,
> doesn't it?
It uses dpkg-buildflags in addition to doing more or less the same in a
different way.

-- 
WBR, wRAR


signature.asc
Description: Digital signature

Re: Lintian warning: hardening-no-fortify-functions & version numbering

[José Luis Segura Lucas]

El 19/06/12 16:56, Andrey Rahmatullin escribió:
> I see several solutions there, and the hardening-wrapper one is in my
> opinion the worst one: it adds a build dependency and it uses own set of
> configuration variables, not compatible with dpkg-buildflags ones.
Yes, it adds a build-dependency... but it still use dpkg-buildflags,
doesn't it?
> Then I don't understand what do you mean by "packaging directly from VCS".
I mean to package the software directly from a VCS version, instead for
a tarball. Now, I'm getting the sources with git, generate a tarball
myself with the appropriate name, put it in the parent directory and
build the package. I think that it can be a way to avoid the manual
tarball generation. I'm reading about git-buildpackage, because it can help.
>
> I thought you've meant exactly the -27-g55c0f4e part.
>
The problem are the "-", but I hope that when uploading to Debian they
have a freeze stable version :-D

-- 
José Luis Segura Lucas




signature.asc
Description: OpenPGP digital signature

Re: Lintian warning: hardening-no-fortify-functions & version numbering

[Andrey Rahmatullin]

On Tue, Jun 19, 2012 at 04:42:33PM +0200, José Luis Segura Lucas wrote:
> > Why do you need hardening-wrapper? You should use flags set by
> > dpkg-buildflags.
> Because that
> (http://wiki.debian.org/Hardening#Notes_for_packages_using_CMake),
> referred by lintian-info too. Using it I only need to define "export
> DEB_BUILD_HARDENING=1" on my debian/rules and it adds the CPPFLAGS to
> CFLAGS and CXXFLAGS (Cmake ignores CPPFLAGS).
I see several solutions there, and the hardening-wrapper one is in my
opinion the worst one: it adds a build dependency and it uses own set of
configuration variables, not compatible with dpkg-buildflags ones.

> > file:///usr/share/doc/git-buildpackage/manual-html/gbp.import.html#GBP.IMPORT.UPSTREAM-GIT
> > (if you use git-buildpackage)
> I don't use git-buildpackage 
Then I don't understand what do you mean by "packaging directly from VCS".

> and git describe on their repository
> returns a bit estrange string...
> 
> $ git describe --tags
> v0.1.1-27-g55c0f4e
I thought you've meant exactly the -27-g55c0f4e part.

-- 
WBR, wRAR


signature.asc
Description: Digital signature

Re: Lintian warning: hardening-no-fortify-functions & version numbering

[José Luis Segura Lucas]

El 19/06/12 16:10, Andrey Rahmatullin escribió:
> Why do you need hardening-wrapper? You should use flags set by
> dpkg-buildflags.
Because that
(http://wiki.debian.org/Hardening#Notes_for_packages_using_CMake),
referred by lintian-info too. Using it I only need to define "export
DEB_BUILD_HARDENING=1" on my debian/rules and it adds the CPPFLAGS to
CFLAGS and CXXFLAGS (Cmake ignores CPPFLAGS).
> You should read http://bugs.debian.org/673112 mentioned in the lintian tag
> description and use hardening-check --verbose on binaries reported. If
> only memcpy and memmove are printed by hardening-check, you should ignore
> the warning.
Done. I have emmove, memcpy and read... I will read this link to learn
more about removing this warning.
> That's right, 0.1.0~2012..git-1 is less than 0.1.0. If you need
> versions that are greater than 0.1.0, use + instead of ~.
Ok, thanks
> git-describe(1)
>
> file:///usr/share/doc/git-buildpackage/manual-html/gbp.import.html#GBP.IMPORT.UPSTREAM-GIT
> (if you use git-buildpackage)
I don't use git-buildpackage and git describe on their repository
returns a bit estrange string...

$ git describe --tags
v0.1.1-27-g55c0f4e

Thanks for your quick help :-)

-- 
José Luis Segura Lucas




signature.asc
Description: OpenPGP digital signature

Re: Lintian warning: hardening-no-fortify-functions & version numbering

[Andrey Rahmatullin]

On Tue, Jun 19, 2012 at 04:04:31PM +0200, José Luis Segura Lucas wrote:
> I have read the output of lintian-info -t about
> hardening-no-fortify-functions, and it helps a lot. The software uses
> Cmake as build tool, and the "hardening-wrapper" solution solved some
> lintian warnings, but not the latest one.
Why do you need hardening-wrapper? You should use flags set by
dpkg-buildflags.

> I have looked at the buld logs, and I can see that the CPPFLAGS
> "-D_FORTIFY_SOURCE=2" is included in all the compiler calls, but the
> warning is still present.
> 
> What's the problem with this?
You should read http://bugs.debian.org/673112 mentioned in the lintian tag
description and use hardening-check --verbose on binaries reported. If
only memcpy and memmove are printed by hardening-check, you should ignore
the warning.

> My another question is about the version numbering: the software is
> still in development and they make a new minor version each week
> (approximately). Sometimes I need to package something that is in their
> repository but not still in a numbered version, so, I tried to use the
> latest known version and add a ~TIMESTAMPgit... to the minor version
> number, but debuild warns me about the version 0.1.0~2012..git-1 is
> less than 0.1.0.
That's right, 0.1.0~2012..git-1 is less than 0.1.0. If you need
versions that are greater than 0.1.0, use + instead of ~.

> The latest thing is that I have seen several packages with ~TIMESTAMP
> (screen, by example): they add a alpha-numeric string after the "git"
> word... what does it mean?
git-describe(1)

> Where can I found some information about packaging directly from VCS?
file:///usr/share/doc/git-buildpackage/manual-html/gbp.import.html#GBP.IMPORT.UPSTREAM-GIT
(if you use git-buildpackage)

-- 
WBR, wRAR


signature.asc
Description: Digital signature

Lintian warning: hardening-no-fortify-functions & version numbering

[José Luis Segura Lucas]

Hi!

I'm intending to package a new software for Debian [1]. I just completed
most of the package work and have a lintian-error free package, but I
still have a warning that is driving me crazy.

I have read the output of lintian-info -t about
hardening-no-fortify-functions, and it helps a lot. The software uses
Cmake as build tool, and the "hardening-wrapper" solution solved some
lintian warnings, but not the latest one.

I have looked at the buld logs, and I can see that the CPPFLAGS
"-D_FORTIFY_SOURCE=2" is included in all the compiler calls, but the
warning is still present.

What's the problem with this?

My another question is about the version numbering: the software is
still in development and they make a new minor version each week
(approximately). Sometimes I need to package something that is in their
repository but not still in a numbered version, so, I tried to use the
latest known version and add a ~TIMESTAMPgit... to the minor version
number, but debuild warns me about the version 0.1.0~2012..git-1 is
less than 0.1.0.

The latest thing is that I have seen several packages with ~TIMESTAMP
(screen, by example): they add a alpha-numeric string after the "git"
word... what does it mean? Where can I found some information about
packaging directly from VCS?

Best regard and thanks in advance

-- 
José Luis Segura Lucas




signature.asc
Description: OpenPGP digital signature