Re: RFC: More C errors by default in GCC 14 (no more implicit function declarations etc.)

[Florian Weimer]

* G. Branden Robinson:

> Perhaps the thing to do here is have, , yet another command-line
> option for GCC.  The Ada language did something similar a couple of
> decades ago to tighten up the language for hard real-time demands, with
> what it called the "Ravenscar profile".[1]  That proved successful (as
> successful as anything was in poor neglected Ada).

The C++ front end calls this -fpermissive, which would probably match
here as well.

> Whatever its name, some advantages to this approach are that
> distributors could opt-in to such a thing, make it a clear matter of
> policy, and more easily track adoption and progress.  You could also
> version the contour much like the C standard itself.

I'm not sure if we need this fine level of control.  Either you want to
compile at all costs, or you are willing to make the effort to clean up
the sources.  At a package level, the required changes are almost always
minor (but of course there are packages that are different), so fixing
everything in one go (and the implied commitment to keep up with future
cleanups_ is not very onerous.  For one package, that is.

I'm not sure if opt-in is that easy because packages drop build flags
all the time.  Compiler wrapper scripts might be required.  The
instrumented compiler we use in Fedora sometimes reveals things that go
unnoticed with other approaches.

Thanks,
Florian

Re: RFC: More C errors by default in GCC 14 (no more implicit function declarations etc.)

[Sam James]

Oskari Pirhonen  writes:

> [[PGP Signed Part:Undecided]]
> On Tue, Apr 18, 2023 at 16:07:45 +0200, Florian Weimer wrote:
>> Gentoo has been fixing various packages for building with Clang, which
>> covers a superset of the issues that need to be addressed:
>> 
>>   [TRACKER] Support LLVM/Clang as alternative system compiler 
>>   
>> 
>> IIRC, Gentoo has its own mechanism to detect silent build breakage, but
>> I think it's mostly focused on autoconf, so it's less comprehensive, and
>> also fixes the stuff that is actually relevant to the distribution.
>> 
>
> For Gentoo, I wrote (with some help from others) this QA check [1] which
> Portage uses to scan Autoconf, CMake, and Meson config logs for implicit
> function declarations. It's inspired by a similar bit of code from the
> Macports folks [2] and written with both Clang and GCC in mind.
>
> It should be possible to adapt for use by others if you feed it the
> right dirs and replace a few functions (`has` and `eqa*` OTTOMH) since
> nothing about the core logic is Portage-specific.
>
> Although not so much for silent failures, but maybe still useful for
> someone, there's also this QA check [3] which is used to detect other
> warnings at build-time.

Note for others: this is distinct from the initial approach we took (and
are still using) with compiler wrapping and diffing old/new config.log
which Florian was referring to.

>
> - Oskari
>

best,
sam



signature.asc
Description: PGP signature

Re: RFC: More C errors by default in GCC 14 (no more implicit function declarations etc.)

[Oskari Pirhonen]

On Tue, Apr 18, 2023 at 16:07:45 +0200, Florian Weimer wrote:
> Gentoo has been fixing various packages for building with Clang, which
> covers a superset of the issues that need to be addressed:
> 
>   [TRACKER] Support LLVM/Clang as alternative system compiler 
>   
> 
> IIRC, Gentoo has its own mechanism to detect silent build breakage, but
> I think it's mostly focused on autoconf, so it's less comprehensive, and
> also fixes the stuff that is actually relevant to the distribution.
> 

For Gentoo, I wrote (with some help from others) this QA check [1] which
Portage uses to scan Autoconf, CMake, and Meson config logs for implicit
function declarations. It's inspired by a similar bit of code from the
Macports folks [2] and written with both Clang and GCC in mind.

It should be possible to adapt for use by others if you feed it the
right dirs and replace a few functions (`has` and `eqa*` OTTOMH) since
nothing about the core logic is Portage-specific.

Although not so much for silent failures, but maybe still useful for
someone, there's also this QA check [3] which is used to detect other
warnings at build-time.

- Oskari

[1]: 
https://gitweb.gentoo.org/proj/portage.git/tree/bin/install-qa-check.d/90config-impl-decl
[2]: 
https://github.com/macports/macports-base/blob/15270571ab178ddff693fbd722163fe1d638ed55/src/port1.0/portconfigure.tcl#L1853
[3]: 
https://gitweb.gentoo.org/proj/portage.git/tree/bin/install-qa-check.d/90gcc-warnings


signature.asc
Description: PGP signature

Re: RFC: More C errors by default in GCC 14 (no more implicit function declarations etc.)

[Arsen Arsenović]

Paul Wise  writes:

> On Tue, 2023-04-18 at 16:07 +0200, Florian Weimer wrote:
>
>> TL;DR: I want to propose a GCC 14 change which will impact
>> distributions, so I'd like to gather some feedback from Debian.
>
> Is this change being made the upstream defaults?
>
> Or will it be a distro override like the hardening flags are?

The intention is to make this a default, in order to brace people for a
future C default standard version changes.

>> I would appreciate some discussion on the Debian impact.
>
> Since most of the Debian archive can be reproducibly built, it seems
> like the way to gauge the impact of this change on Debian would be to
> do two archive rebuilds, once without the flags and once with the
> flags, then compare the two builds for each package using diffoscope.

This seems reasonable.  I suspect changes largely in configure scripts
based on our prior experience[1].

Have a lovely night.

> The reproducible builds fuzzing tests get 96.2% reproducible,
> this would only go up when not varying most of the fuzzed things.
>
> https://tests.reproducible-builds.org/debian/reproducible.html
>
> The existing documentation for Debian archive rebuilds is outdated and
> deleted, but Lucas Nusbaum and others have been doing them for a while.
>
> https://wiki.debian.org/MassRebuilds
> https://wiki.debian.org/qa.debian.org/ArchiveTesting?action=recall=23

[1] https://bugs.gentoo.org/870412
-- 
Arsen Arsenović


signature.asc
Description: PGP signature

Re: RFC: More C errors by default in GCC 14 (no more implicit function declarations etc.)

[Paul Wise]

On Tue, 2023-04-18 at 16:07 +0200, Florian Weimer wrote:

> TL;DR: I want to propose a GCC 14 change which will impact
> distributions, so I'd like to gather some feedback from Debian.

Is this change being made the upstream defaults?

Or will it be a distro override like the hardening flags are?

> I would appreciate some discussion on the Debian impact.

Since most of the Debian archive can be reproducibly built, it seems
like the way to gauge the impact of this change on Debian would be to
do two archive rebuilds, once without the flags and once with the
flags, then compare the two builds for each package using diffoscope.

The reproducible builds fuzzing tests get 96.2% reproducible,
this would only go up when not varying most of the fuzzed things.

https://tests.reproducible-builds.org/debian/reproducible.html

The existing documentation for Debian archive rebuilds is outdated and
deleted, but Lucas Nusbaum and others have been doing them for a while.

https://wiki.debian.org/MassRebuilds
https://wiki.debian.org/qa.debian.org/ArchiveTesting?action=recall=23

-- 
bye,
pabs

https://wiki.debian.org/PaulWise


signature.asc
Description: This is a digitally signed message part

Re: RFC: More C errors by default in GCC 14 (no more implicit function declarations etc.)

[Guillem Jover]

Hi!

On Tue, 2023-04-18 at 17:54:20 -0500, G. Branden Robinson wrote:
> At 2023-04-18T16:07:45+0200, Florian Weimer wrote:
> > TL;DR: I want to propose a GCC 14 change which will impact
> > distributions, so I'd like to gather some feedback from Debian.

> > I would appreciate some discussion on the Debian impact.  As Debian
> > generally doesn't do mass rebuilds and we have upstreamed the fixes,
> > maybe the impact is somewhat reduced?  Given that you'll get the fixes
> > as part of the rebases.  Of course, other mass-changes might trigger
> > rebuilds at a larger scale.  I guess it also depends on when you want
> > to update to GCC 14.  The later, the more likely other packages have
> > already imported the upstream fixes.  The alternative would be to
> > apply the fixes in a proactive manner, like we are trying to in
> > Fedora.
> 
> Debian has little fear of sweeping transitions, but it does like them to
> be precisely defined in scope, and easily measured in adoption.  The
> addition of a versioned contour flag to "DEB_BUILD_OPTIONS" or
> similar[3] is straightforward to measure by scanning source packages.

At the time I got aware of this effort, I implemented a dpkg-buildflags
feature (qa=+c99) to enable most of these flags (see [F]), but didn't
mention it because it was not clear whether this was a desired
direction and the current implementation seemed slightly restrictive,
also due to the freeze.

But perhaps switching more gradually does indeed work better, at least
in the Debian context. I think the feature name is not great, because it
is not very descriptive and precisely because once used it could not be
currently extended, and that ties with work I've been doing to version
feature flags (but I guess this one could always be introduced and be
considered v0 or similar, probably with a better name(?)).

Thanks,
Guillem

[F] 
https://git.hadrons.org/git/debian/dpkg/dpkg.git/commit/?h=next/modern-c=3316845bf415436299d61501db655fd2c1813436

Re: RFC: More C errors by default in GCC 14 (no more implicit function declarations etc.)

[G. Branden Robinson]

[I am not subscribed to debian-gcc or c-std-porting]

Hi Florian,

At 2023-04-18T16:07:45+0200, Florian Weimer wrote:
> TL;DR: I want to propose a GCC 14 change which will impact
> distributions, so I'd like to gather some feedback from Debian.
> 
> Clang has disabled support for a few historic C features by default
> over the last few releases.  This mirrors a process that Apple has
> begun in Xcode even earlier (perhaps motivated in part by their
> AArch64 Darwin ABI, which is pretty much incompatible with some of the
> C89-only features).
> 
> These changes bring real benefits to C programmers because errors are
> much harder to miss during the build than warnings.  In many cases,
> the compiler is not able to generate correct code when such issues are
> present, and programmers who look at the generated machine code
> suspect a compiler bug.  And all this happens because they missed a
> warning.  So we want this change for GCC, too.
[...]
> Specifically, I'm investigating the following changes:
> 
> * -Werror=implicit-function-declaration: Functions can no longer be
>called without be declaring first.  Fixing this may need additional
>prototypes in package header files, or inclusion of additional
>header files (both package-specific and system headers).
> 
> * -Werror=implict-int: int types can no longer be omitted in old-style
>function definitions, function return types, or variable
>declarations or definitions.  Fixing that involves adding the int
>type (or the correct type if it is not actually int).  If there is
>already a matching declaration in scope that has a different type,
>that needs to be resolved somehow, too.
[reordered]
> I want to propose at least the first two for GCC 14.
[reordered]
> The exact mechanism how the default is changed may not be -Werror=,
> perhaps something along the lines of -fpermissive for C++.  The C89
> modes (-std=gnu89 etc.) will likely still enable all these features
> (even if they are not part of C89).  As an opt-out mechanism,
> -std=gnu89 is insufficient because there are packages out there which
> use features which are C99-and-later-only in GCC (predominantly
> C99-style inlining, I believe) together with
> implicit-int/implicit-function-declaration.

Perhaps the thing to do here is have, , yet another command-line
option for GCC.  The Ada language did something similar a couple of
decades ago to tighten up the language for hard real-time demands, with
what it called the "Ravenscar profile".[1]  That proved successful (as
successful as anything was in poor neglected Ada).

The term "profile" is perhaps unfortunate since it already has another
meaning in software performance measurement.  Maybe we could use the
term "contour", which isn't any longer, unlike "silhouette".  A
"contour" furthermore suggests boundaries, which would seem to be
apropos for what you're trying to do in two respects: (a) "ruling out"
certain coding practices that are accepted as standard, or at least
tolerated, otherwise; and (b) reducing the amount of undefined behavior
compiled code exhibits.

Whatever its name, some advantages to this approach are that
distributors could opt-in to such a thing, make it a clear matter of
policy, and more easily track adoption and progress.  You could also
version the contour much like the C standard itself.  So, for instance,
you introduce a contour called "clave23"[2] with the above features...

> * (tentative) -Werror=int-conversion: Conversion between pointer and
>   integer types without an explicit cast is now a compiler error.
>   Usually fixed by one of the two things above.  I do not have data
>   yet how many other cases remain after the initial issues are fixed,
>   but should have that in the coming weeks.  (Quite frankly, I'm
>   amazed that we created 64-bit ports without making this an error.)

So was I, 20 years ago when working on IA-64 ports...

> * (very tentative) -Werror=incompatible-pointer-types: GCC will no
>   longer automatically convert between pointer values of unrelated
>   pointer types (except when one of them is void * or its qualified
>   versions).  The fallout from this could be quite large, I do not
>   have data yet.  Clang does this for function pointer types only
>   (probably based on their ABI issues), but I'm not sure if it's a
>   reasonable distinction for our ABIs (the ppc64le cases I've seen had
>   explicit casts and no warnings).

And then perhaps adopt the above two for "clave26" in 2026 or so.

> * For -Wdiscarded-qualifies (e.g., using const pointers as non-const),
>   and -Wpointe-rsign (using char * as unsigned char *) I do not have
>   any plans.

With a versioned "profile" or "contour", you need not eat the elephant
all at once, and maintain ambitions that seem hopelessly distant today.

> I would appreciate some discussion on the Debian impact.  As Debian
> generally doesn't do mass rebuilds and we have upstreamed the fixes,
> maybe the impact is somewhat 

RFC: More C errors by default in GCC 14 (no more implicit function declarations etc.)

[Florian Weimer]

TL;DR: I want to propose a GCC 14 change which will impact
distributions, so I'd like to gather some feedback from Debian.

Clang has disabled support for a few historic C features by default over
the last few releases.  This mirrors a process that Apple has begun in
Xcode even earlier (perhaps motivated in part by their AArch64 Darwin
ABI, which is pretty much incompatible with some of the C89-only
features).

These changes bring real benefits to C programmers because errors are
much harder to miss during the build than warnings.  In many cases, the
compiler is not able to generate correct code when such issues are
present, and programmers who look at the generated machine code suspect
a compiler bug.  And all this happens because they missed a warning.  So
we want this change for GCC, too.

On the other hand, many distributions use GCC as the system compiler,
and there the focus is not so much on developing software, but building
the sources as they exist today.  It's somewhat different the usual GCC
C++ updates (both language changes and libstdc++ header changes) because
it impacts pre-build feature probing (mostly autoconf).  If that happens
and the probe goes wrong due to a new compiler error, it's possible that
a build still succeeds, passes its test suite, but lacks the intended
feature or ABI because parts got automatically disabled due to the
failing configure check.  With C++ transitions, that seems more rare
(C++ programs—if they use autoconf—often run the checks with the C
compiler).

Specifically, I'm investigating the following changes:

* -Werror=implicit-function-declaration: Functions can no longer be
   called without be declaring first.  Fixing this may need additional
   prototypes in package header files, or inclusion of additional header
   files (both package-specific and system headers).

* -Werror=implict-int: int types can no longer be omitted in old-style
   function definitions, function return types, or variable declarations
   or definitions.  Fixing that involves adding the int type (or the
   correct type if it is not actually int).  If there is already a
   matching declaration in scope that has a different type, that needs
   to be resolved somehow, too.

* (tentative) -Werror=int-conversion: Conversion between pointer and
  integer types without an explicit cast is now a compiler error.
  Usually fixed by one of the two things above.  I do not have data yet
  how many other cases remain after the initial issues are fixed, but
  should have that in the coming weeks.  (Quite frankly, I'm amazed that
  we created 64-bit ports without making this an error.)

* (very tentative) -Werror=incompatible-pointer-types: GCC will no
  longer automatically convert between pointer values of unrelated
  pointer types (except when one of them is void * or its qualified
  versions).  The fallout from this could be quite large, I do not have
  data yet.  Clang does this for function pointer types only (probably
  based on their ABI issues), but I'm not sure if it's a reasonable
  distinction for our ABIs (the ppc64le cases I've seen had explicit
  casts and no warnings).

* For -Wdiscarded-qualifies (e.g., using const pointers as non-const),
  and -Wpointe-rsign (using char * as unsigned char *) I do not have any
  plans.

I want to propose at least the first two for GCC 14.

The exact mechanism how the default is changed may not be -Werror=,
perhaps something along the lines of -fpermissive for C++.  The C89
modes (-std=gnu89 etc.) will likely still enable all these features
(even if they are not part of C89).  As an opt-out mechanism, -std=gnu89
is insufficient because there are packages out there which use features
which are C99-and-later-only in GCC (predominantly C99-style inlining, I
believe) together with implicit-int/implicit-function-declaration.

For Fedora, we are using an instrumented compiler to find packages that
need fixing.  More details on the process are here (but please ask if
you are interested in specifics):

  
  

The numbers so far don't look great, but are manageable.  Fedora has
23,101 source package last time a looked.  We have fixed 796 of them,
and 85 are still pending investigation (with 5-10 false positives
expected remaining).  This puts the per-package failure rate at 3.8%.  I
don't have data on silent failures; most issues do not seem to be
silent, and fully-silent packages are rare.  The silent output-changing
issues definitely exist, they are usually accompanied by something else.
Those 3.8% also include some packages which we fixed by removing C89
constructs, but where the relevant autoconf tests failed for other
reasons.

Fedora would be hit pretty hard if we made the GCC switch without this
preparation because we do a mass rebuild of the entire distribution
right after importing a new GCC upstream release.  I have considered
automating