Re: Bug#1040062: dpkg-dev: Please drop pie-{compile,link}.spec
On Fri, Nov 24, 2023 at 01:53:04AM +0100, Guillem Jover wrote: > Hi! Hi Guillem! Apologies for not replying to these emails earlier. > On Tue, 2023-10-31 at 10:52:40 +0100, Guillem Jover wrote: >... > > If PIE (via specs files) appears to work on x32, and changing the > > defaults in gcc is too much to bother, I think leaving it as is looks > > fine by me. But if this is causing problems as well and the x32 porters > > (if there's any remaining :), want to mask it alongside the other ports, > > let me know and I can also flip the switch for that one. > > If the porters would also like to see x32 masked, let me know and I > can also include it, otherwise I'll leave it as it is now. AFAIK for all ports architectures except Hurd and hppa the other Adrian is a porter, so should be able to ACK it. As I already wrote in [1] there is architecture other than alpha/ia64/x32 where the pie specs did actually enable PIE. On m68k and sh4 the specs might still be passed and cause FTBFS in packages like gluegen2 that pass LDFLAGS to ld, but there is no PIE enabling effect. > Thanks, > Guillem cu Adrian [1] https://lists.debian.org/debian-alpha/2023/10/msg2.html
Re: Bug#1040062: dpkg-dev: Please drop pie-{compile,link}.spec
Hi! On Tue, 2023-10-31 at 10:52:40 +0100, Guillem Jover wrote: > I guess I could do it the other way, and given this is apparently > causing issues as reported by Adrian, and as seen recently from > the referenced bug report which might require patching a specific > package to disable PIE there, I'm inclined to completely mask PIE > in dpkg-buildflags for alpha and ia64 in around a couple of weeks, > if I don't hear anything. I've now queued locally a commit masking this for alpha and ia64 and will be including it in dpkg 1.22.2, probably by the end of the weekend. To the porters, please let me know if you'd prefer these not to be masked. > If PIE (via specs files) appears to work on x32, and changing the > defaults in gcc is too much to bother, I think leaving it as is looks > fine by me. But if this is causing problems as well and the x32 porters > (if there's any remaining :), want to mask it alongside the other ports, > let me know and I can also flip the switch for that one. If the porters would also like to see x32 masked, let me know and I can also include it, otherwise I'll leave it as it is now. Thanks, Guillem
Re: Bug#1040062: dpkg-dev: Please drop pie-{compile,link}.spec
Hi!
On Tue, 2023-07-04 at 13:12:48 +0300, Adrian Bunk wrote:
> On Tue, Jul 04, 2023 at 09:23:43AM +0200, Guillem Jover wrote:
> > On Sun, 2023-07-02 at 00:02:46 +0300, Adrian Bunk wrote:
> > > There are some problems with this:
> > >
> > > 1. PIE should either be default or not be used
> > >
> > > I suspect x32 might be able to default to PIE without problems
> > > (there might just not be enough interest left to change the default).
> > >
> > > On alpha the toolchain has already become quite brittle
> > > with frequent issues like (reproducible) linker segfaults,
> > > any variations that affects the toolchain are bad.
> > >
> > > It is for the port maintainers to decide whether or not PIE
> > > is considered stable on a port, and accordingly either make
> > > it default (which also avoids the other issues below) or not.
> > >
> > > It is clear that a non-PIE architecture would no longer be
> > > considered suitable as release architecture.
> >
> > In general the way this is handled in dpkg, is that if the flags
> > supposedly work on that arch they are allowed, but if they are not
> > supported or are broken then they are masked.
I recalled this report before the 1.22.1 release, but didn't feel it
appropriate to push a last minute change for it, before giving some
advance notice, and because I was also expecting some word from the
relevant arch porters.
I guess I could do it the other way, and given this is apparently
causing issues as reported by Adrian, and as seen recently from
the referenced bug report which might require patching a specific
package to disable PIE there, I'm inclined to completely mask PIE
in dpkg-buildflags for alpha and ia64 in around a couple of weeks,
if I don't hear anything.
> > > Please drop pie-{compile,link}.spec, on the architectures
> > > where it has any effect it is doing more harm than good.
> >
> > For example hppa has pie masked for build flags. If the porters for
> > alpha and/or ia64 consider that they should also get pie masked for
> > those arches, I'm fine doing the changes. Although that means on those
> > ports it will not be possible to enable pie at all, even if asked for
> > explicitly, as in «hardening=+pie».
>
> Semi-PIE non-release architectures shouldn't exist, PIE on an·
> architecture should be a binary option decided by the porters
> for this architecture.
>
> If there are any porters left on an architecture and they consider PIE·
> stable, they can always ask the gcc maintainer to change the default.[1]
>
> >...
> > > The lowest effort fix would be to patch debian/rules of affected
> > > packages to disable hardening=+pie on affected architectures,
> > > but that would still be spending time on working around a problem
> > > that shouldn't exist.
> >
> > Yeah, that does not look like the right thing to be spending time on.
> >...
>
> As long as we have at least one semi-PIE architecture, this is the only·
> realistic option for porters. If the porters for alpha and/or ia64 want·
> to have pie masked, such changes will still be required for x32.
If PIE (via specs files) appears to work on x32, and changing the
defaults in gcc is too much to bother, I think leaving it as is looks
fine by me. But if this is causing problems as well and the x32 porters
(if there's any remaining :), want to mask it alongside the other ports,
let me know and I can also flip the switch for that one.
Thanks,
Guillem
Re: Bug#1040062: dpkg-dev: Please drop pie-{compile,link}.spec
On Tue, Jul 04, 2023 at 09:23:43AM +0200, Guillem Jover wrote: > On Sun, Jul 02, 2023 at 12:02:46AM +0300, Adrian Bunk wrote: > >... > > Linking a package with hardening=+all against a static library > > from a package not using hardening=+all cannot work on the > > affected architectures. >... > For example hppa has pie masked for build flags. If the porters for > alpha and/or ia64 consider that they should also get pie masked for > those arches, I'm fine doing the changes. Adrian, could you as porter request to get PIE masked on all ports architectures? This should fix FTBFS of more than a dozen packages on alpha/ia64/x32. FTR, the current status in ports is: 1. Architectures defaulting to PIE hurd-i386 powerpc ppc64 sparc64 2. PIE already masked hppa 3. affected architectures with several FTBFS due to it alpha ia64 x32 4. architectures where PIE is not masked but the dpkg PIE spec do not appear to make a difference loong64 m68k sh4 On Tue, Jul 04, 2023 at 01:12:48PM +0300, Adrian Bunk wrote: >... > PIE is unusual in being enabled by default on all release architectures, > but not being enabled by default on some non-release architectures. > > With some teams like Debian Med putting hardening=+all into every source > package, 20% of the source packages in the archive have it already set. > > This makes both sides of the PIE/non-PIE archive split on the affected > semi-PIE architectures huge. >... This is about the architectures in 3. above, where some FTBFS issues will be fixed by dpkg no longer creating a PIE/non-PIE archive split. I haven't checked whether the architectures in 4. have PIE always enabled or do not support PIE at all. > Thanks, > Guillem Thanks Adrian
Re: Bug#1040062: dpkg-dev: Please drop pie-{compile,link}.spec
On Tue, Jul 04, 2023 at 09:23:43AM +0200, Guillem Jover wrote:
> Hi!
Hi Guillem!
> On Sun, 2023-07-02 at 00:02:46 +0300, Adrian Bunk wrote:
>...
> > There are some problems with this:
> >
> > 1. PIE should either be default or not be used
> >
> > I suspect x32 might be able to default to PIE without problems
> > (there might just not be enough interest left to change the default).
> >
> > On alpha the toolchain has already become quite brittle
> > with frequent issues like (reproducible) linker segfaults,
> > any variations that affects the toolchain are bad.
> >
> > It is for the port maintainers to decide whether or not PIE
> > is considered stable on a port, and accordingly either make
> > it default (which also avoids the other issues below) or not.
> >
> > It is clear that a non-PIE architecture would no longer be
> > considered suitable as release architecture.
>
> In general the way this is handled in dpkg, is that if the flags
> supposedly work on that arch they are allowed, but if they are not
> supported or are broken then they are masked.
PIE is unusual in being enabled by default on all release architectures,
but not being enabled by default on some non-release architectures.
With some teams like Debian Med putting hardening=+all into every source
package, 20% of the source packages in the archive have it already set.
This makes both sides of the PIE/non-PIE archive split on the affected
semi-PIE architectures huge.
> > 2. It causes weird issues on undersupported architectures
> >
> > gluegen2 passes LDFLAGS to ld instead of gcc.
> >
> > Several packages have relocation errors only on affected
> > architectures.
> >
> > ...
> >
> > Such issues could be debugged and fixed, but in practice
> > trying to handle such issues that happen only with
> > pie-{compile,link}.spec creates additional work that frustrates
> > the few people keeping these non-release architectures alive.
>
> Regardless of this report, I think this would still be worthwhile,
> as otherwise you cannot for example disable them globally on arches
> where it is a builtin in the compiler (as those will also need the
> spec files.
Packages like python3.*-nopie could also set -fno-pie/-no-pie in
CFLAGS/LDFLAGS manually instead of using hardening=-pie, but different
from hardening=+all disabling PIE is not something that has unexpected
effects only on some ports architectures.
>...
> > 3. It breaks some cases of static linking
> >
> > Linking a package with hardening=+all against a static library
> > from a package not using hardening=+all cannot work on the
> > affected architectures.
> >
> > Static linking is relatively rare, but I remember requesting binNMUs
> > for static linking cases to fix FTBFS on release architectures when
> > the default changed before stretch.
>
> Hmm, ah or even packages not respecting DEB_BUILD_OPTIONS, right.
Bugs would fall under point 2.
Point 3 is about cases that cannot work despite no package doing
anything wrong.
Many packages needed fixing (or still have to be fixed) to enable
PIE because they had set hardening=+all,-pie before the default
changed in stretch. When all architectures defaulted to non-PIE,
hardening=+all did also cause PIE-related FTBFS on release architectures.
Semi-PIE architectures still have such problems between packages on
different sides of the PIE/non-PIE archive split.
> > Please drop pie-{compile,link}.spec, on the architectures
> > where it has any effect it is doing more harm than good.
>
> For example hppa has pie masked for build flags. If the porters for
> alpha and/or ia64 consider that they should also get pie masked for
> those arches, I'm fine doing the changes. Although that means on those
> ports it will not be possible to enable pie at all, even if asked for
> explicitly, as in «hardening=+pie».
Semi-PIE non-release architectures shouldn't exist, PIE on an
architecture should be a binary option decided by the porters
for this architecture.
If there are any porters left on an architecture and they consider PIE
stable, they can always ask the gcc maintainer to change the default.[1]
>...
> > The lowest effort fix would be to patch debian/rules of affected
> > packages to disable hardening=+pie on affected architectures,
> > but that would still be spending time on working around a problem
> > that shouldn't exist.
>
> Yeah, that does not look like the right thing to be spending time on.
>...
As long as we have at least one semi-PIE architecture, this is the only
realistic option for porters. If the porters for alpha and/or ia64 want
to have pie masked, such changes will still be required for x32.
> Thanks,
> Guillem
Thanks
Adrian
[1] no transition with rebuilds required, in the rare cases of FTBFS
when linking with a non-PIE static library this static library
package could be binNMU'ed when the problem appears
Re: Bug#1040062: dpkg-dev: Please drop pie-{compile,link}.spec
Hi!
On Sun, 2023-07-02 at 00:02:46 +0300, Adrian Bunk wrote:
> Package: dpkg-dev
> Version: 1.21.22
> Severity: normal
> X-Debbugs-Cc: debian-al...@lists.debian.org, debian-ia64@lists.debian.org
> Since stretch all release architectures are using PIE by default,
> and all future release architectures (including riscv64) will also
> use PIE by default.
>
> Many packages in Debian are building with hardening=+all, and the
> effect regarding PIE is "enable PIE for this package on some obscure
> ports architectures that don't have it enabled by default" which is
> unlikely to be what the maintainer intended.
>
> There are also some pre-stretch "hardening=+pie" left
> in some packages.
Yeah, I've never been very satisfied with our pie handling. :/
> There are some problems with this:
>
> 1. PIE should either be default or not be used
>
> I suspect x32 might be able to default to PIE without problems
> (there might just not be enough interest left to change the default).
>
> On alpha the toolchain has already become quite brittle
> with frequent issues like (reproducible) linker segfaults,
> any variations that affects the toolchain are bad.
>
> It is for the port maintainers to decide whether or not PIE
> is considered stable on a port, and accordingly either make
> it default (which also avoids the other issues below) or not.
>
> It is clear that a non-PIE architecture would no longer be
> considered suitable as release architecture.
In general the way this is handled in dpkg, is that if the flags
supposedly work on that arch they are allowed, but if they are not
supported or are broken then they are masked.
> 2. It causes weird issues on undersupported architectures
>
> gluegen2 passes LDFLAGS to ld instead of gcc.
>
> Several packages have relocation errors only on affected
> architectures.
>
> ...
>
> Such issues could be debugged and fixed, but in practice
> trying to handle such issues that happen only with
> pie-{compile,link}.spec creates additional work that frustrates
> the few people keeping these non-release architectures alive.
Regardless of this report, I think this would still be worthwhile,
as otherwise you cannot for example disable them globally on arches
where it is a builtin in the compiler (as those will also need the
spec files.
> The lowest effort fix would be to patch debian/rules of affected
> packages to disable hardening=+pie on affected architectures,
> but that would still be spending time on working around a problem
> that shouldn't exist.
Yeah, that does not look like the right thing to be spending time on.
> 3. It breaks some cases of static linking
>
> Linking a package with hardening=+all against a static library
> from a package not using hardening=+all cannot work on the
> affected architectures.
>
> Static linking is relatively rare, but I remember requesting binNMUs
> for static linking cases to fix FTBFS on release architectures when
> the default changed before stretch.
Hmm, ah or even packages not respecting DEB_BUILD_OPTIONS, right.
> Please drop pie-{compile,link}.spec, on the architectures
> where it has any effect it is doing more harm than good.
For example hppa has pie masked for build flags. If the porters for
alpha and/or ia64 consider that they should also get pie masked for
those arches, I'm fine doing the changes. Although that means on those
ports it will not be possible to enable pie at all, even if asked for
explicitly, as in «hardening=+pie».
Thanks,
Guillem
