Re: Bug#1040062: dpkg-dev: Please drop pie-{compile,link}.spec
On Fri, Nov 24, 2023 at 01:53:04AM +0100, Guillem Jover wrote: > Hi! Hi Guillem! Apologies for not replying to these emails earlier. > On Tue, 2023-10-31 at 10:52:40 +0100, Guillem Jover wrote: >... > > If PIE (via specs files) appears to work on x32, and changing the > > defaults in gcc is too much to bother, I think leaving it as is looks > > fine by me. But if this is causing problems as well and the x32 porters > > (if there's any remaining :), want to mask it alongside the other ports, > > let me know and I can also flip the switch for that one. > > If the porters would also like to see x32 masked, let me know and I > can also include it, otherwise I'll leave it as it is now. AFAIK for all ports architectures except Hurd and hppa the other Adrian is a porter, so should be able to ACK it. As I already wrote in [1] there is architecture other than alpha/ia64/x32 where the pie specs did actually enable PIE. On m68k and sh4 the specs might still be passed and cause FTBFS in packages like gluegen2 that pass LDFLAGS to ld, but there is no PIE enabling effect. > Thanks, > Guillem cu Adrian [1] https://lists.debian.org/debian-alpha/2023/10/msg2.html
Re: Bug#1040062: dpkg-dev: Please drop pie-{compile,link}.spec
On Tue, Jul 04, 2023 at 09:23:43AM +0200, Guillem Jover wrote: > On Sun, Jul 02, 2023 at 12:02:46AM +0300, Adrian Bunk wrote: > >... > > Linking a package with hardening=+all against a static library > > from a package not using hardening=+all cannot work on the > > affected architectures. >... > For example hppa has pie masked for build flags. If the porters for > alpha and/or ia64 consider that they should also get pie masked for > those arches, I'm fine doing the changes. Adrian, could you as porter request to get PIE masked on all ports architectures? This should fix FTBFS of more than a dozen packages on alpha/ia64/x32. FTR, the current status in ports is: 1. Architectures defaulting to PIE hurd-i386 powerpc ppc64 sparc64 2. PIE already masked hppa 3. affected architectures with several FTBFS due to it alpha ia64 x32 4. architectures where PIE is not masked but the dpkg PIE spec do not appear to make a difference loong64 m68k sh4 On Tue, Jul 04, 2023 at 01:12:48PM +0300, Adrian Bunk wrote: >... > PIE is unusual in being enabled by default on all release architectures, > but not being enabled by default on some non-release architectures. > > With some teams like Debian Med putting hardening=+all into every source > package, 20% of the source packages in the archive have it already set. > > This makes both sides of the PIE/non-PIE archive split on the affected > semi-PIE architectures huge. >... This is about the architectures in 3. above, where some FTBFS issues will be fixed by dpkg no longer creating a PIE/non-PIE archive split. I haven't checked whether the architectures in 4. have PIE always enabled or do not support PIE at all. > Thanks, > Guillem Thanks Adrian
Re: LibreOffice bridges/smoketest on mips(64)el (was: Re: unbreaking LibreOffices tests on at least release architectures)
On Mon, Jul 03, 2023 at 09:31:29PM +0200, Rene Engelhard wrote: > Hi, > > Am 25.06.23 um 13:37 schrieb Rene Engelhard: > > > what about the > > > following: > > > - make all test failures fatal on a*64 (since upstream tests these), and > > > - make smoketest failures fatal on all architectures (including ports) > > That was implemented (+ two more important tests) in experimental. See > https://buildd.debian.org/status/package.php?p=libreoffice > > It does > - bridgetest > - smoketest > - pyuno > > What fails for release archs astonishingly is only mips(64)el. It also failed on riscv64 (and powerpc), so that seems to be a criteria that catches the known-broken builds. >... > This test extension to be installed is a Java extension. > So I am running a nojava build on eller now... I don't really like disabling > Java since this opens Pandoras box but for mips64el we probably could do > that. It would also hint at a MIPS problem in LibreOffice, which might or might not be specific to Java. AFAIK OpenJDK on MIPS does not have any known major issues. The Zero build of OpenJDK on MIPS is of course slow, but that's also true on armel where the build succeeded. > Regards, > > Rene cu Adrian BTW: The MIPS-specific discussion should continue on debian-mips instead of debian-ports.
Re: Bug#1040062: dpkg-dev: Please drop pie-{compile,link}.spec
On Tue, Jul 04, 2023 at 09:23:43AM +0200, Guillem Jover wrote:
> Hi!
Hi Guillem!
> On Sun, 2023-07-02 at 00:02:46 +0300, Adrian Bunk wrote:
>...
> > There are some problems with this:
> >
> > 1. PIE should either be default or not be used
> >
> > I suspect x32 might be able to default to PIE without problems
> > (there might just not be enough interest left to change the default).
> >
> > On alpha the toolchain has already become quite brittle
> > with frequent issues like (reproducible) linker segfaults,
> > any variations that affects the toolchain are bad.
> >
> > It is for the port maintainers to decide whether or not PIE
> > is considered stable on a port, and accordingly either make
> > it default (which also avoids the other issues below) or not.
> >
> > It is clear that a non-PIE architecture would no longer be
> > considered suitable as release architecture.
>
> In general the way this is handled in dpkg, is that if the flags
> supposedly work on that arch they are allowed, but if they are not
> supported or are broken then they are masked.
PIE is unusual in being enabled by default on all release architectures,
but not being enabled by default on some non-release architectures.
With some teams like Debian Med putting hardening=+all into every source
package, 20% of the source packages in the archive have it already set.
This makes both sides of the PIE/non-PIE archive split on the affected
semi-PIE architectures huge.
> > 2. It causes weird issues on undersupported architectures
> >
> > gluegen2 passes LDFLAGS to ld instead of gcc.
> >
> > Several packages have relocation errors only on affected
> > architectures.
> >
> > ...
> >
> > Such issues could be debugged and fixed, but in practice
> > trying to handle such issues that happen only with
> > pie-{compile,link}.spec creates additional work that frustrates
> > the few people keeping these non-release architectures alive.
>
> Regardless of this report, I think this would still be worthwhile,
> as otherwise you cannot for example disable them globally on arches
> where it is a builtin in the compiler (as those will also need the
> spec files.
Packages like python3.*-nopie could also set -fno-pie/-no-pie in
CFLAGS/LDFLAGS manually instead of using hardening=-pie, but different
from hardening=+all disabling PIE is not something that has unexpected
effects only on some ports architectures.
>...
> > 3. It breaks some cases of static linking
> >
> > Linking a package with hardening=+all against a static library
> > from a package not using hardening=+all cannot work on the
> > affected architectures.
> >
> > Static linking is relatively rare, but I remember requesting binNMUs
> > for static linking cases to fix FTBFS on release architectures when
> > the default changed before stretch.
>
> Hmm, ah or even packages not respecting DEB_BUILD_OPTIONS, right.
Bugs would fall under point 2.
Point 3 is about cases that cannot work despite no package doing
anything wrong.
Many packages needed fixing (or still have to be fixed) to enable
PIE because they had set hardening=+all,-pie before the default
changed in stretch. When all architectures defaulted to non-PIE,
hardening=+all did also cause PIE-related FTBFS on release architectures.
Semi-PIE architectures still have such problems between packages on
different sides of the PIE/non-PIE archive split.
> > Please drop pie-{compile,link}.spec, on the architectures
> > where it has any effect it is doing more harm than good.
>
> For example hppa has pie masked for build flags. If the porters for
> alpha and/or ia64 consider that they should also get pie masked for
> those arches, I'm fine doing the changes. Although that means on those
> ports it will not be possible to enable pie at all, even if asked for
> explicitly, as in «hardening=+pie».
Semi-PIE non-release architectures shouldn't exist, PIE on an
architecture should be a binary option decided by the porters
for this architecture.
If there are any porters left on an architecture and they consider PIE
stable, they can always ask the gcc maintainer to change the default.[1]
>...
> > The lowest effort fix would be to patch debian/rules of affected
> > packages to disable hardening=+pie on affected architectures,
> > but that would still be spending time on working around a problem
> > that shouldn't exist.
>
> Yeah, that does not look like the right thing to be spending time on.
>...
As long as we have at least one semi-PIE architecture, this is the only
realistic option for porters. If the porters for alpha and/or ia64 want
to have pie masked, such changes will still be required for x32.
> Thanks,
> Guillem
Thanks
Adrian
[1] no transition with rebuilds required, in the rare cases of FTBFS
when linking with a non-PIE static library this static library
package could be binNMU'ed when the problem appears
Bug#1040062: dpkg-dev: Please drop pie-{compile,link}.spec
Package: dpkg-dev
Version: 1.21.22
Severity: normal
X-Debbugs-Cc: debian-al...@lists.debian.org, debian-ia64@lists.debian.org
[ Cc set to debian-alpha@ and debian-ia64@ since they are most affected ]
Since stretch all release architectures are using PIE by default,
and all future release architectures (including riscv64) will also
use PIE by default.
Many packages in Debian are building with hardening=+all, and the
effect regarding PIE is "enable PIE for this package on some obscure
ports architectures that don't have it enabled by default" which is
unlikely to be what the maintainer intended.
There are also some pre-stretch "hardening=+pie" left
in some packages.
There are some problems with this:
1. PIE should either be default or not be used
I suspect x32 might be able to default to PIE without problems
(there might just not be enough interest left to change the default).
On alpha the toolchain has already become quite brittle
with frequent issues like (reproducible) linker segfaults,
any variations that affects the toolchain are bad.
It is for the port maintainers to decide whether or not PIE
is considered stable on a port, and accordingly either make
it default (which also avoids the other issues below) or not.
It is clear that a non-PIE architecture would no longer be
considered suitable as release architecture.
2. It causes weird issues on undersupported architectures
gluegen2 passes LDFLAGS to ld instead of gcc.
Several packages have relocation errors only on affected
architectures.
...
Such issues could be debugged and fixed, but in practice
trying to handle such issues that happen only with
pie-{compile,link}.spec creates additional work that frustrates
the few people keeping these non-release architectures alive.
The lowest effort fix would be to patch debian/rules of affected
packages to disable hardening=+pie on affected architectures,
but that would still be spending time on working around a problem
that shouldn't exist.
3. It breaks some cases of static linking
Linking a package with hardening=+all against a static library
from a package not using hardening=+all cannot work on the
affected architectures.
Static linking is relatively rare, but I remember requesting binNMUs
for static linking cases to fix FTBFS on release architectures when
the default changed before stretch.
Please drop pie-{compile,link}.spec, on the architectures
where it has any effect it is doing more harm than good.
Thanks
Re: unbreaking LibreOffices tests on at least release architectures
On Tue, Jun 20, 2023 at 05:52:44AM +0200, Rene Engelhard wrote: > Hi, > > Am 19.06.23 um 23:29 schrieb Rene Engelhard: > > > The pragmatic option would be to run only a smoketest for build success > > > on architectures not tested by upstream. > > > > And have Format->Character in Impress crash with Bus error like on > > mipsel? That doesn't sound too good for basic quality. > > > > There is a "smoketest" but it does just basic start. open, close stuff. > > Not even basic usage. > > That said, that is the smoketest on mipsel: >... Assuming the smoketest currently also fails on riscv64, what about the following: - make all test failures fatal on a*64 (since upstream tests these), and - make smoketest failures fatal on all architectures (including ports) > Regards, > > Rene cu Adrian
Re: unbreaking LibreOffices tests on at least release architectures
On Mon, Jun 19, 2023 at 11:29:34PM +0200, Rene Engelhard wrote: >... > Am 19.06.23 um 23:19 schrieb Adrian Bunk: >... > > For such a complex package I would expect 32bit breakage in every > > release if upstream no longer tests on 32bit. > Indeed, though at least for 32bit *build* issues they keep fixing them if I > report them. > > The pragmatic option would be to run only a smoketest for build success > > on architectures not tested by upstream. > > And have Format->Character in Impress crash with Bus error like on mipsel? > That doesn't sound too good for basic quality. > > There is a "smoketest" but it does just basic start. open, close stuff. Not > even basic usage. Let's be realistic regarding the available options, because the one you want is not available. You want every !a*64 architecture to have a porter spending time on fixing libreoffice. And thinking this through, since regressions in new upstream versions are expected to be frequent you want new upstream versions of libreoffice blocked from testing migration by any regression on one architecture until a porter for this architecture has fixed the regression. A new architecture like riscv64 might have enough porters for fixing issues once or for some limited duration. That's it. For each architecture you have the options: 1. declare libreoffice good enough on this architecture, or 2. don't build libreoffice on this architecture There is no third option that architectures will provide porters fixing your package all the time. There are several other packages of comparable complexity, size and testsuite (e.g. mozjs* or rustc). For a successful build they are using either just a smoketest, or a maximum number of tolerable testsuite failures. > Regards, > > Rene cu Adrian
Re: unbreaking LibreOffices tests on at least release architectures
On Sun, Jun 18, 2023 at 09:31:05AM +0200, Rene Engelhard wrote: >... > I won't be of much help here unfortunately, except > maybe testing patches, but then again there's porterboxes >... You are the only one who could realistically debug many of these. E.g. on armel it says: Fatal exception: Signal 6 Stack: /<>/instdir/program/libuno_sal.so.3(+0x3c2e4)[0xb6ec32e4] /<>/instdir/program/libuno_sal.so.3(+0x3c534)[0xb6ec3534] /lib/arm-linux-gnueabi/libc.so.6(__default_rt_sa_restorer+0x0)[0xb6ad58f0] /lib/arm-linux-gnueabi/libc.so.6(+0x7f47c)[0xb6b1e47c] /lib/arm-linux-gnueabi/libc.so.6(gsignal+0x14)[0xb6ad4360] Aborted (core dumped) Fixing something like this would involve generating a backtrace, and then you are likely the only person in Debian who could tell what is actually going on there. There are likely also build or debug tricks you know that a porter would not know. Debugging something like this is only feasible with reasonable effort if a porter who knows the port with its caveats debugs it together with a package maintainer who knows the internals of the package. On Mon, Jun 19, 2023 at 12:04:45AM +0200, Kurt Roeckx wrote: >... > Do you think Debian doesn't have any developers/porters anymore? >... For porters that's actually close to being true. There were times when porter numbers for a release architecture were numbers like 6 or 9. No release architecture in bookworm had more than 2 porters. No porters were required on amd64, the number of distinct people who are listed as porter for one or more of the 8 other bookworm release architecture is 5 DDs and 2 non-DDs. On Sun, Jun 18, 2023 at 09:31:05AM +0200, Rene Engelhard wrote: >... > For riscv64 I already pointed that out in the thread starting at > https://lists.debian.org/debian-riscv/2023/06/msg0.html, but for the > other architectures there is the mail now. riscv64 is different because > the failures are even more big than any other down below and it's actually a > new architecture anyway. > > Also note I am not talking about the debian-ports architectures. Those I > forgot and I have no problems making them stay into "testsuite ran but > results ignored" set. > > Right now, the only architectures where the test actually work (ignoring the > occassional breakage on arm64 which is fixed upstream since they do > aarch64 flatpak builds) is amd64 and arm64. > > With various different 32-bit, endian and whatever else breakage > ppopping up the other architectures constantly were moved in the set > where the testsuite was run but the results were ignored. For s390x, > where the macros tests hangs it even was in the set where the tests even > were not ran, since a hang then also ends up in > "E: Build killed with signal TERM after 150 minutes of inactivity". > > This was sweeping under the carpet for sure, but this was needed due to > it being a release architecture :( >... For such a complex package I would expect 32bit breakage in every release if upstream no longer tests on 32bit. The pragmatic option would be to run only a smoketest for build success on architectures not tested by upstream. cu Adrian
Re: Porter roll call for Debian Bullseye
On Sun, Dec 06, 2020 at 01:03:17PM +0100, Matthias Klose wrote: > On 12/1/20 5:02 AM, YunQiang Su wrote: > > I am sorry for the later response. > >Hi, > > > > I am an active porter for the following architectures and I intend > > to continue this for the lifetime of the Bullseye release (est. end > > of 2024): > > > > For mipsel and mips64el, I > > - test most packages on this architecture > > - run a Debian testing or unstable system on port that I use regularly > > - fix toolchain issues > > great ;-) gcc-cross-mipsen and gcc-10-cross-mipsen have never been in > testing ... >... This problem has now been resolved: https://tracker.debian.org/pkg/gcc-defaults-mipsen https://tracker.debian.org/pkg/gcc-10-cross-mipsen cu Adrian
Re: Porter roll call for Debian Bullseye
On Sun, Dec 06, 2020 at 01:03:17PM +0100, Matthias Klose wrote: > On 12/1/20 5:02 AM, YunQiang Su wrote: > > I am sorry for the later response. > >Hi, > > > > I am an active porter for the following architectures and I intend > > to continue this for the lifetime of the Bullseye release (est. end > > of 2024): > > > > For mipsel and mips64el, I > > - test most packages on this architecture > > - run a Debian testing or unstable system on port that I use regularly > > - fix toolchain issues > > great ;-) gcc-cross-mipsen and gcc-10-cross-mipsen have never been in > testing ... The main blocker for that seems to be a bug that was fixed in gcc-10 10.2.0-20, a new source upload with the gcc-10-source build dependency bumped to (>= 10.2.0-20~) should fix that. binNMU won't work due to binary-all. > > - triage arch-specific bugs > > - fix arch-related bugs > > any help with #972269 ? I looked into it back then, at least for me there was nothing obvious why dbus-python failed and not other packages. A few months earlier one other package had a similar problem, but it seems rare enough that it shouldn't be a high priority for anyone. cu Adrian
Re: Bug#845193: dpkg: recent -specs PIE changes break openssl
On Thu, Nov 24, 2016 at 04:35:28PM +0100, Guillem Jover wrote: >... > On Thu, 2016-11-24 at 14:52:33 +, Thorsten Glaser wrote: >... > > Worse, they break *differently* on whether… > > > > >Precisely to make the behavior consistent on all architectures, dpkg > > >enables PIE (conditionally if no other flags marks it as to be > > >disabled) on all architectures were gcc has not enabled this by > > >default. > > > > … that. And that is just plain wrong. Either dpkg should inject > > -specs= stuff on all architectures or on none. Differing like this > > just invites hidden and hard to track down bugs. > > As long as gcc enables PIE on a subset, there will be need to inject > some form of specs on either subset of those arches, either on > hardening=+pie or on hardening=-pie, pick yout poison. :( >... Both gcc and dpkg playing with PIE just increased the number of bugs without bringing any benefit. I fixed many PIE related issues in packages when the gcc change was. And now we got a new batch of FTBFS bugs for cases where the dpkg specs change broke packages using "hardening=+all,-pie". Please do the following: 1. discuss with porters whether PIE is working on their architecture 2. gcc and dpkg maintainers have to agree which package enables PIE > Thanks, > Guillem cu Adrian -- "Is there not promise of rain?" Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. "Only a promise," Lao Er said. Pearl S. Buck - Dragon Seed
Re: Architecture qualification meeting, scheduling
[ fullquote adding -ports, for people not following -release or -devel ]
On Fri, Oct 07, 2016 at 06:35:07PM +0100, Jonathan Wiltshire wrote:
> Hi,
>
> I am arranging the final architecture qualification meeting for Stretch.
> This is primarily of interest to the release team, but I will also take
> input from porters.
>
> As the schedule is currently wide open, please express your availability in
> the linked Doodle poll. There are 56 slots available, mostly in the European
> evening but a handful are daytime coinciding with the Cambridge
> mini-Debconf.
>
> Porters, please note your architecture in your response ("name (arch)").
>
> About the format of the meeting:
> Much like the Jessie meeting, it will be held via IRC in
> oftc.net/#debian-release and will be primarily a discussion amongst the
> release team. We will evaluate each port on the most up-to-date information
> available to us, and determine if it will be a release architecture for
> Stretch. We may ask for clarification from porters who are present if there
> are points at issue, but we ask that you are read-only otherwise.
>
> http://doodle.com/poll/362qvb89cvu43d4z
Is https://release.debian.org/stretch/arch_qualify.html the up-to-date
information available to you, and the "candidate" line how a decision
would look like based on the current information?
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
