Re: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Sat, Jan 06, 2024 at 10:01:30AM -0700, Sam Hartman wrote: > > "Steve" == Steve Langasek writes: > >> At one level, krb5-multidev only has an rdep of 5, but I suspect > >> the rdep count for libkrb5-dev is somewhat larger:-) I don't know > >> how many packages would be removed from the transition if we > >> decide most of the krb5 libraries do not need to transition. > Steve> If we determined only libkdb5-10 and libgssrpc4 were > Steve> affected, that does significantly reduce the number of > Steve> packages needing to be rebuilt because of krb5, though maybe > Steve> most of those packages also depend on other libraries that > Steve> are transitioning. > I confirm libgssrpc is also affected. > (It would be nice to get rid of libgssrpc entirely and just use the > system tli-rpc). > My bad; I was so focused on time_t I forgot about timeval. > However, I do think that we're limited only to libkdb5-10 and > libgssrpc4. > I would prefer not to change the abi of the other libraries, especially > libkrb5, libk5crypto3 and libgssapi-krb5-2. > What can I do to help facilitate that? I've updated the override list here. Reduces the count of revdeps to be rebuilt from 5450+168 to 5439+168. Thanks, -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developer https://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: PGP signature
Re: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
> "Steve" == Steve Langasek writes: >> At one level, krb5-multidev only has an rdep of 5, but I suspect >> the rdep count for libkrb5-dev is somewhat larger:-) I don't know >> how many packages would be removed from the transition if we >> decide most of the krb5 libraries do not need to transition. Steve> If we determined only libkdb5-10 and libgssrpc4 were Steve> affected, that does significantly reduce the number of Steve> packages needing to be rebuilt because of krb5, though maybe Steve> most of those packages also depend on other libraries that Steve> are transitioning. I confirm libgssrpc is also affected. (It would be nice to get rid of libgssrpc entirely and just use the system tli-rpc). My bad; I was so focused on time_t I forgot about timeval. However, I do think that we're limited only to libkdb5-10 and libgssrpc4. I would prefer not to change the abi of the other libraries, especially libkrb5, libk5crypto3 and libgssapi-krb5-2. What can I do to help facilitate that? --Sam signature.asc Description: PGP signature
Re: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Fri, Jan 05, 2024 at 12:23:00AM -0800, Steve Langasek wrote: > I am also attaching here the dd-list output for the packages that will need > to be sourcefully NMUed for the transition, for your review. I could readily identify a number of packages (incomplete) also affected by DEP17. Whenever you face files in aliased locations (other than systemd units), please go via experimental to let dumat judge your upload. Check the bookworm package for files in aliased locations, not the unstable one. >uhd DEP17-affected, probably harmless >btrfs-progs DEP17-affected >pacemaker (U) DEP17-affected >libmtp DEP17-affected, probably harmless >openafs (U) DEP17-affected, probably harmless >samba (U) DEP17-affected, probably harmless >apt DEP17-affected, probably harmless >ceph DEP17-affected >util-linux (U) Do not upload to unstable directly. Will need mitigations. >libapogee3 Do not upload to unstable directly. Will need mitigations. >libfishcamp Do not upload to unstable directly. Will need mitigations. >libplayerone Do not upload to unstable directly. Will need mitigations. >libricohcamerasdk Do not upload to unstable directly. Will need mitigations. >libsbig Do not upload to unstable directly. Will need mitigations. >boinc DEP17-affected >gcc-13 If this affects libgcc-s1, do not upload to unstable as you risk deleting libgcc_s.so.1. Will need mitigations in that case. >libosmo-sccp DEP17-affected, probably harmless >libgpod DEP17-affected, probably harmless >libselinux Do not upload to unstable directly. Will need mitigations. >zfs-linux Do not upload to unstable directly. Will need mitigations. >libguestfs (U) DEP17-affected, probably harmless >libtirpc Do not upload to unstable directly. Will need mitigations. >fuse >fuse3 Do not upload to unstable directly. Will need mitigations. >audit Do not upload to unstable directly. Will need mitigations. >zlib Do not upload to unstable. Will cause file loss unless mitigated. >readline Do not upload to unstable. Will cause file loss unless mitigated. >openrc (U) Do not upload to unstable directly. Will need mitigations. >krb5 DEP17-affected As predicted, this is going to have annoying interactions and the list here definitely is incomplete. Helmut
Re: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Fri, Jan 05, 2024 at 02:23:59PM -0700, Sam Hartman wrote: > > "Steve" == Steve Langasek writes: > Steve> - In multi-library packages, there is no reliable way to map > Steve> from a set of headers in a dev package to specific shared > Steve> libraries in a runtime library package that's not > Steve> additionally computationally prohibitive; we therefore > Steve> conservatively assume that if any headers from a source > Steve> package show time_t ABI changes, all the runtime library > Steve> packages from the source package are affected by the > Steve> transition. This seems sensible in general, but er, in the > Steve> pathological case we have Source: zlib that now ships both > Steve> zlib1g-dev and libminizip-dev, zlib1g-dev is confirmed to not > Steve> be ABI breaking, libminizip-dev has failed to analyze, and we > Steve> don't want to force a transition for zlib1g because of > Steve> libminizip (!). Current plan is to simply special case > Steve> zlib1g, but there could be other problem packages we haven't > Steve> identified. > I think krb5 may be in this category. > In particular, it looks like time_t is only exposed in the kdb.h API, > which appears to have no reverse depends outside of krb5. > I actually think that the freeIPA server may have a plugin that depends > on kdb.h, and Samba4 can be built in such a way that it depends on > kdb.h, but it looks like neither of those applies to the Debian archive. https://adrien.dcln.fr/misc/armhf-time_t/2023-12-18/compat_reports/libkrb5-dev/lfs_to_time_t/compat_report.html indicates that the CLIENT struct is affected because the cl_call function pointer takes a 'struct timeval' as an argument. So that's not internal to kdb, it looks like an actual ABI change? > At one level, krb5-multidev only has an rdep of 5, but I suspect the > rdep count for libkrb5-dev is somewhat larger:-) > I don't know how many packages would be removed from the transition if > we decide most of the krb5 libraries do not need to transition. If we determined only libkdb5-10 and libgssrpc4 were affected, that does significantly reduce the number of packages needing to be rebuilt because of krb5, though maybe most of those packages also depend on other libraries that are transitioning. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developer https://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: PGP signature
Re: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Fri, Jan 05, 2024 at 05:28:37PM +0100, Guillem Jover wrote: > > Guillem Jover > >libaio > >libmd > >liburing > I checked these, and it looks like libmd and liburing are > false-positives? > * libmd uses AC_SYS_LARGEFILE, and on 32-bit arches it is already built >with LFS, the problem is that the header exposes off_t which means >the code linking against it needs to match its build flags, >otherwise they would already be broken now. You might want to look >into this as a potential pattern for other false-positives probably. >(I should probably update upstream the .pc file to include the >-D_FILE_OFFSET_BITS=64 flags if enabled, but I don't think the >analysis used .pc files anyway.) Thanks - yes, the analysis didn't use .pc files, though I'm checking if that's something we could improve on. But as you say it doesn't help with the current libmd anyway, so I'm adding that to the manual exclusion list. Reduces the count of revdeps to be rebuilt from 5450+178 to 5450+169. > * liburing is marked as LFS-sensitive, but that comes from inline >functions using off_t which end up casting that into an u32 type, >so I don't think this should affect the ABI. It is also forcibly >built with -D_FILE_OFFSET_BITS=64 in the upstream build system >(and with future=+lfs for good measure in the packaging, which >I'll switch to abi=+lfs). io_uring_prep_fadvise and io_uring_prep_madvise appear to be symbols publicly exposed in the ABI of /usr/lib/x86_64-linux-gnu/liburing-ffi.so.2 so I don't think it's true that the marking is only because of inline functions? But as this is also already built with LFS and is a false-positive, I've removed this as well. Reduces the count of revdeps to be rebuilt from 5450+169 to 5450+168 (not much improvement :). -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developer https://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: PGP signature
Re: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Fri, Jan 05, 2024 at 06:59:20PM +0100, Sebastian Ramacher wrote: > > > > I am also attaching here the dd-list output for the packages that will > > > > need > > > > to be sourcefully NMUed for the transition, for your review. > > > Why do the need sourceful NMUs if they just need to be rebuilt? > > Sorry, if the original message hadn't been lost somewhere in the mail > > system before being delivered to debian-devel (I've now tried to resend it), > > this might have been clearer from context. Guillem points out the mail has > > been delivered to debian-release, so you can read the whole thing there: > > https://lists.debian.org/debian-release/2024/01/msg00033.html > > Anyway, this is the list of source packages containing libraries whose ABI > > will change. So the packages need to be renamed in order to expose the ABI > > incompatibility to reverse-dependencies. > I am confused. Above you say: > > these in turn have 174 additional > > reverse-dependencies that would need rebuilt (list attached). > This sounds to me like those are packages that are involved in the > transition and need rebuilds, but do not change their ABI. And in fact, > for most of packages that I maintain on the list, the ABI does not > change. > Can you please clarify which of the packages in your lists require > changes to the binary package names and which do not? Sorry for the confusion. The two lists requiring binary package name changes are the attachments named 'source-packages' and 'lfs-and-depends-time_t'. This is what I fed into dd-list, and encompass 1248 source packages (1195 + 53). -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developer https://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: PGP signature
Re: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
> "Steve" == Steve Langasek writes: Steve> - In multi-library packages, there is no reliable way to map Steve> from a set of headers in a dev package to specific shared Steve> libraries in a runtime library package that's not Steve> additionally computationally prohibitive; we therefore Steve> conservatively assume that if any headers from a source Steve> package show time_t ABI changes, all the runtime library Steve> packages from the source package are affected by the Steve> transition. This seems sensible in general, but er, in the Steve> pathological case we have Source: zlib that now ships both Steve> zlib1g-dev and libminizip-dev, zlib1g-dev is confirmed to not Steve> be ABI breaking, libminizip-dev has failed to analyze, and we Steve> don't want to force a transition for zlib1g because of Steve> libminizip (!). Current plan is to simply special case Steve> zlib1g, but there could be other problem packages we haven't Steve> identified. I think krb5 may be in this category. In particular, it looks like time_t is only exposed in the kdb.h API, which appears to have no reverse depends outside of krb5. I actually think that the freeIPA server may have a plugin that depends on kdb.h, and Samba4 can be built in such a way that it depends on kdb.h, but it looks like neither of those applies to the Debian archive. At one level, krb5-multidev only has an rdep of 5, but I suspect the rdep count for libkrb5-dev is somewhat larger:-) I don't know how many packages would be removed from the transition if we decide most of the krb5 libraries do not need to transition. signature.asc Description: PGP signature
Re: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hi Steve On 2024-01-05 09:42:06 -0800, Steve Langasek wrote: > Hi Sebastian, > > On Fri, Jan 05, 2024 at 06:34:38PM +0100, Sebastian Ramacher wrote: > > On 2024-01-05 00:23:00 -0800, Steve Langasek wrote: > > > On Fri, Jan 05, 2024 at 12:17:04AM -0800, Steve Langasek wrote: > > > > == Results == > > > > > The overall findings of this analysis are 1,745 "dev" packages which > > > > either are confirmed to have ABI changes or could not be checked; > > > > mapping to 2,154 runtime libraries (list attached) from 1,195 source > > > > packages (list attached) and 5,477 reverse-dependencies requiring > > > > no-change rebuilds (list attached). This is within the previously > > > > calculated range of "5300 to 5600", but there are a number of > > > > newly-identified packages that fail to compile and have a large number > > > > of reverse-dependencies. I will continue to work to identify > > > > false-positives here in the hopes of bringing this count down before > > > > pulling the trigger on an actual transition. > > > > [...] > > > > > In addition, Guillem pointed out that if there are libraries whose > > > > ABIs are lfs-sensitive but not time_t-sensitive, and either they > > > > themselves depend on libraries which are time_t-sensitive or they have > > > > reverse-dependencies that do, so they will also need to be included in > > > > the transition. I have identified a list of 53 packages in this > > > > category (list attached); these in turn have 174 additional > > > > reverse-dependencies that would need rebuilt (list attached). > > > > I am also attaching here the dd-list output for the packages that will > > > need > > > to be sourcefully NMUed for the transition, for your review. > > > Why do the need sourceful NMUs if they just need to be rebuilt? > > Sorry, if the original message hadn't been lost somewhere in the mail > system before being delivered to debian-devel (I've now tried to resend it), > this might have been clearer from context. Guillem points out the mail has > been delivered to debian-release, so you can read the whole thing there: > > https://lists.debian.org/debian-release/2024/01/msg00033.html > > Anyway, this is the list of source packages containing libraries whose ABI > will change. So the packages need to be renamed in order to expose the ABI > incompatibility to reverse-dependencies. I am confused. Above you say: > these in turn have 174 additional > reverse-dependencies that would need rebuilt (list attached). This sounds to me like those are packages that are involved in the transition and need rebuilds, but do not change their ABI. And in fact, for most of packages that I maintain on the list, the ABI does not change. Can you please clarify which of the packages in your lists require changes to the binary package names and which do not? Cheers -- Sebastian Ramacher
Re: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hi Sebastian,
On Fri, Jan 05, 2024 at 06:34:38PM +0100, Sebastian Ramacher wrote:
> On 2024-01-05 00:23:00 -0800, Steve Langasek wrote:
> > On Fri, Jan 05, 2024 at 12:17:04AM -0800, Steve Langasek wrote:
> > > == Results ==
> > > The overall findings of this analysis are 1,745 "dev" packages which
> > > either are confirmed to have ABI changes or could not be checked;
> > > mapping to 2,154 runtime libraries (list attached) from 1,195 source
> > > packages (list attached) and 5,477 reverse-dependencies requiring
> > > no-change rebuilds (list attached). This is within the previously
> > > calculated range of "5300 to 5600", but there are a number of
> > > newly-identified packages that fail to compile and have a large number
> > > of reverse-dependencies. I will continue to work to identify
> > > false-positives here in the hopes of bringing this count down before
> > > pulling the trigger on an actual transition.
> > [...]
> > > In addition, Guillem pointed out that if there are libraries whose
> > > ABIs are lfs-sensitive but not time_t-sensitive, and either they
> > > themselves depend on libraries which are time_t-sensitive or they have
> > > reverse-dependencies that do, so they will also need to be included in
> > > the transition. I have identified a list of 53 packages in this
> > > category (list attached); these in turn have 174 additional
> > > reverse-dependencies that would need rebuilt (list attached).
> > I am also attaching here the dd-list output for the packages that will need
> > to be sourcefully NMUed for the transition, for your review.
> Why do the need sourceful NMUs if they just need to be rebuilt?
Sorry, if the original message hadn't been lost somewhere in the mail
system before being delivered to debian-devel (I've now tried to resend it),
this might have been clearer from context. Guillem points out the mail has
been delivered to debian-release, so you can read the whole thing there:
https://lists.debian.org/debian-release/2024/01/msg00033.html
Anyway, this is the list of source packages containing libraries whose ABI
will change. So the packages need to be renamed in order to expose the ABI
incompatibility to reverse-dependencies. That's not a no-change rebuild.
This is consistent with historical practice in Debian for
toolchain-triggered ABI changes ('g' for libc5->glibc; c102; c2; v5; ldbl)
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer https://www.debian.org/
slanga...@ubuntu.com vor...@debian.org
signature.asc
Description: PGP signature
Re: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On 2024-01-05 00:23:00 -0800, Steve Langasek wrote: > On Fri, Jan 05, 2024 at 12:17:04AM -0800, Steve Langasek wrote: > > == Results == > > > > The overall findings of this analysis are 1,745 "dev" packages which either > > are confirmed to have ABI changes or could not be checked; mapping to 2,154 > > runtime libraries (list attached) from 1,195 source packages (list attached) > > and 5,477 reverse-dependencies requiring no-change rebuilds (list attached). > > This is within the previously calculated range of "5300 to 5600", but there > > are a number of newly-identified packages that fail to compile and have a > > large number of reverse-dependencies. I will continue to work to identify > > false-positives here in the hopes of bringing this count down before pulling > > the trigger on an actual transition. > > [...] > > > In addition, Guillem pointed out that if there are libraries whose ABIs are > > lfs-sensitive but not time_t-sensitive, and either they themselves depend on > > libraries which are time_t-sensitive or they have reverse-dependencies that > > do, so they will also need to be included in the transition. I have > > identified a list of 53 packages in this category (list attached); these in > > turn have 174 additional reverse-dependencies that would need rebuilt (list > > attached). > > I am also attaching here the dd-list output for the packages that will need > to be sourcefully NMUed for the transition, for your review. Why do the need sourceful NMUs if they just need to be rebuilt? Cheers -- Sebastian Ramacher
Re: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hi! [ It seems the original post didn't get through to debian-devel (yet?), I found it on debian-release though https://lists.debian.org/debian-release/2024/01/msg00033.html, you might want to repost it here though, so that it can be commented on properly? :) ] On Fri, 2024-01-05 at 00:23:00 -0800, Steve Langasek wrote: > On Fri, Jan 05, 2024 at 12:17:04AM -0800, Steve Langasek wrote: > > == Results == > > > > The overall findings of this analysis are 1,745 "dev" packages which either > > are confirmed to have ABI changes or could not be checked; mapping to 2,154 > > runtime libraries (list attached) from 1,195 source packages (list attached) > > and 5,477 reverse-dependencies requiring no-change rebuilds (list attached). > > This is within the previously calculated range of "5300 to 5600", but there > > are a number of newly-identified packages that fail to compile and have a > > large number of reverse-dependencies. I will continue to work to identify > > false-positives here in the hopes of bringing this count down before pulling > > the trigger on an actual transition. > Guillem Jover >libaio >libmd >liburing I checked these, and it looks like libmd and liburing are false-positives? * libmd uses AC_SYS_LARGEFILE, and on 32-bit arches it is already built with LFS, the problem is that the header exposes off_t which means the code linking against it needs to match its build flags, otherwise they would already be broken now. You might want to look into this as a potential pattern for other false-positives probably. (I should probably update upstream the .pc file to include the -D_FILE_OFFSET_BITS=64 flags if enabled, but I don't think the analysis used .pc files anyway.) * liburing is marked as LFS-sensitive, but that comes from inline functions using off_t which end up casting that into an u32 type, so I don't think this should affect the ABI. It is also forcibly built with -D_FILE_OFFSET_BITS=64 in the upstream build system (and with future=+lfs for good measure in the packaging, which I'll switch to abi=+lfs). Thanks, Guillem
