Bug#1026204: tar FTBFS on armel, armhf, i386, hppa, powerpc and sparc64
Control: tags -1 pending patch Hi all, On Tue, 31 Jan 2023 18:27:27 +0100 Andreas Henriksson wrote: > So, please use this hunk instead. It compiles for me on amd64 and 32-bit hppa. > > export DEB_BUILD_MAINT_OPTIONS = future=+lfs > export DEB_CFLAGS_MAINT_APPEND = -Wall -Wno-analyzer-null-argument Might be useful to add a comment here saying: # Workaround gnulib issue: The below three lines can be dropped once tar >= 1.35 is used. > ifneq ($(shell dpkg-architecture -qDEB_TARGET_ARCH_BITS),64) > export DEB_CPPFLAGS_MAINT_APPEND = -D_TIME_BITS=64 > endif > > DPKG_EXPORT_BUILDFLAGS = 1 > include /usr/share/dpkg/buildflags.mk > --- I have uploaded the attached debdiff to DELAYED/2 and pushed my changes to salsa. If nobody shouts that I understood the intentions correctly and/or cancels the upload, it should land in two days in unstable Paul diff -Nru tar-1.34+dfsg/debian/changelog tar-1.34+dfsg/debian/changelog --- tar-1.34+dfsg/debian/changelog 2022-11-20 15:52:41.0 +0100 +++ tar-1.34+dfsg/debian/changelog 2023-04-06 16:25:47.0 +0200 @@ -1,3 +1,11 @@ +tar (1.34+dfsg-1.2) unstable; urgency=medium + + * Non-maintainer upload. + * Build with lfs and -D_TIME_BITS=64 on 32 bits archs (Closes: #1026204) +Thanks to Andreas Henriksson and Helge Deller + + -- Paul Gevers Thu, 06 Apr 2023 16:25:47 +0200 + tar (1.34+dfsg-1.1) unstable; urgency=medium * Non-maintainer upload diff -Nru tar-1.34+dfsg/debian/rules tar-1.34+dfsg/debian/rules --- tar-1.34+dfsg/debian/rules 2022-11-19 16:38:39.0 +0100 +++ tar-1.34+dfsg/debian/rules 2023-04-06 16:25:47.0 +0200 @@ -1,15 +1,23 @@ #!/usr/bin/make -f +include /usr/share/dpkg/architecture.mk + DEB_BUILD_GNU_TYPE ?= $(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE) DEB_HOST_GNU_TYPE ?= $(shell dpkg-architecture -qDEB_HOST_GNU_TYPE) ifneq ($(DEB_BUILD_GNU_TYPE),$(DEB_HOST_GNU_TYPE)) CONFARGS = --host=$(DEB_HOST_GNU_TYPE) endif -CFLAGS = `dpkg-buildflags --get CFLAGS` -CFLAGS += -Wall -Wno-analyzer-null-argument -LDFLAGS += `dpkg-buildflags --get LDFLAGS` -CPPFLAGS = `dpkg-buildflags --get CPPFLAGS` +export DEB_BUILD_MAINT_OPTIONS = future=+lfs +export DEB_CFLAGS_MAINT_APPEND = -Wall -Wno-analyzer-null-argument +# Workaround gnulib issue: The below three lines can be dropped once +# tar >= 1.35 is used. +ifeq (32,$(DEB_HOST_ARCH_BITS)) +export DEB_CPPFLAGS_MAINT_APPEND = -D_TIME_BITS=64 +endif + +DPKG_EXPORT_BUILDFLAGS = 1 +include /usr/share/dpkg/buildflags.mk export BUILD_DATE = $(shell dpkg-parsechangelog | sed -n -e 's/^Date: //p') OpenPGP_signature Description: OpenPGP digital signature
Bug#1026204: tar FTBFS on armel, armhf, i386, hppa, powerpc and sparc64
Hi, Is the issue that with older glibc versions, it was silently casted to a 32 bit value, but now that glibc supports 64 bit, it knows it can't represent it, and gives an error? Maybe for bookworm, we should just ignore the test error? Kurt
Bug#1026204: tar FTBFS on armel, armhf, i386, hppa, powerpc and sparc64
Just noticed this bug. The discussion in this bug makes me worry that people do not fully understand the implications of enabling 64-bit time and large file system support respectively. It's great to see people starting to care about this issue and fix things (it's overdue), but I'm just chiming in to point out that some care is needed before turning these options on. Debian is in the process of developing a plan for this transition, but has not got very far yet. I have just started a wiki page to cover the issues (and tagged this bug): https://wiki.debian.org/ReleaseGoals/64bit-time https://bugs.debian.org/cgi-bin/pkgreport.cgi?users=debian-de...@lists.debian.org;tag=time64 If tar is currently building without LFS enabled, I'd suggest maybe turning that on, checking things, and uploading before also turning on 64bit time_t. They can be done separately (in that order only). Both have the potential to change file formats and ABIs (although tar is a program not a library so no ABI is exposed). Hopefully upstream has looked at all this carefully and is reasonably sure nothing important will break? For LFS enablement you should be aware that LARGEFILE_SOURCE and FILE_OFFSET_BITS=64 do different things. The former enables both 32 and 64-bit interfaces, the latter chooses the 64-bit interface only. You probably only want the latter. (depending how these are used in the codebase, it may not make any practical difference) For 64-bit time_t enablement you might want to wait for the dpkg standard debian mechanism to appear and use that: https://bugs.debian.org/1030159 You certainly want to be sure that things tar depends on (and expose ABIs or file formats that change) have switched first. Now tar is close to the bottom of the stack so this may well be fine, but it's linked against libacl, libselinux, libc and libpcre2, so we should check those. I am in the process of running abi-compliance-checker over all debian libraries to get a list of ones that expose an ABI change from either enabling LFS or 64bit-time_t. tests so far say: libacl1 is safe (no change) libselinux is unknown (did not build) libpcre2 is unknown (did not build) (I'll take a look at those now as they are pretty fundamental) If you choose to use gnulib, note that it turns 64-bit time on by default if detected in glibc, unless you set a macro to explicitly keep 32-bit time. So in summary, tar is a good candidate for enabling LFS and time64 early but some checks should be done first, unless it is known that there are no external interface changes other than to glibc. I hope that is helpful. Wookey -- Principal hats: Debian, Wookware, ARM http://wookware.org/ signature.asc Description: PGP signature
Bug#1026204: tar FTBFS on armel, armhf, i386, hppa, powerpc and sparc64
On Tue, 31 Jan 2023 18:27:27 +0100 Andreas Henriksson wrote: > > export DEB_BUILD_MAINT_OPTIONS = future=+lfs > export DEB_CFLAGS_MAINT_APPEND = -Wall -Wno-analyzer-null-argument Might be useful to add a comment here saying: # Workaround gnulib issue: The below three lines can be dropped once tar >= 1.35 is used. > ifneq ($(shell dpkg-architecture -qDEB_TARGET_ARCH_BITS),64) > export DEB_CPPFLAGS_MAINT_APPEND = -D_TIME_BITS=64 > endif Just for your info: I opened this ticket: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1030159. which asks for a "future=+time64" option, which was accepted. So, in future, just adding "future=+time64" avoids the "ifneq" check above. I assume you also have no idea how to use src:gnulib when building tar (or maybe it's just too complicated change)? (The problem should already be fixed in the version of src:gnulib we have in debian...) I don't know. Sorry. Will you go ahead and upload this Helge? I'd prefer if some "tar" maintainer would do that... Helge
Bug#1026204: tar FTBFS on armel, armhf, i386, hppa, powerpc and sparc64
Hello, On Tue, Jan 31, 2023 at 02:26:50AM +0100, Helge Deller wrote: > Hi David & Andreas, > > On 1/28/23 12:10, David Prévot wrote: > > Le Wed, Jan 18, 2023 at 01:24:37AM +0100, Helge Deller a écrit : > > > On Sat, 14 Jan 2023 20:38:38 +0100 Andreas Henriksson > > > wrote: > > > > Here's a slightly different patch to implement basically the same thing > > > > > > Unfortunately, even if both patches allow me to build tar on i386, they > > also both lead to an FTBFS on amd64. > > That's right. glibc headers used by configure complains then that > _TIME_BITS=64 can only be > used if FILE_OFFSET_BITS=64 is defined too (which isn't the case on amd64 > as it's not needed there). > > So, please use this hunk instead. It compiles for me on amd64 and 32-bit hppa. In my attempt at https://salsa.debian.org/debian/tar/-/commits/wip/bug1026204 I tried to avoid hardcoding assumptions about flags (and rely on gnulib DTRT), but I agree what I came up with is just to convoluted and it's probably a better idea to just use Helges suggestion below. > > export DEB_BUILD_MAINT_OPTIONS = future=+lfs > export DEB_CFLAGS_MAINT_APPEND = -Wall -Wno-analyzer-null-argument Might be useful to add a comment here saying: # Workaround gnulib issue: The below three lines can be dropped once tar >= 1.35 is used. > ifneq ($(shell dpkg-architecture -qDEB_TARGET_ARCH_BITS),64) > export DEB_CPPFLAGS_MAINT_APPEND = -D_TIME_BITS=64 > endif > > DPKG_EXPORT_BUILDFLAGS = 1 > include /usr/share/dpkg/buildflags.mk > --- > Helge I assume you also have no idea how to use src:gnulib when building tar (or maybe it's just too complicated change)? (The problem should already be fixed in the version of src:gnulib we have in debian...) Will you go ahead and upload this Helge? Regards, Andreas Henriksson
Bug#1026204: tar FTBFS on armel, armhf, i386, hppa, powerpc and sparc64
Hi David & Andreas, On 1/28/23 12:10, David Prévot wrote: Le Wed, Jan 18, 2023 at 01:24:37AM +0100, Helge Deller a écrit : On Sat, 14 Jan 2023 20:38:38 +0100 Andreas Henriksson wrote: Here's a slightly different patch to implement basically the same thing Unfortunately, even if both patches allow me to build tar on i386, they also both lead to an FTBFS on amd64. That's right. glibc headers used by configure complains then that _TIME_BITS=64 can only be used if FILE_OFFSET_BITS=64 is defined too (which isn't the case on amd64 as it's not needed there). So, please use this hunk instead. It compiles for me on amd64 and 32-bit hppa. export DEB_BUILD_MAINT_OPTIONS = future=+lfs export DEB_CFLAGS_MAINT_APPEND = -Wall -Wno-analyzer-null-argument ifneq ($(shell dpkg-architecture -qDEB_TARGET_ARCH_BITS),64) export DEB_CPPFLAGS_MAINT_APPEND = -D_TIME_BITS=64 endif DPKG_EXPORT_BUILDFLAGS = 1 include /usr/share/dpkg/buildflags.mk --- Helge
Bug#1026204: tar FTBFS on armel, armhf, i386, hppa, powerpc and sparc64
Hello taffit, On Sat, Jan 28, 2023 at 12:10:53PM +0100, David Prévot wrote: > Hi, > > Le Wed, Jan 18, 2023 at 01:24:37AM +0100, Helge Deller a écrit : > > On Sat, 14 Jan 2023 20:38:38 +0100 Andreas Henriksson > > wrote: > > > Here's a slightly different patch to implement basically the same thing > > > > Yes, I like this patch better too. > > Unfortunately, even if both patches allow me to build tar on i386, they > also both lead to an FTBFS on amd64. Thanks for the feedback. I ran short on time when looking at this and tried to cheap out on just setting `-D_TIME_BITS=64` directly which causes problems. I generally have no clue when it comes to gnulib but I've now made an attempt at backporting the `year2038` gnulib module that upstream has enabled. I've pushed this here: https://salsa.debian.org/debian/tar/-/commits/wip/bug1026204 This time I've build-tested both on a sid amd64 chroot and a sid i386 chroot (on top of amd64). Maybe there's a better/cleaner way of doing this backport. I also have no idea if this impacts the output format of tar in any incompatible way, but hopefully it doesn't since upstream seems to now have it enabled by default in git atleast. Hopefully someone finds this useful... I'm not confident enough to actually upload this myself. Reviews welcome. Also note that the debian gnulib package seems to have the year2038 stuff in largefile.m4 already, so maybe it would be better to try to use that instead of the bundled m4 files in tar?!... (That should hopefully also sort out anyones worry about shipping generated (diff) files without source.) Regards, Andreas Henriksson
Bug#1026204: tar FTBFS on armel, armhf, i386, hppa, powerpc and sparc64
Hi, Le Wed, Jan 18, 2023 at 01:24:37AM +0100, Helge Deller a écrit : > On Sat, 14 Jan 2023 20:38:38 +0100 Andreas Henriksson > wrote: > > Here's a slightly different patch to implement basically the same thing > > Yes, I like this patch better too. Unfortunately, even if both patches allow me to build tar on i386, they also both lead to an FTBFS on amd64. Regards taffit tar_1.34+dfsg-1.2_amd64-2023-01-28T10:47:33Z.build.xz Description: application/xz signature.asc Description: PGP signature
Bug#1026204: tar FTBFS on armel, armhf, i386, hppa, powerpc and sparc64
On Sat, 14 Jan 2023 20:38:38 +0100 Andreas Henriksson wrote: Here's a slightly different patch to implement basically the same thing Yes, I like this patch better too. Helge
Bug#1026204: tar FTBFS on armel, armhf, i386, hppa, powerpc and sparc64
Hello, On Fri, Dec 16, 2022 at 09:13:44AM +0100, Helge Deller wrote: > Package: tar > Version: 1.34+dfsg-1.1 > Tags: hppa, patch, ftbfs, lfs [...] > I've found, that changing the line 12 in debian/rules from > CPPFLAGS = `dpkg-buildflags --get CPPFLAGS` > to: > CPPFLAGS = `dpkg-buildflags --get CPPFLAGS` -D_LARGEFILE_SOURCE > -D_FILE_OFFSET_BITS=64 -D__USE_TIME_BITS64 > does solve the issue. [...] Thanks for identifying the solution already. Here's a slightly different patch to implement basically the same thing (and I can confirm it builds in an i386 chroot which previously failed). (Note: The initial two lines in debian/rules could also be replaced by `include /usr/share/dpkg/architecture.mk` for further slight modernization.) Regards, Andreas Henriksson --- a/debian/rules 2023-01-14 19:20:32.572449385 + +++ b/debian/rules 2023-01-14 19:19:02.745766528 + @@ -6,10 +6,12 @@ CONFARGS = --host=$(DEB_HOST_GNU_TYPE) endif -CFLAGS = `dpkg-buildflags --get CFLAGS` -CFLAGS += -Wall -Wno-analyzer-null-argument -LDFLAGS += `dpkg-buildflags --get LDFLAGS` -CPPFLAGS = `dpkg-buildflags --get CPPFLAGS` +export DEB_BUILD_MAINT_OPTIONS = future=+lfs +export DEB_CFLAGS_MAINT_APPEND = -Wall -Wno-analyzer-null-argument +export DEB_CPPFLAGS_MAINT_APPEND = -D_TIME_BITS=64 + +DPKG_EXPORT_BUILDFLAGS = 1 +include /usr/share/dpkg/buildflags.mk export BUILD_DATE = $(shell dpkg-parsechangelog | sed -n -e 's/^Date: //p')
Bug#1026204: tar FTBFS on armel, armhf, i386, hppa, powerpc and sparc64
Package: tar
Version: 1.34+dfsg-1.1
Tags: hppa, patch, ftbfs, lfs
I wondered why tar now suddenly fails to build on those platforms.
Testcase # 151 does create files whith access dates which are outside of the
reach for
32-bit plaforms with a default of 32-bit timestamps in glibc.
This does succeed on filesystems which support 64-bit timestamps, but
stat'ing this file from 32-bit tar executable fails like this:
+tar: dir/f2038-01-19T03\:14\:08.9: Cannot stat: Value too large for defined
data type
(this message comes from glibc!)
Full log is here:
https://buildd.debian.org/status/fetch.php?pkg=tar=hppa=1.34%2Bdfsg-1.1=1670958554=0
I've found, that changing the line 12 in debian/rules from
CPPFLAGS = `dpkg-buildflags --get CPPFLAGS`
to:
CPPFLAGS = `dpkg-buildflags --get CPPFLAGS` -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64 -D__USE_TIME_BITS64
does solve the issue.
So, either please add this line, or glibc needs fixing to allow
64-bit timestamps (which it probably did in earlier versions, since
otherwise tar would have failed in the past).
Here is a trivial testcase:
1. run:
touch -d 2106-02-07T06:28:15.001 f2106-02-07T06:28:15.001
2. compile and run this program, with and without __USE_TIME_BITS64 defined:
-
#define _FILE_OFFSET_BITS 64
#define _LARGEFILE_SOURCE
// #define __USE_TIME_BITS64
#include
#include
#include
#include
// run before starting this program: touch -d 2106-02-07T06:28:15.001
f2106-02-07T06:28:15.001
int main(void)
{
struct stat statbuf = { 0 };
int ret;
ret = fstatat(AT_FDCWD, "f2106-02-07T06:28:15.001", ,
AT_SYMLINK_NOFOLLOW);
fprintf(stderr, "return code %d errno %d st_atim.tv_sec %llx\n", ret,
errno, statbuf.st_atim.tv_sec);
return ret;
}
-
