Bug#1086769: uid-wrapper FTBFS on hppa, powerpc and x32
Thanks for analysis -- my latest perception is that tests_syscall.c is broken on compilers that default tv_sec to 64-bit since it invokes a 32-bit syscall and compares the results. I've modified debian/tests/tests to build and run the test suite twice in two variants: 1) build using system compiler but disable test_syscall, and 2) build using system compiler with -U's to disable 64-bit time_t, so that the test suite should work on all architectures. Uploading, let's hope we are getting further this time... /Simon signature.asc Description: PGP signature
Bug#1086769: uid-wrapper FTBFS on hppa, powerpc and x32
On 11/6/24 17:00, Simon Josefsson wrote:
Helge Deller writes:
Hi Simon,
On 11/6/24 09:26, Simon Josefsson wrote:
Hi. The uid-wrapper autopkgtest fails on armel and armhf:
https://tracker.debian.org/pkg/uid-wrapper
https://ci.debian.net/packages/u/uid-wrapper/testing/armel/54125843/
https://ci.debian.net/packages/u/uid-wrapper/testing/armhf/54125865/
The C flags are hidden, and I am working on unhiding them via
-DCMAKE_VERBOSE_MAKEFILE=ON but it doesn't look like the self-tests are
built with -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64, unless that comes
from config.h: https://salsa.debian.org/jas/uid-wrapper/-/jobs/6543041
The failing code here is uid-wrapper/tests/test_syscall.c line 53:
assert_int_equal(tv1.tv_sec, tv2.tv_sec);
Maybe the previous arm32 patch is still relevant?
No, the previous arm32 patch was just a hack and is completely wrong.
This is the error reported:
[ ERROR ] --- 0x672a9fe1 != 0xf2839672a9fe1
0xf2839672a9fe1 is the value which is stored in tv2.tv_sec.
That number is a 64-bit value (you need more than 32bits to store that value).
So, "-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64" was definitively set in your
build,
otherwise tv2.tv_sec would not be 64-bits wide.
I think time_t and timeval's tv_sec are 64-bit on 32-bit architectures
now: https://wiki.debian.org/ReleaseGoals/64bit-time
The following may confirms this, although amdahl is running a 64-bit
kernel so maybe this isn't correct anyway. Is there any real 32-bit
armel/armhf porter box to test things on?
jas@amdahl:~$ sessionid=$(schroot -b -c sid_armel-dchroot)
I: 00check: Untarring chroot environment. This might take a minute or two.
I: 99porterbox-extra-sources: o To install build dependencies run
I: 99porterbox-extra-sources: dd-schroot-cmd -c
sid_armel-dchroot-6d957f01-d2d5-424e-8367-94b5ba7473eb apt-get update
I: 99porterbox-extra-sources: followed by build-dep/install as appropriate in
the host system.
I: 99porterbox-extra-sources: o If you started this session with schroot -b,
please do not forget to run
I: 99porterbox-extra-sources: schroot --end-session -c
sid_armel-dchroot-6d957f01-d2d5-424e-8367-94b5ba7473eb
I: 99porterbox-extra-sources: when you no longer need this environment.
jas@amdahl:~$ schroot -r -c $sessionid
(sid_armel-dchroot)jas@amdahl:~$ uname -a
Linux amdahl 6.1.0-26-arm64 #1 SMP Debian 6.1.112-1 (2024-09-30) armv8l
GNU/Linux
(sid_armel-dchroot)jas@amdahl:~$ cat>foo.c
#include
#include
#include
int main (void) {
struct timeval tv;
printf ("size %d %d\n", sizeof (tv.tv_sec), sizeof (time_t));
}
(sid_armel-dchroot)jas@amdahl:~$ cc -o foo foo.c
(sid_armel-dchroot)jas@amdahl:~$ ./foo
size 8 8
(sid_armel-dchroot)jas@amdahl:~$
I agree, I see size=8 as well. This comes from here:
/usr/include/arm-linux-gnueabihf/bits/types/struct_timeval.h
struct timeval
{
#ifdef __USE_TIME64_REDIRECTS
__time64_t tv_sec;/* Seconds. */
__suseconds64_t tv_usec; /* Microseconds. */
#else
__time_t tv_sec; /* Seconds. */
__suseconds_t tv_usec;/* Microseconds. */
#endif
};
So, when using "cc -o foo foo.c", __USE_TIME64_REDIRECTS is somehow
automatically set.
Running "cc --dumpspecs" shows that there are default values:
*distro_defaults_cpp:
%{!m16:%{!m64:%{!D_DISTRO_EVADE_TIME_BITS:%{!D_TIME_BITS=*:%{!U_TIME_BITS:-D_TIME_BITS=64%{!D_FILE_OFFSET_BITS=*:%{!U_FILE_OFFSET_BITS:
-D_FILE_OFFSET_BITS=64}}}
So, you need to use "-U" to undefine the defines and try again.
gcc -U_TIME_BITS -U_FILE_OFFSET_BITS foo.c
Note, that "DEB_BUILD_MAINT_OPTIONS = abi=-time64" does exactly this.
Is the problem that the SYS_gettimeofday call assumes 32-bit int's, but
is passed and compared with a 64-bit struct, which causes the problem?
Yes.
As discussed in
https://gitlab.com/cwrap/uid_wrapper/-/issues/1 one fix would be to use
a SYS_gettimeofday64 call instead, but that doesn't seem to exist.
Right. There is no such syscall.
Instead glibc uses clock_gettime() to provide a 64-bit time on 32-bit platforms.
So, there is no need for uid-wrapper to emulate such a call.
Okay, that also suggests test_syscall.c shouldn't use gettimeofday() for
test purposes since it is not trivial.
Yes, maybe.
Helge
Bug#1086769: uid-wrapper FTBFS on hppa, powerpc and x32
Helge Deller writes:
> Hi Simon,
>
> On 11/6/24 09:26, Simon Josefsson wrote:
>> Hi. The uid-wrapper autopkgtest fails on armel and armhf:
>>
>> https://tracker.debian.org/pkg/uid-wrapper
>> https://ci.debian.net/packages/u/uid-wrapper/testing/armel/54125843/
>> https://ci.debian.net/packages/u/uid-wrapper/testing/armhf/54125865/
>>
>> The C flags are hidden, and I am working on unhiding them via
>> -DCMAKE_VERBOSE_MAKEFILE=ON but it doesn't look like the self-tests are
>> built with -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64, unless that comes
>> from config.h: https://salsa.debian.org/jas/uid-wrapper/-/jobs/6543041
>>
>> The failing code here is uid-wrapper/tests/test_syscall.c line 53:
>>
>> assert_int_equal(tv1.tv_sec, tv2.tv_sec);
>>
>> Maybe the previous arm32 patch is still relevant?
>
> No, the previous arm32 patch was just a hack and is completely wrong.
>
> This is the error reported:
> [ ERROR ] --- 0x672a9fe1 != 0xf2839672a9fe1
> 0xf2839672a9fe1 is the value which is stored in tv2.tv_sec.
> That number is a 64-bit value (you need more than 32bits to store that value).
> So, "-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64" was definitively set in your
> build,
> otherwise tv2.tv_sec would not be 64-bits wide.
I think time_t and timeval's tv_sec are 64-bit on 32-bit architectures
now: https://wiki.debian.org/ReleaseGoals/64bit-time
The following may confirms this, although amdahl is running a 64-bit
kernel so maybe this isn't correct anyway. Is there any real 32-bit
armel/armhf porter box to test things on?
jas@amdahl:~$ sessionid=$(schroot -b -c sid_armel-dchroot)
I: 00check: Untarring chroot environment. This might take a minute or two.
I: 99porterbox-extra-sources: o To install build dependencies run
I: 99porterbox-extra-sources: dd-schroot-cmd -c
sid_armel-dchroot-6d957f01-d2d5-424e-8367-94b5ba7473eb apt-get update
I: 99porterbox-extra-sources: followed by build-dep/install as appropriate in
the host system.
I: 99porterbox-extra-sources: o If you started this session with schroot -b,
please do not forget to run
I: 99porterbox-extra-sources: schroot --end-session -c
sid_armel-dchroot-6d957f01-d2d5-424e-8367-94b5ba7473eb
I: 99porterbox-extra-sources: when you no longer need this environment.
jas@amdahl:~$ schroot -r -c $sessionid
(sid_armel-dchroot)jas@amdahl:~$ uname -a
Linux amdahl 6.1.0-26-arm64 #1 SMP Debian 6.1.112-1 (2024-09-30) armv8l
GNU/Linux
(sid_armel-dchroot)jas@amdahl:~$ cat>foo.c
#include
#include
#include
int main (void) {
struct timeval tv;
printf ("size %d %d\n", sizeof (tv.tv_sec), sizeof (time_t));
}
(sid_armel-dchroot)jas@amdahl:~$ cc -o foo foo.c
(sid_armel-dchroot)jas@amdahl:~$ ./foo
size 8 8
(sid_armel-dchroot)jas@amdahl:~$
Is the problem that the SYS_gettimeofday call assumes 32-bit int's, but
is passed and compared with a 64-bit struct, which causes the problem?
It should have used a SYS_gettimeofday64 syscall to use 64-bit int's but
there is no such syscall. There are for many other 64-bit syscalls like
SYS_fstat64 and SYS_clock_gettime64 but nothing for SYS_gettimeofday64.
>> uid-wrapper generally: the self test checks gettimeofday vs
>> SYS_gettimeofday, but uid-wrapper never does anything related to
>> gettimeofday.
>
> Right.
Perhaps we can get upstream to use another syscall to test things here.
Is there some harmless syscall we could use instead? That doesn't have
32-bit vs 64-bit alignment issues...
>> As discussed in
>> https://gitlab.com/cwrap/uid_wrapper/-/issues/1 one fix would be to use
>> a SYS_gettimeofday64 call instead, but that doesn't seem to exist.
>
> Right. There is no such syscall.
> Instead glibc uses clock_gettime() to provide a 64-bit time on 32-bit
> platforms.
> So, there is no need for uid-wrapper to emulate such a call.
Okay, that also suggests test_syscall.c shouldn't use gettimeofday() for
test purposes since it is not trivial.
/Simon
signature.asc
Description: PGP signature
Bug#1086769: uid-wrapper FTBFS on hppa, powerpc and x32
Hi Simon, On 11/6/24 09:26, Simon Josefsson wrote: Hi. The uid-wrapper autopkgtest fails on armel and armhf: https://tracker.debian.org/pkg/uid-wrapper https://ci.debian.net/packages/u/uid-wrapper/testing/armel/54125843/ https://ci.debian.net/packages/u/uid-wrapper/testing/armhf/54125865/ The C flags are hidden, and I am working on unhiding them via -DCMAKE_VERBOSE_MAKEFILE=ON but it doesn't look like the self-tests are built with -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64, unless that comes from config.h: https://salsa.debian.org/jas/uid-wrapper/-/jobs/6543041 The failing code here is uid-wrapper/tests/test_syscall.c line 53: assert_int_equal(tv1.tv_sec, tv2.tv_sec); Maybe the previous arm32 patch is still relevant? No, the previous arm32 patch was just a hack and is completely wrong. This is the error reported: [ ERROR ] --- 0x672a9fe1 != 0xf2839672a9fe1 0xf2839672a9fe1 is the value which is stored in tv2.tv_sec. That number is a 64-bit value (you need more than 32bits to store that value). So, "-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64" was definitively set in your build, otherwise tv2.tv_sec would not be 64-bits wide. Reading that self-test makes me wonder if it really support -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64. The code is low-level so shouldn't it have internal magic for the 32-bit vs 64-bit case? This library should do no magic. It should simply emulate the original syscalls correctly. uid-wrapper generally: the self test checks gettimeofday vs SYS_gettimeofday, but uid-wrapper never does anything related to gettimeofday. Right. As discussed in https://gitlab.com/cwrap/uid_wrapper/-/issues/1 one fix would be to use a SYS_gettimeofday64 call instead, but that doesn't seem to exist. Right. There is no such syscall. Instead glibc uses clock_gettime() to provide a 64-bit time on 32-bit platforms. So, there is no need for uid-wrapper to emulate such a call. Maybe the purpose of this self-test is to merely test the syscall facility, and that they happened to pick gettimeofday which used to be an uncomplicated interface but lately haven't been? Yes, I think so. As an example, the x32 build currently fails because the syscall isn't executed. Thoughts? Maybe just add back Draw's patch? No. Helge
Bug#1086769: uid-wrapper FTBFS on hppa, powerpc and x32
Hi. The uid-wrapper autopkgtest fails on armel and armhf:
https://tracker.debian.org/pkg/uid-wrapper
https://ci.debian.net/packages/u/uid-wrapper/testing/armel/54125843/
https://ci.debian.net/packages/u/uid-wrapper/testing/armhf/54125865/
The C flags are hidden, and I am working on unhiding them via
-DCMAKE_VERBOSE_MAKEFILE=ON but it doesn't look like the self-tests are
built with -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64, unless that comes
from config.h: https://salsa.debian.org/jas/uid-wrapper/-/jobs/6543041
The failing code here is uid-wrapper/tests/test_syscall.c line 53:
assert_int_equal(tv1.tv_sec, tv2.tv_sec);
Maybe the previous arm32 patch is still relevant?
Reading that self-test makes me wonder if it really support
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64. The code is low-level so
shouldn't it have internal magic for the 32-bit vs 64-bit case?
I also wonder what relevance that particular self-test has with
uid-wrapper generally: the self test checks gettimeofday vs
SYS_gettimeofday, but uid-wrapper never does anything related to
gettimeofday. As discussed in
https://gitlab.com/cwrap/uid_wrapper/-/issues/1 one fix would be to use
a SYS_gettimeofday64 call instead, but that doesn't seem to exist.
Maybe the purpose of this self-test is to merely test the syscall
facility, and that they happened to pick gettimeofday which used to be
an uncomplicated interface but lately haven't been?
Thoughts? Maybe just add back Draw's patch?
The entire code is fairly simple, simplified:
static void test_uwrap_syscall(void **state)
{
long int rc;
struct timeval tv1, tv2;
struct timezone tz1, tz2;
(void) state; /* unused */
rc = syscall(SYS_getpid);
assert_int_equal(rc, getpid());
ZERO_STRUCT(tv1);
ZERO_STRUCT(tv2);
ZERO_STRUCT(tz1);
ZERO_STRUCT(tz2);
rc = gettimeofday(&tv1, &tz1);
assert_int_equal(rc, 0);
rc = syscall(SYS_gettimeofday, &tv2, &tz2);
assert_int_equal(rc, 0);
assert_int_equal(tz1.tz_dsttime, tz2.tz_dsttime);
assert_int_equal(tz1.tz_minuteswest, tz2.tz_minuteswest);
assert_int_equal(tv1.tv_sec, tv2.tv_sec);
}
/Simon
105s 15/33 Test #15: test_syscall .***Failed0.00 sec
105s [==] uwrap_tests: Running 1 test(s).
105s [ RUN ] test_uwrap_syscall
105s [ ERROR ] --- 0x672a9fe1 != 0xf2839672a9fe1
105s [ LINE ] ---
/tmp/autopkgtest-lxc._gpjzze_/downtmp/autopkgtest_tmp/tests/test_syscall.c:53:
error: Failure!
105s [ FAILED ] test_uwrap_syscall
105s [==] uwrap_tests: 1 test(s) run.
105s [ PASSED ] 0 test(s).
105s [ FAILED ] uwrap_tests: 1 test(s), listed below:
105s [ FAILED ] test_uwrap_syscall
105s
105s 1 FAILED TEST(S)
signature.asc
Description: PGP signature
Bug#1086769: uid-wrapper FTBFS on hppa, powerpc and x32
Looks like this solved the problem on hppa and powerpc but not x32: https://buildd.debian.org/status/package.php?p=uid-wrapper https://buildd.debian.org/status/fetch.php?pkg=uid-wrapper&arch=x32&ver=1.3.1-2&stamp=1730833724&raw=0 Any ideas? However x32 is not a release arch, and the package now builds including self-tests on armel+armhf again, so I think we should be fairly happy. /Simon 15: Test command: /<>/obj-x86_64-linux-gnux32/tests/test_syscall 15: Working Directory: /<>/obj-x86_64-linux-gnux32/tests 15: Environment variables: 15: LD_PRELOAD=/<>/obj-x86_64-linux-gnux32/tests/libuwrap_fake_socket_wrapper.so:/<>/obj-x86_64-linux-gnux32/src/libuid_wrapper.so 15: UID_WRAPPER=1 15: UID_WRAPPER_ROOT=1 15: Test timeout computed to be: 1500 14: [==] uwrap_tests: Running 2 test(s). 14: [ RUN ] test_uwrap_getgroups 14: [ OK ] test_uwrap_getgroups 14: [ RUN ] test_uwrap_setgroups 14: [ OK ] test_uwrap_setgroups 14: [==] uwrap_tests: 2 test(s) run. 14: [ PASSED ] 2 test(s). 15: [==] uwrap_tests: Running 1 test(s). 15: [ RUN ] test_uwrap_syscall 15: [ ERROR ] --- 0x != 0x226e57 15: [ LINE ] --- ./tests/test_syscall.c:34: error: Failure! 15: [ FAILED ] test_uwrap_syscall 15: [==] uwrap_tests: 1 test(s) run. 15: [ PASSED ] 0 test(s). 15: [ FAILED ] uwrap_tests: 1 test(s), listed below: 15: [ FAILED ] test_uwrap_syscall 15: 15: 1 FAILED TEST(S) 14/26 Test #14: test_setgroups ... Passed0.00 sec 15/26 Test #15: test_syscall .***Failed0.00 sec [==] uwrap_tests: Running 1 test(s). [ RUN ] test_uwrap_syscall [ ERROR ] --- 0x != 0x226e57 [ LINE ] --- ./tests/test_syscall.c:34: error: Failure! [ FAILED ] test_uwrap_syscall [==] uwrap_tests: 1 test(s) run. [ PASSED ] 0 test(s). [ FAILED ] uwrap_tests: 1 test(s), listed below: [ FAILED ] test_uwrap_syscall 1 FAILED TEST(S) test 16 Start 16: test_syscall_swrap signature.asc Description: PGP signature
Bug#1086769: [Pkg-sssd-devel] Bug#1086769: uid-wrapper FTBFS on hppa, powerpc and x32
Helge Deller writes: > -export DEB_BUILD_MAINT_OPTIONS = hardening=+all > +export DEB_BUILD_MAINT_OPTIONS = hardening=+all abi=-time64 Thank you for figuring this out! I agree with your analysis and suggested remedy. Hopefully this can help on socket-wrapper and priv-wrapper too. I don't feel confident that this is the entire story to this set of *-wrapper FTBFS bugs, but I'll upload uid-wrapper first to test the hypothesis. /Simon signature.asc Description: PGP signature
Bug#1086769: uid-wrapper FTBFS on hppa, powerpc and x32
Source: uid-wrapper
Version: 1.3.1-1
Tags: ftbfs
The test15 (test_uwrap_syscall) fails on some 32-bit platforms due to the
time64 transition like this:
15: [ RUN ] test_uwrap_syscall
15: [ ERROR ] --- 0x66234648 != 0x42c0c66234648
15: [ LINE ] --- ./tests/test_syscall.c:53: error: Failure!
15: [ FAILED ] test_uwrap_syscall
Because of the transition, "-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64"
gets added and as such the "struct timeval tv2" gets extended to 64bit
which does not fit the struct layout (and size (!)) which the syscall expects.
Because of that, the syscall returns values in memory locations which
the userspace application doesn't expect. This shows up as either zero
values, or like above as 0x66234648 != 0x42c0c66234648.
The time64 transition is enabled in via /usr/share/perl5/Dpkg/Vendor/Debian.pm,
line ~303:
# In Debian gcc enables time64 (and lfs) for the following architectures
# by injecting pre-processor flags, though the libc ABI has not changed.
if (any { $arch eq $_ } qw(armel armhf hppa m68k mips mipsel powerpc sh4)) {
$flags->set_option_value('cc-abi-time64', 1);
The uid-wrapper is a very low-level package and as such it should not
be built with time64 enabled (same as glibc is built without it).
The obvious fix for this package is the trivial patch below.
I tried it on the hppa architecture, but it should fix the issues
on the other arches as well.
This patch has no impact on 64-bit architectures!
diff -up ./debian/rules.bak ./debian/rules
--- ./debian/rules.bak 2024-11-05 16:53:45.182697375 +
+++ ./debian/rules 2024-11-05 16:04:57.142488073 +
@@ -1,6 +1,6 @@
#!/usr/bin/make -f
-export DEB_BUILD_MAINT_OPTIONS = hardening=+all
+export DEB_BUILD_MAINT_OPTIONS = hardening=+all abi=-time64
Please note, that I suggest to remove the skip-arm-32bit-syscall.patch.
PS: This enhances & fixes the findings in bug 1069425
