Bug#1067639: sasl2-bin: terminates with smashed stack and kills qemu-user?!
Bernhard Übelacker dixit:
> On Thu, 4 Apr 2024 21:00:59 + (UTC) Thorsten Glaser
> wrote:
>> Sometimes, it does not crash with a smashed stack but instead:
>>
>> Setting up sasl2-bin (2.1.28+dfsg1-6+b1) ...
>> BDB0002 __fop_file_setup: Retry limit (100) exceeded
>> saslpasswd2: generic failure
>
> This looks to be a result of the pre-existing /etc/__db.sasldb2.
> If this file gets removed the stack smashing occurs again.
Right, I got there as well but not any further.
> By some experimenting I could convince gdb to load the debug symbols.
Massive detective work, thanks!
> And the stack seems to point into function __os_unique_id from libdb-5.3.so.
>
> Unfortunately I am not sure where the canary gets overwritten.
I had an immediate hunch as I saw this:
> 38 __os_gettime(env, , 1);
And:
> (gdb) ptype /o v
> type = struct {
> /* 0 | 8 */time_t tv_sec;
> /* 8 | 4 */long tv_nsec;
>
> /* total size (bytes): 12 */
> }
This is, in the source:
typedef struct {
time_t tv_sec; /* seconds */
#ifdef HAVE_MIXED_SIZE_ADDRESSING
int32_t tv_nsec;
#else
longtv_nsec;/* nanoseconds */
#endif
} db_timespec;
Compare the newer system header:
struct timespec
{
#ifdef __USE_TIME_BITS64
__time64_t tv_sec;/* Seconds. */
#else
__time_t tv_sec; /* Seconds. */
#endif
#if __WORDSIZE == 64 \
|| (defined __SYSCALL_WORDSIZE && __SYSCALL_WORDSIZE == 64) \
|| (__TIMESIZE == 32 && !defined __USE_TIME_BITS64)
__syscall_slong_t tv_nsec;/* Nanoseconds. */
#else
# if __BYTE_ORDER == __BIG_ENDIAN
int: 32; /* Padding. */
long int tv_nsec; /* Nanoseconds. */
# else
long int tv_nsec; /* Nanoseconds. */
int: 32; /* Padding. */
# endif
#endif
};
This is actually longer and (IMHO) really stupid. But Linux has:
struct __kernel_timespec {
__kernel_time64_t tv_sec; /* seconds */
long long tv_nsec;/* nanoseconds */
};
So this is actually expected. *checks POSIX* which says:
| The header shall declare the timespec structure, which shall
| include at least the following members:
|
| time_t tv_sec Whole seconds.
| long tv_nsec Nanoseconds [0, 999 999 999].
So both the kernel definition (tv_nsec must be long, not long long,
which is incompatible on ILP32 big endian platforms) and the one by
db5.3 (struct timespec may include extra members and be in any order)
actually violate POSIX… *sigh*
And yes, it does cast to struct timespec and passes it
to clock_gettime().
But it does give us a possible fix, which I’ll be testing.
bye,
//mirabilos
--
22:20⎜ The crazy that persists in his craziness becomes a master
22:21⎜ And the distance between the craziness and geniality is
only measured by the success 18:35⎜ "Psychotics are consistently
inconsistent. The essence of sanity is to be inconsistently inconsistent
Bug#1067639: sasl2-bin: terminates with smashed stack and kills qemu-user?!
Hello, I tried to find some more information, with the help of a prebuilt full-system VM image. On Thu, 4 Apr 2024 21:00:59 + (UTC) Thorsten Glaser wrote: Sometimes, it does not crash with a smashed stack but instead: Setting up sasl2-bin (2.1.28+dfsg1-6+b1) ... BDB0002 __fop_file_setup: Retry limit (100) exceeded saslpasswd2: generic failure This looks to be a result of the pre-existing /etc/__db.sasldb2. If this file gets removed the stack smashing occurs again. By some experimenting I could convince gdb to load the debug symbols. And the stack seems to point into function __os_unique_id from libdb-5.3.so. Unfortunately I am not sure where the canary gets overwritten. Kind regards, Bernhard https://people.debian.org/~gio/dqib/ https://gitlab.com/giomasce/dqib/-/artifacts https://gitlab.com/giomasce/dqib/-/jobs/6565595565/artifacts/download?file_type=archive apt install gdb sasl2-bin sasl2-bin-dbgsym libsasl2-2-dbgsym libsasl2-modules-db-dbgsym apt install libc6-dbg libc6-dbgsym db-util db5.3-util libldap-2.5-0 libldap-common libsasl2-2 libsasl2-2-dbgsym libsasl2-modules libsasl2-modules-db export DEBUGINFOD_URLS="https://debuginfod.debian.net; rm /etc/__db.sasldb2 echo -e "test\ntest" > exclam gdb -q file /usr/sbin/saslpasswd2 run -c 'no:such:user' https://debuginfod.debian.net> Enable debuginfod for this session? (y or [n]) y Debuginfod has been enabled. To make this setting permanent, add 'set debuginfod enabled on' to .gdbinit. Downloading separate debug info for /usr/sbin/saslpasswd2 (No debugging symbols found in /usr/sbin/saslpasswd2) (gdb) run -c 'no:such:user' , fname=0xc0605cb9 "/etc/sasldb2", dname=0x0, type=DB_HASH, flags=1, mode=432) at ../src/db/db_iface.c:1193 #13 0xc0604248 in ?? () #14 0xd00087b0 in ?? () #15 0x in ?? () (gdb) shell objdump --all-headers /usr/lib/m68k-linux-gnu/sasl2/libsasldb.so.2.0.25 | grep .text 11 .text 27f0 138c 138c 138c 2**2 (gdb) print/x 0x138c + 0xc0602000 $4 = 0xc060338c (gdb) add-symbol-file /usr/lib/m68k-linux-gnu/sasl2/libsasldb.so.2.0.25 0xc060338c add symbol table from file "/usr/lib/m68k-linux-gnu/sasl2/libsasldb.so.2.0.25" at .text_addr = 0xc060338c (y or n) y Reading symbols from /usr/lib/m68k-linux-gnu/sasl2/libsasldb.so.2.0.25... Reading symbols from /usr/lib/debug/.build-id/29/c8e688eb61b57bcd21794b5403feefe1272dfd.debug... (gdb) bt ... #15 0xc0603572 in sasldb_auxprop_store (glob_context=0x0, sparams=0xd00077b8, ctx=0xd0007a58, user=0xeed9 "no:such:user", ulen=12) at ../../plugins/sasldb.c:258 #16 0xc002d26c in ?? () #17 0x in ?? () (gdb) shell cat /proc/10276/maps | grep -i -E "^c00" c000-c002 r-xp 08:01 535730 /usr/lib/m68k-linux-gnu/ld.so.1 c002-c0021000 rw-p 00:00 0 c0021000-c0022000 r--p 00021000 08:01 535730 /usr/lib/m68k-linux-gnu/ld.so.1 c0022000-c0024000 rw-p 00022000 08:01 535730 /usr/lib/m68k-linux-gnu/ld.so.1 c0028000-c003c000 r-xp 08:01 539155 /usr/lib/m68k-linux-gnu/libsasl2.so.2.0.25 c003c000-c003d000 ---p 00014000 08:01 539155 /usr/lib/m68k-linux-gnu/libsasl2.so.2.0.25 c003d000-c003e000 r--p 00015000 08:01 539155 /usr/lib/m68k-linux-gnu/libsasl2.so.2.0.25 c003e000-c003f000 rw-p 00016000 08:01 539155 /usr/lib/m68k-linux-gnu/libsasl2.so.2.0.25 c004-c01b1000 r-xp 08:01 535733 /usr/lib/m68k-linux-gnu/libc.so.6 (gdb) shell objdump --all-headers /usr/lib/m68k-linux-gnu/libsasl2.so.2.0.25 | grep .text 12 .text e284 3db0 3db0 3db0 2**2 (gdb) print/x 0x3db0 + 0xc0028000 $5 = 0xc002bdb0 (gdb) add-symbol-file /usr/lib/m68k-linux-gnu/libsasl2.so.2.0.25 0xc002bdb0 add symbol table from file "/usr/lib/m68k-linux-gnu/libsasl2.so.2.0.25" at .text_addr = 0xc002bdb0 (y or n) y Reading symbols from /usr/lib/m68k-linux-gnu/libsasl2.so.2.0.25... Reading symbols from /usr/lib/debug/.build-id/0f/8954c0644d1a9efec7973fb3198b8fd7649d5f.debug... (gdb) set width 0 (gdb) set pagination off (gdb) bt ... #17 0xc00366dc in sasl_setpass (conn=0xd0006670, user=0xeed9 "no:such:user", pass=0xd0006608 "test\ntest", passlen=9, oldpass=0x0, oldpasslen=0, flags=1) at ../../lib/server.c:186 #18 0xd0001534 in ?? () ... (gdb) shell cat /proc/10276/maps | grep -i -E "^d00" d000-d0002000 r-xp 08:01 539212 /usr/sbin/saslpasswd2 d0003000-d0004000 r--p 3000 08:01 539212 /usr/sbin/saslpasswd2 d0004000-d0005000 rw-p 4000 08:01 539212 /usr/sbin/saslpasswd2 d0005000-d0026000 rwxp 00:00 0 [heap] (gdb) shell objdump --all-headers /usr/sbin/saslpasswd2 | grep .text 13 .text 0950 10b8 10b8 10b8 2**2 (gdb) print/x 0x10b8 + 0xd000 $6 = 0xd00010b8 (gdb) add-symbol-file /usr/sbin/saslpasswd2 0xd00010b8 add symbol table from file "/usr/sbin/saslpasswd2" at .text_addr = 0xd00010b8 (y or n) y Reading symbols from /usr/sbin/saslpasswd2... Reading symbols from
Bug#1067639: sasl2-bin: terminates with smashed stack and kills qemu-user?!
Sometimes, it does not crash with a smashed stack but instead: Setting up sasl2-bin (2.1.28+dfsg1-6+b1) ... BDB0002 __fop_file_setup: Retry limit (100) exceeded saslpasswd2: generic failure dpkg: error processing package sasl2-bin (--configure): installed sasl2-bin package post-installation script subprocess returned error exit status 1 (I tried rebuilding it, but that didn’t fix it either.)
Bug#1067639: sasl2-bin: terminates with smashed stack and kills qemu-user?!
Dixi quod…
>OK, it’s not qemu. On ARAnyM (Atari):
I was able to strace this:
(pbuild-31733)root@ara2:/# echo '!' | strace -f saslpasswd2 -c 'no:such:user'
execve("/usr/sbin/saslpasswd2", ["saslpasswd2", "-c", "no:such:user"],
0xefd2a90c /* 52 vars */) = 0
brk(NULL) = 0xd0005000
openat(AT_FDCWD, "/usr/lib/libeatmydata/libeatmydata.so",
O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
statx(AT_FDCWD, "/usr/lib/libeatmydata", AT_STATX_SYNC_AS_STAT|AT_NO_AUTOMOUNT,
STATX_BASIC_STATS, 0xef935c28) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_NO_AUTOMOUNT|AT_EMPTY_PATH,
STATX_BASIC_STATS, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_attributes=0,
stx_mode=S_IFREG|0644, stx_size=6940, ...}) = 0
mmap2(NULL, 6940, PROT_READ, MAP_PRIVATE, 3, 0) = 0xc0024000
close(3)= 0
openat(AT_FDCWD, "/lib/m68k-linux-gnu/libeatmydata.so",
O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
read(3, "\177ELF\1\2\1\0\0\0\0\0\0\0\0\0\0\3\0\4\0\0\0\1\0\0\0\0\0\0\0004"...,
512) = 512
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_NO_AUTOMOUNT|AT_EMPTY_PATH,
STATX_BASIC_STATS, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_attributes=0,
stx_mode=S_IFREG|0644, stx_size=9460, ...}) = 0
mmap2(NULL, 24584, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xc0026000
mmap2(0xc0026000, 16392, PROT_READ|PROT_EXEC,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xc0026000
munmap(0xc002b000, 4104)= 0
mprotect(0xc0027000, 8192, PROT_NONE) = 0
mmap2(0xc0029000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0xc0029000
close(3)= 0
openat(AT_FDCWD, "/usr/lib/cowdancer/libcowdancer.so",
O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
read(3, "\177ELF\1\2\1\0\0\0\0\0\0\0\0\0\0\3\0\4\0\0\0\1\0\0\34\4\0\0\0004"...,
512) = 512
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_NO_AUTOMOUNT|AT_EMPTY_PATH,
STATX_BASIC_STATS, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_attributes=0,
stx_mode=S_IFREG|0644, stx_size=25936, ...}) = 0
mmap2(NULL, 41044, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xc002b000
mmap2(0xc002c000, 32852, PROT_READ|PROT_EXEC,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xc002c000
munmap(0xc002b000, 4096)= 0
munmap(0xc0035000, 84) = 0
mprotect(0xc0031000, 8192, PROT_NONE) = 0
mmap2(0xc0033000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x5000) = 0xc0033000
close(3)= 0
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/lib/m68k-linux-gnu/libsasl2.so.2",
O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
read(3, "\177ELF\1\2\1\0\0\0\0\0\0\0\0\0\0\3\0\4\0\0\0\1\0\0\0\0\0\0\0004"...,
512) = 512
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_NO_AUTOMOUNT|AT_EMPTY_PATH,
STATX_BASIC_STATS, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_attributes=0,
stx_mode=S_IFREG|0644, stx_size=91752, ...}) = 0
mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0xc0035000
mmap2(NULL, 98724, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xc0037000
mmap2(0xc0038000, 90532, PROT_READ|PROT_EXEC,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xc0038000
munmap(0xc0037000, 4096)= 0
munmap(0xc004f000, 420) = 0
mprotect(0xc004c000, 4096, PROT_NONE) = 0
mmap2(0xc004d000, 8192, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x15000) = 0xc004d000
close(3)= 0
openat(AT_FDCWD, "/lib/m68k-linux-gnu/libc.so.6",
O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
read(3,
"\177ELF\1\2\1\0\0\0\0\0\0\0\0\0\0\3\0\4\0\0\0\1\0\2\320\210\0\0\0004"..., 512)
= 512
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_NO_AUTOMOUNT|AT_EMPTY_PATH,
STATX_BASIC_STATS, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_attributes=0,
stx_mode=S_IFREG|0755, stx_size=1535504, ...}) = 0
mmap2(NULL, 1585296, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xc004f000
mmap2(0xc005, 1577104, PROT_READ|PROT_EXEC,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xc005
munmap(0xc004f000, 4096)= 0
munmap(0xc01d2000, 144) = 0
mprotect(0xc01c1000, 4096, PROT_NONE) = 0
mmap2(0xc01c2000, 24576, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17) = 0xc01c2000
mmap2(0xc01c8000, 37008, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xc01c8000
close(3)= 0
openat(AT_FDCWD, "/lib/m68k-linux-gnu/libdl.so.2",
O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
read(3, "\177ELF\1\2\1\0\0\0\0\0\0\0\0\0\0\3\0\4\0\0\0\1\0\0\0\0\0\0\0004"...,
512) = 512
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_NO_AUTOMOUNT|AT_EMPTY_PATH,
STATX_BASIC_STATS, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_attributes=0,
stx_mode=S_IFREG|0644, stx_size=9528, ...}) = 0
mmap2(NULL, 24636, PROT_NONE,
Bug#1067639: sasl2-bin: terminates with smashed stack and kills qemu-user?!
Dixi quod… >The openldap build on an m68k qemu-user buildd cannot install sasl2-bin in the >chroot: OK, it’s not qemu. On ARAnyM (Atari): […] Setting up libldap-2.5-0:m68k (2.5.16+dfsg-2+b1) ... Setting up sasl2-bin (2.1.28+dfsg1-5) ... *** stack smashing detected ***: terminated Aborted dpkg: error processing package sasl2-bin (--configure): installed sasl2-bin package post-installation script subprocess returned error exit status 134 Processing triggers for libc-bin (2.37-15.1+b1) ... Processing triggers for man-db (2.12.0-3+b2) ... Not building database; man-db/auto-update is not 'true'. Errors were encountered while processing: sasl2-bin E: Sub-process /usr/bin/dpkg returned an error code (1) bye, //mirabilos -- 22:20⎜ The crazy that persists in his craziness becomes a master 22:21⎜ And the distance between the craziness and geniality is only measured by the success 18:35⎜ "Psychotics are consistently inconsistent. The essence of sanity is to be inconsistently inconsistent
Bug#1067639: sasl2-bin: terminates with smashed stack and kills qemu-user?!
Package: sasl2-bin Version: 2.1.28+dfsg1-5 X-Debbugs-Cc: t...@mirbsd.de, debian-...@lists.debian.org The openldap build on an m68k qemu-user buildd cannot install sasl2-bin in the chroot: […] Setting up pkg-config:m68k (1.8.1-1) ... Setting up libsasl2-2:m68k (2.1.28+dfsg1-5) ... Setting up libsasl2-modules-gssapi-mit:m68k (2.1.28+dfsg1-5) ... Setting up unixodbc-dev:m68k (2.3.12-1+b1) ... Setting up libgnutls28-dev:m68k (3.8.3-1.1+b2) ... Setting up libhcrypto5t64-heimdal:m68k (7.8.git20221117.28daf24+dfsg-5+b2) ... Setting up libotp0t64-heimdal:m68k (7.8.git20221117.28daf24+dfsg-5+b2) ... Setting up db-util (5.3.3) ... Setting up bind9-libs:m68k (1:9.19.21-1+b1) ... Setting up libsl0t64-heimdal:m68k (7.8.git20221117.28daf24+dfsg-5+b2) ... Setting up sasl2-bin (2.1.28+dfsg1-5) ... *** stack smashing detected ***: terminated qemu: uncaught target signal 6 (Aborted) - core dumped Aborted dpkg: error processing package sasl2-bin (--configure): installed sasl2-bin package post-installation script subprocess returned error exit status 134 Setting up libperl-dev:m68k (5.38.2-3.2+b1) ... Setting up libsasl2-dev (2.1.28+dfsg1-5) ... Setting up libgssrpc4t64:m68k (1.20.1-6+b1) ... Setting up libhx509-5t64-heimdal:m68k (7.8.git20221117.28daf24+dfsg-5+b2) ... dpkg: dependency problems prevent configuration of sbuild-build-depends-main-dummy: sbuild-build-depends-main-dummy depends on sasl2-bin; however: Package sasl2-bin is not configured yet. dpkg: error processing package sbuild-build-depends-main-dummy (--configure): dependency problems - leaving unconfigured […] Unpacking sbuild-build-depends-dose3-dummy (0.invalid.0) ... Setting up sasl2-bin (2.1.28+dfsg1-5) ... BDB0002 __fop_file_setup: Retry limit (100) exceeded saslpasswd2: generic failure dpkg: error processing package sasl2-bin (--configure): installed sasl2-bin package post-installation script subprocess returned error exit status 1 […] See: https://buildd.debian.org/status/fetch.php?pkg=openldap=m68k=2.5.16%2Bdfsg-2%2Bb2=1711312418=0 This does not seem to be specific to one buildd. Any idea how this can be debugged?
