Re: [Libguestfs] libldm crashes in a linux-sandbox context
Hi Richard and Lersek, Thanks for your help on this issue and thanks for picking up my patch and applying it upstream! On Tue. 20 June 2023 à 17:10, Richard W.M. Jones wrote: > I think you've solved the problem now, but for future reference you > can run: > > $ virt-rescue Perfect! This last comment was what I needed for my final investigation. The UUID 65534 problem showed up again. Within the qemu VM, the active user is indeed root. However, $ ls -al /bin/mount -rwsr-xr-x 1 65534 65534 55528 May 30 15:42 /bin/mount Where 65534 corresponds to the user "nobody" and the group "nogroup". So the root cause was that the bazel sandbox created an environment in which SUID programs had a different UID and GID than expected. The guestfs-tools would just copy those IDs when creating the qemu rootfs. Even if /bin/mount was executed as root, the SUID makes it run effectively as nobody. This is kind of comical: it is the first time that I see a SUID resulting in a drop of privilege ¯\_(ツ)_/¯. At this point, I just gave up on using the bazel sandbox for the particular target in which I need guestfs-tools. For the record, and in case anyone has the same issue as I did and find this thread, the sandbox can be disabled for a particular target by using the "no-sandbox" tag. Example: genrule( name = "rootfs", srcs = ["rootfs.tar"], outs = ["rootfs.ext4"], tags = ["no-sandbox"], cmd = "virt-make-fs --format=raw --type=ext4 --size=+500M $< $@", ) Yours sincerely, Vincent Mailhol ___ Libguestfs mailing list Libguestfs@redhat.com https://listman.redhat.com/mailman/listinfo/libguestfs
[Libguestfs] [PATCH v3] ldmtool: fix NULL pointer dereference
If /sys/block can not be opened, get_devices() returns NULL.
cmdline() does not check this result and below code snippet:
scanned = get_devices();
devices = (gchar **) scanned->data;
results in a segmentation fault.
Add a check on scanned.
Relevant logs:
Unable to open /sys/block: No such file or directory
[0.777352] ldmtool[164]: segfault at 0 ip 563a225cd6a5 sp
7ffe54965a60 error 4 in ldmtool[563a225cb000+3000]
[0.778278] Code: 18 64 48 33 1c 25 28 00 00 00 75 5e 48 83 c4 28 5b 5d 41
5c 41 5d 41 5e 41 5f c3 66 2e 0f 1f 84 00 00 00 00 00 e8 db fd ff ff <4c> 8b 20
48 89 44 24 08 4c 89 e7 e8 0b e1 ff ff 45 31 c0 4c 89 e1
Fixes: 25d9635e4ee5 ("Add ldmtool")
Signed-off-by: Vincent Mailhol
---
* Changelog *
v2 -> v3
* Fix the From: tag (incorrect e-mail address, sorry for the noise).
v1 -> v2
* Directly return FALSE instead of goto error. Jumping to the error
label bypasses jb's declaration thus resulting in an undefined
behavior.
---
src/ldmtool.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/src/ldmtool.c b/src/ldmtool.c
index 6957c1a..dbe2c8c 100644
--- a/src/ldmtool.c
+++ b/src/ldmtool.c
@@ -746,6 +746,8 @@ cmdline(LDM * const ldm, gchar **devices,
GArray * scanned = NULL;
if (!devices) {
scanned = get_devices();
+if (!scanned)
+return FALSE;
devices = (gchar **) scanned->data;
}
--
2.25.1
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs
[Libguestfs] [PATCH v2] ldmtool: fix NULL pointer dereference
From: Vincent Mailhol
If /sys/block can not be opened, get_devices() returns NULL.
cmdline() does not check this result and below code snippet:
scanned = get_devices();
devices = (gchar **) scanned->data;
results in a segmentation fault.
Add a check on scanned.
Relevant logs:
Unable to open /sys/block: No such file or directory
[0.777352] ldmtool[164]: segfault at 0 ip 563a225cd6a5 sp
7ffe54965a60 error 4 in ldmtool[563a225cb000+3000]
[0.778278] Code: 18 64 48 33 1c 25 28 00 00 00 75 5e 48 83 c4 28 5b 5d 41
5c 41 5d 41 5e 41 5f c3 66 2e 0f 1f 84 00 00 00 00 00 e8 db fd ff ff <4c> 8b 20
48 89 44 24 08 4c 89 e7 e8 0b e1 ff ff 45 31 c0 4c 89 e1
Fixes: 25d9635e4ee5 ("Add ldmtool")
Signed-off-by: Vincent Mailhol
---
* Changelog *
v1 -> v2
* Directly return FALSE instead of goto error. Jumping to the error
label bypasses jb's declaration thus resulting in an undefined
behavior.
---
src/ldmtool.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/src/ldmtool.c b/src/ldmtool.c
index 6957c1a..dbe2c8c 100644
--- a/src/ldmtool.c
+++ b/src/ldmtool.c
@@ -746,6 +746,8 @@ cmdline(LDM * const ldm, gchar **devices,
GArray * scanned = NULL;
if (!devices) {
scanned = get_devices();
+if (!scanned)
+return FALSE;
devices = (gchar **) scanned->data;
}
--
2.25.1
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs
[Libguestfs] [PATCH v1] ldmtool: fix NULL pointer dereference
If /sys/block can not be opened, get_devices() returns NULL.
cmdline() does not check this result and below code snippet:
scanned = get_devices();
devices = (gchar **) scanned->data;
results in a segmentation fault.
Add a check on scanned.
Relevant logs:
Unable to open /sys/block: No such file or directory
[0.777352] ldmtool[164]: segfault at 0 ip 563a225cd6a5 sp
7ffe54965a60 error 4 in ldmtool[563a225cb000+3000]
[0.778278] Code: 18 64 48 33 1c 25 28 00 00 00 75 5e 48 83 c4 28 5b 5d 41
5c 41 5d 41 5e 41 5f c3 66 2e 0f 1f 84 00 00 00 00 00 e8 db fd ff ff <4c> 8b 20
48 89 44 24 08 4c 89 e7 e8 0b e1 ff ff 45 31 c0 4c 89 e1
Fixes: 25d9635e4ee5 ("Add ldmtool")
Signed-off-by: Vincent Mailhol
---
This thread did not yet show-up in
https://listman.redhat.com/archives/libguestfs/2023-June/subject.html
not sure why.
For this reason, I couln't add a link reference.
---
src/ldmtool.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/src/ldmtool.c b/src/ldmtool.c
index 6957c1a..87aaccc 100644
--- a/src/ldmtool.c
+++ b/src/ldmtool.c
@@ -746,6 +746,8 @@ cmdline(LDM * const ldm, gchar **devices,
GArray * scanned = NULL;
if (!devices) {
scanned = get_devices();
+if (!scanned)
+goto error;
devices = (gchar **) scanned->data;
}
--
2.25.1
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs
Re: [Libguestfs] libldm crashes in a linux-sandbox context
On Fri. 16 juin 2023 at 16:34, Richard W.M. Jones wrote: (...) > > Last thing, the segfault on ldmtool [1] still seems a valid issue. > > Even if I now do have a workaround for my problem, that segfault might > > be worth a bit more investigation. > > Yes that does look like a real problem. Does it crash if you just run > ldmtool as a normal command, nothing to do with libguestfs? Might be > a good idea to try to get a stack trace of the crash. The fact is that it only crashes with the UUID 65534 in the qemu VM. I am not sure what command line is passed to ldmtool for this crash to occur. I can help to gather information, but my biggest issue is that I do not know how to interact with the VM under /tmp/.guestfs-1001/ [0.777352] ldmtool[164]: segfault at 0 ip 563a225cd6a5 sp 7ffe54965a60 error 4 in ldmtool[563a225cb000+3000] ^^^ This smells like a NULL pointer dereference. The instruction pointer being 563a225cd6a5, I installed libguestfs-tools-dbgsym and tried a: addr2line -e /usr/bin/ldmtool 564a892506a5 Results: ??:0 Without conviction, I also tried in GDB: $ gdb /usr/bin/ldmtool (...) Reading symbols from /usr/bin/ldmtool... Reading symbols from /usr/lib/debug/.build-id/21/37b4a64903ebe427c242be08b8d496ba570583.debug... (gdb) info line *0x564a892506a5 No line number information available for address 0x564a892506a5 Debug symbols are correctly installed but impossible to convert that instruction pointer into a line number. It is as if the ldmtool on my host and the ldmtool in the qemu VM were from a different build. I tried to mount /tmp/.guestfs-1001/appliance.d/root but that disk image did not contain ldmtool. I am not sure how to generate a stack trace or a core dump within that qemu VM. If you can tell me how to get an interactive prompt (or any other guidance) I can try to collect more information. Yours sincerely, Vincent Mailhol ___ Libguestfs mailing list Libguestfs@redhat.com https://listman.redhat.com/mailman/listinfo/libguestfs
Re: [Libguestfs] libldm crashes in a linux-sandbox context
Hi Richard, On Fri. 16 Jun. 2023 à 03:08, Richard W.M. Jones wrote: > On Thu, Jun 15, 2023 at 09:18:38PM +0900, Vincent Mailhol wrote: > > Hello, > > > > I am using libguestfs in a Bazel's linux-sandbox environment[1]. > > > > When executing in that sandbox environment, I got frequent crashes. > > > > Please find attached below the results of libguestfs-test-tool when > > run into that linux-sandbox environment. The most relevant part seems > > to be: > > > > [0.797233] ldmtool[164]: segfault at 0 ip 564a892506a5 sp > > 7fff8ee5b900 error 4 in ldmtool[564a8924e000+3000] > > [0.798117] Code: 18 64 48 33 1c 25 28 00 00 00 75 5e 48 83 c4 28 5b > > 5d 41 5c 41 5d 41 5e 41 5f c3 66 2e 0f 1f 84 00 00 00 00 00 e8 db fd ff ff > > <4c> 8b 20 48 89 44 24 08 4c 89 e7 e8 0b e1 ff ff 45 31 c0 4c 89 e1 > > /init: line 154: 164 Segmentation fault ldmtool create all > > > > So the root cause seems to be around libldm. This mailing list seems > > to cover both libguestfs and libldm, so hopefully, I am at the right > > place to ask :) > > > > Needless to say, when run outside of the sandbox environment, no crash > > were observed. > > > > [1] linux-sandbox.cc > > Link: > > https://github.com/bazelbuild/bazel/blob/master/src/main/tools/linux-sandbox.cc > > > > --- > ... > > supermin: picked /sys/block/sdb/dev (8:16) as root device > > supermin: creating /dev/root as block special 8:16 > > supermin: mounting new root on /root > > [0.678248] EXT4-fs (sdb): mounting ext2 file system using the ext4 > > subsystem > > [0.679832] EXT4-fs (sdb): mounted filesystem without journal. Opts: . > > Quota mode: none. > > supermin: deleting initramfs files > > supermin: chroot > > Starting /init script ... > > mount: only root can use "--types" option (effective UID is 65534) > > /init: line 38: /proc/cmdline: No such file or directory > > mount: only root can use "--types" option (effective UID is 65534) > > mount: only root can use "--options" option (effective UID is 65534) > > mount: only root can use "--types" option (effective UID is 65534) > > mount: only root can use "--types" option (effective UID is 65534) > > mount: only root can use "--options" option (effective UID is 65534) > > It really goes wrong from here, where apparently it's not running as > root (instead UID 65534), even though we're supposed to be running > inside a Linux appliance virtual machine. > > Any idea why that would be? > > I looked at the sandbox and that would run the qemu process as UID > "nobody" (which might be 65534). However I don't understand why that > would affect anything running on the new kernel inside the appliance. And you were right. It was a fact that I got a crash in the sandbox but did not outside of it and I jumped to the conclusion that the root cause was linked to the sandbox. I continued the analysis and looked at all the differences between a successful libguestfs-test-tool log and the failed one. It turned out that the sandbox was not the cause. The culprit turns out to be the first line of the log: TMPDIR=/tmp. If I force TMPDIR=/var/tmp, the problem disappears !! This gave me a minimal reproducer: TMPDIR=/tmp/ libguestfs-test-tool That one crashed outside the sandbox. Next, my attention went to this line: libguestfs: checking for previously cached test results of /usr/bin/qemu-system-x86_64, in /tmp/.guestfs-1001 I did a: rm -rf /tmp/.guestfs-1001 and that solved my issue \o/ I still do not understand how I could get the issue of running of UID 65534 instead of root in the first place. I did other qemu experimentation, so not sure how, but I somehow got a corrupted environment under /tmp/.guestfs-1001. Last thing, the segfault on ldmtool [1] still seems a valid issue. Even if I now do have a workaround for my problem, that segfault might be worth a bit more investigation. Regardless, thanks a lot for your quick answer, that helped me to continue the troubleshooting. [1] ldmtool line 164 Link: https://github.com/mdbooth/libldm/blob/master/src/ldmtool.c#L164 ___ Libguestfs mailing list Libguestfs@redhat.com https://listman.redhat.com/mailman/listinfo/libguestfs
[Libguestfs] libldm crashes in a linux-sandbox context
Hello, I am using libguestfs in a Bazel's linux-sandbox environment[1]. When executing in that sandbox environment, I got frequent crashes. Please find attached below the results of libguestfs-test-tool when run into that linux-sandbox environment. The most relevant part seems to be: [0.797233] ldmtool[164]: segfault at 0 ip 564a892506a5 sp 7fff8ee5b900 error 4 in ldmtool[564a8924e000+3000] [0.798117] Code: 18 64 48 33 1c 25 28 00 00 00 75 5e 48 83 c4 28 5b 5d 41 5c 41 5d 41 5e 41 5f c3 66 2e 0f 1f 84 00 00 00 00 00 e8 db fd ff ff <4c> 8b 20 48 89 44 24 08 4c 89 e7 e8 0b e1 ff ff 45 31 c0 4c 89 e1 /init: line 154: 164 Segmentation fault ldmtool create all So the root cause seems to be around libldm. This mailing list seems to cover both libguestfs and libldm, so hopefully, I am at the right place to ask :) Needless to say, when run outside of the sandbox environment, no crash were observed. [1] linux-sandbox.cc Link: https://github.com/bazelbuild/bazel/blob/master/src/main/tools/linux-sandbox.cc --- *IMPORTANT NOTICE * * When reporting bugs, include the COMPLETE, UNEDITED * output below in your bug report. * TMPDIR=/tmp PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin SELinux: sh: 1: getenforce: not found guestfs_get_append: (null) guestfs_get_autosync: 1 guestfs_get_backend: direct guestfs_get_backend_settings: [] guestfs_get_cachedir: /tmp guestfs_get_hv: /usr/bin/qemu-system-x86_64 guestfs_get_memsize: 768 guestfs_get_network: 0 guestfs_get_path: /usr/lib/x86_64-linux-gnu/guestfs guestfs_get_pgroup: 0 guestfs_get_program: libguestfs-test-tool guestfs_get_recovery_proc: 1 guestfs_get_smp: 1 guestfs_get_sockdir: /tmp guestfs_get_tmpdir: /tmp guestfs_get_trace: 0 guestfs_get_verbose: 1 host_cpu: x86_64 Launching appliance, timeout set to 600 seconds. libguestfs: launch: program=libguestfs-test-tool libguestfs: launch: version=1.40.2 libguestfs: launch: backend registered: unix libguestfs: launch: backend registered: uml libguestfs: launch: backend registered: libvirt libguestfs: launch: backend registered: direct libguestfs: launch: backend=direct libguestfs: launch: tmpdir=/tmp/libguestfsART0fq libguestfs: launch: umask=0022 libguestfs: launch: euid=1001 libguestfs: begin building supermin appliance libguestfs: run supermin libguestfs: command: run: /usr/bin/supermin libguestfs: command: run: \ --build libguestfs: command: run: \ --verbose libguestfs: command: run: \ --if-newer libguestfs: command: run: \ --lock /tmp/.guestfs-1001/lock libguestfs: command: run: \ --copy-kernel libguestfs: command: run: \ -f ext2 libguestfs: command: run: \ --host-cpu x86_64 libguestfs: command: run: \ /usr/lib/x86_64-linux-gnu/guestfs/supermin.d libguestfs: command: run: \ -o /tmp/.guestfs-1001/appliance.d supermin: version: 5.1.20 supermin: package handler: debian/dpkg supermin: acquiring lock on /tmp/.guestfs-1001/lock supermin: if-newer: output does not need rebuilding libguestfs: finished building supermin appliance libguestfs: begin testing qemu features libguestfs: checking for previously cached test results of /usr/bin/qemu-system-x86_64, in /tmp/.guestfs-1001 libguestfs: loading previously cached test results libguestfs: qemu version: 4.2 libguestfs: qemu mandatory locking: yes libguestfs: qemu KVM: enabled libguestfs: finished testing qemu features /usr/bin/qemu-system-x86_64 \ -global virtio-blk-pci.scsi=off \ -no-user-config \ -enable-fips \ -nodefaults \ -display none \ -machine accel=kvm:tcg \ -cpu host \ -m 768 \ -no-reboot \ -rtc driftfix=slew \ -no-hpet \ -global kvm-pit.lost_tick_policy=discard \ -kernel /tmp/.guestfs-1001/appliance.d/kernel \ -initrd /tmp/.guestfs-1001/appliance.d/initrd \ -object rng-random,filename=/dev/urandom,id=rng0 \ -device virtio-rng-pci,rng=rng0 \ -device virtio-scsi-pci,id=scsi \ -drive file=/tmp/libguestfsART0fq/scratch1.img,cache=unsafe,format=raw,id=hd0,if=none \ -device scsi-hd,drive=hd0 \ -drive file=/tmp/.guestfs-1001/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none,format=raw \ -device scsi-hd,drive=appliance \ -device virtio-serial-pci \ -serial stdio \ -device sga \ -chardev socket,path=/tmp/libguestfsx6wHdq/guestfsd.sock,id=channel0 \ -device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \ -append "panic=1 console=ttyS0 edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=linux" libguestfs: responding to serial console Device Status Report \x1b[1;256r\x1b[256;256H\x1b[6n Google, Inc. Serial Graphics Adapte
