On 29.9.2024 7.31 ip., Andrea Bolognani wrote:
On Mon, Sep 23, 2024 at 11:14:02PM +0300, Mikko Rasa wrote:
After upgrading from 10.6.0-1 to 10.7.0-3 the scripts in /etc/init.d are not
executable and libvirtd fails to start. Easily rectified by doing chmod +x
on the scripts, after which everything works.
Also affects libvirt-daemon-log (virtlogd), libvirt-daemon-common
(libvirt-guests) and possibly others which I do not use.
This seems weird, since the executable bit is present in the package
itself:
$ dpkg-deb -x libvirt-daemon_10.7.0-3_amd64.deb .
$ ls -l etc/init.d/libvirtd
-rwxr-xr-x 1 abologna abologna 6136 Sep 16 21:41 etc/init.d/libvirtd
I have personally tested upgrades from 10.6.0-1 to 10.7.0-3 on hosts
using sysvinit and openrc instead of systemd to ensure that things
would work as expected in those scenarios, and I never encountered a
scenario like the one you're describing.
Did you customize the init scripts? We have some logic to carry
existing customizations over, and it might have gotten confused
somehow, though it should just move files around so I can't imagine
the executable bits getting lost in the process.
Any other local customizations that might be relevant?
I have not customized the init scripts.
I upgraded another computer from 10.2.0-1 to 10.7.0-3 and the same thing
happened. Then I downgraded the packages to 10.6.0-1 and re-upgraded,
but this time the executable bits stayed. Puzzled by this I did some
investigation and noticed in dpkg.log that my downgrade had marked
libvirt-daemon-system-sysv as manually installed so it didn't get
removed in the re-upgrade (it's now a transitional package and not
needed). I did another downgrade, marked the package as automatically
installed (using aptitude as I usually do), and upgraded again. The
executable bits were gone.
I had run aptitude under strace to capture what it does (perhaps a bit
of a sledgehammer, but it's the tool I know), so I went digging into the
log. This revealed that the postrm script for
libvirt-daemon-system-sysv does a chmod -x on the various init scripts
that used to belong to that package. Later on, when dpkg is replacing
the init scripts with the new versions, it appears to retain the modes
of the existing files.
The key detail appears to be the removal of libvirt-daemon-system-sysv
before most of the packages are upgraded. If the obsolete sysv package
is upgraded to the transitional version instead, then the exec bits are
not touched.
Attached are excerpts from dpkg.log and the aptitude trace (all syscalls
involving /etc/init.d/libvirtd plus the calls which spawned those
processes). I hope this plus my analysis above is enough to reproduce
the issue and come up with a solution. I'm hesitant to attach the full
files to a public bug report out of fear that they contain sensitive
details, but let me know if you want me to send them privately or if you
need any other information.
--
Mikko
10671 execve("/usr/bin/aptitude", ["aptitude", "install", "libvirt-daemon"],
0x7ffe9c05d418 /* 10 vars */) = 0
10671 clone(child_stack=NULL,
flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
child_tidptr=0x7ff293454b90) = 11068
11068 execve("/usr/bin/dpkg", ["/usr/bin/dpkg", "--status-fd", "39",
"--no-triggers", "--force-depends", "--abort-after=1", "--remove",
"libvirt-daemon-system-sysv:all"], 0x55c175a0e390 /* 11 vars */) = 0
11068 clone(child_stack=NULL,
flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
child_tidptr=0x7f19cedd24d0) = 11069
11069 execve("/var/lib/dpkg/info/libvirt-daemon-system-sysv.prerm",
["/var/lib/dpkg/info/libvirt-daemo"..., "remove"], 0x5571055fbde0 /* 20 vars
*/) = 0
11069 faccessat2(AT_FDCWD, "/etc/init.d/libvirtd", X_OK, AT_EACCESS) = 0
11070 execve("/usr/sbin/invoke-rc.d", ["invoke-rc.d", "--skip-systemd-native",
"libvirtd", "stop"], 0x5591f52819d0 /* 20 vars */ <unfinished ...>
11070 <... execve resumed>) = 0
11070 newfstatat(AT_FDCWD, "/etc/init.d/libvirtd", {st_mode=S_IFREG|0755,
st_size=6136, ...}, 0) = 0
11070 faccessat2(AT_FDCWD, "/etc/init.d/libvirtd", X_OK, AT_EACCESS) = 0
11070 faccessat2(AT_FDCWD, "/etc/init.d/libvirtd", X_OK, AT_EACCESS) = 0
11084 execve("/etc/init.d/libvirtd", ["/etc/init.d/libvirtd", "stop"],
0x56107ed96e78 /* 20 vars */ <unfinished ...>
11084 <... execve resumed>) = 0
11084 openat(AT_FDCWD, "/etc/init.d/libvirtd", O_RDONLY) = 3
11068 clone(child_stack=NULL,
flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
child_tidptr=0x7f19cedd24d0) = 11114
11114 execve("/var/lib/dpkg/info/libvirt-daemon-system-sysv.postrm",
["/var/lib/dpkg/info/libvirt-daemo"..., "remove"], 0x55710581d4a0 /* 20 vars
*/) = 0
11114 faccessat2(AT_FDCWD, "/etc/init.d/libvirtd", X_OK, AT_EACCESS) = 0
11116 execve("/usr/bin/chmod", ["chmod", "-x", "/etc/init.d/libvirtd"],
0x55bbab6a3930 /* 20 vars */ <unfinished ...>
11116 <... execve resumed>) = 0
11116 newfstatat(AT_FDCWD, "/etc/init.d/libvirtd", {st_mode=S_IFREG|0755,
st_size=6136, ...}, 0) = 0
11116 fchmodat(AT_FDCWD, "/etc/init.d/libvirtd", 0644) = 0
10671 clone(child_stack=NULL,
flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
child_tidptr=0x7ff293454b90) = 11121
11121 execve("/usr/bin/dpkg", ["/usr/bin/dpkg", "--status-fd", "39",
"--no-triggers", "--unpack", "--auto-deconfigure", "--recursive",
"/tmp/apt-dpkg-install-E0PH3h"], 0x55c1759534c0 /* 11 vars */) = 0
11121 read(9, "./etc/init.d/libvirtd\0\0\0\0\0\0\0\0\0\0\0"..., 512) = 512
11121 newfstatat(AT_FDCWD, "/etc/init.d/libvirtd", {st_mode=S_IFREG|0644,
st_size=6136, ...}, AT_SYMLINK_NOFOLLOW) = 0
11121 newfstatat(AT_FDCWD, "/etc/init.d/libvirtd", {st_mode=S_IFREG|0644,
st_size=6136, ...}, AT_SYMLINK_NOFOLLOW) = 0
11121 rmdir("/etc/init.d/libvirtd.dpkg-new") = -1 ENOENT (No such file or
directory)
11121 rmdir("/etc/init.d/libvirtd.dpkg-tmp") = -1 ENOENT (No such file or
directory)
11121 openat(AT_FDCWD, "/etc/init.d/libvirtd.dpkg-new",
O_WRONLY|O_CREAT|O_EXCL, 000) = 12
11121 utimensat(AT_FDCWD, "/etc/init.d/libvirtd.dpkg-new", [{tv_sec=1728307814,
tv_nsec=0} /* 2024-10-07T16:30:14+0300 */, {tv_sec=1726515675, tv_nsec=0} /*
2024-09-16T22:41:15+0300 */], 0) = 0
11121 openat(AT_FDCWD, "/etc/init.d/libvirtd.dpkg-new", O_WRONLY) = 9
11121 newfstatat(AT_FDCWD, "/etc/init.d/libvirtd", {st_mode=S_IFREG|0644,
st_size=6136, ...}, AT_SYMLINK_NOFOLLOW) = 0
10671 clone(child_stack=NULL,
flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
child_tidptr=0x7ff293454b90) = 11374
11374 execve("/usr/bin/dpkg", ["/usr/bin/dpkg", "--status-fd", "39",
"--configure", "--pending"], 0x55c1759534c0 /* 11 vars */) = 0
11374 newfstatat(AT_FDCWD, "/etc/init.d/libvirtd", {st_mode=S_IFREG|0644,
st_size=6136, ...}, AT_SYMLINK_NOFOLLOW) = 0
11374 openat(AT_FDCWD, "/etc/init.d/libvirtd", O_RDONLY) = 10
11374 newfstatat(AT_FDCWD, "/etc/init.d/libvirtd.dpkg-new",
{st_mode=S_IFREG|0755, st_size=6136, ...}, AT_SYMLINK_NOFOLLOW) = 0
11374 openat(AT_FDCWD, "/etc/init.d/libvirtd.dpkg-new", O_RDONLY) = 10
11374 newfstatat(AT_FDCWD, "/etc/init.d/libvirtd", {st_mode=S_IFREG|0644,
st_size=6136, ...}, 0) = 0
11374 newfstatat(AT_FDCWD, "/etc/init.d/libvirtd", {st_mode=S_IFREG|0644,
st_size=6136, ...}, 0) = 0
11374 chown("/etc/init.d/libvirtd.dpkg-new", 0, 0) = 0
11374 chmod("/etc/init.d/libvirtd.dpkg-new", 0644) = 0
11374 unlink("/etc/init.d/libvirtd.dpkg-new") = 0
11374 clone(child_stack=NULL,
flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
child_tidptr=0x7f1d528774d0) = 11492
11492 execve("/var/lib/dpkg/info/libvirt-daemon.postinst",
["/var/lib/dpkg/info/libvirt-daemo"..., "configure", "10.6.0-1"],
0x558003627d10 /* 20 vars */) = 0
11492 newfstatat(AT_FDCWD, "/etc/init.d/libvirtd.dpkg-transfer",
0x7ffd5aa29cc0, 0) = -1 ENOENT (No such file or directory)
11492 newfstatat(AT_FDCWD, "/etc/init.d/libvirtd.dpkg-disappear",
0x7ffd5aa29d10, 0) = -1 ENOENT (No such file or directory)
11492 faccessat2(AT_FDCWD, "/etc/init.d/libvirtd", X_OK, AT_EACCESS) = -1
EACCES (Permission denied)
Original faulty sequence (exec bits gone):
2024-10-07 15:49:09 upgrade libvirt-daemon-config-nwfilter:all 10.2.0-1 10.7.0-3
2024-10-07 15:49:09 upgrade libvirt-daemon-config-network:all 10.2.0-1 10.7.0-3
2024-10-07 15:49:09 upgrade libvirt-daemon-system:amd64 10.2.0-1 10.7.0-3
2024-10-07 15:49:10 remove libvirt-daemon-system-sysv:all 10.2.0-1 <none>
2024-10-07 15:49:10 upgrade libvirt-daemon-driver-qemu:amd64 10.2.0-1 10.7.0-3
2024-10-07 15:49:10 upgrade libvirt-clients:amd64 10.2.0-1 10.7.0-3
2024-10-07 15:49:10 upgrade libvirt-daemon:amd64 10.2.0-1 10.7.0-3
2024-10-07 15:49:10 install libvirt-daemon-driver-network:amd64 <none> 10.7.0-3
2024-10-07 15:49:10 install libvirt-daemon-driver-nwfilter:amd64 <none> 10.7.0-3
2024-10-07 15:49:10 upgrade libvirt0:amd64 10.2.0-1 10.7.0-3
2024-10-07 15:49:11 install libvirt-common:amd64 <none> 10.7.0-3
2024-10-07 15:49:11 install libvirt-daemon-log:amd64 <none> 10.7.0-3
2024-10-07 15:49:11 install libvirt-daemon-common:amd64 <none> 10.7.0-3
2024-10-07 15:49:11 install libvirt-daemon-driver-nodedev:amd64 <none> 10.7.0-3
2024-10-07 15:49:11 install libvirt-daemon-driver-secret:amd64 <none> 10.7.0-3
2024-10-07 15:49:11 install libvirt-daemon-driver-storage:amd64 <none> 10.7.0-3
Working sequence (exec bits remain):
2024-10-07 16:11:17 upgrade libvirt-daemon-config-nwfilter:all 10.6.0-1 10.7.0-3
2024-10-07 16:11:17 upgrade libvirt-daemon-config-network:all 10.6.0-1 10.7.0-3
2024-10-07 16:11:17 upgrade libvirt-daemon-system:amd64 10.6.0-1 10.7.0-3
2024-10-07 16:11:18 upgrade libvirt-daemon-system-sysv:all 10.6.0-1 10.7.0-3
2024-10-07 16:11:18 upgrade libvirt-daemon-driver-qemu:amd64 10.6.0-1 10.7.0-3
2024-10-07 16:11:18 upgrade libvirt-clients:amd64 10.6.0-1 10.7.0-3
2024-10-07 16:11:18 upgrade libvirt-daemon:amd64 10.6.0-1 10.7.0-3
2024-10-07 16:11:19 install libvirt-daemon-driver-network:amd64 10.7.0-3
10.7.0-3
2024-10-07 16:11:19 install libvirt-daemon-common:amd64 10.7.0-3 10.7.0-3
2024-10-07 16:11:19 install libvirt-daemon-log:amd64 10.7.0-3 10.7.0-3
2024-10-07 16:11:19 upgrade libvirt0:amd64 10.6.0-1 10.7.0-3
2024-10-07 16:11:19 install libvirt-common:amd64 10.7.0-3 10.7.0-3
2024-10-07 16:11:19 install libvirt-daemon-driver-nwfilter:amd64 <none> 10.7.0-3
2024-10-07 16:11:19 install libvirt-daemon-driver-nodedev:amd64 <none> 10.7.0-3
2024-10-07 16:11:19 install libvirt-daemon-driver-secret:amd64 <none> 10.7.0-3
2024-10-07 16:11:19 install libvirt-daemon-driver-storage:amd64 <none> 10.7.0-3
Reproduced faulty sequence (exec bits gone):
2024-10-07 16:30:13 upgrade libvirt-daemon-config-nwfilter:all 10.6.0-1 10.7.0-3
2024-10-07 16:30:13 upgrade libvirt-daemon-config-network:all 10.6.0-1 10.7.0-3
2024-10-07 16:30:13 upgrade libvirt-daemon-system:amd64 10.6.0-1 10.7.0-3
2024-10-07 16:30:14 remove libvirt-daemon-system-sysv:all 10.6.0-1 <none>
2024-10-07 16:30:15 upgrade libvirt-daemon-driver-qemu:amd64 10.6.0-1 10.7.0-3
2024-10-07 16:30:15 upgrade libvirt-clients:amd64 10.6.0-1 10.7.0-3
2024-10-07 16:30:15 upgrade libvirt-daemon:amd64 10.6.0-1 10.7.0-3
2024-10-07 16:30:15 install libvirt-daemon-driver-network:amd64 <none> 10.7.0-3
2024-10-07 16:30:15 install libvirt-daemon-driver-nwfilter:amd64 <none> 10.7.0-3
2024-10-07 16:30:15 upgrade libvirt0:amd64 10.6.0-1 10.7.0-3
2024-10-07 16:30:16 install libvirt-common:amd64 <none> 10.7.0-3
2024-10-07 16:30:16 install libvirt-daemon-log:amd64 <none> 10.7.0-3
2024-10-07 16:30:16 install libvirt-daemon-common:amd64 <none> 10.7.0-3
2024-10-07 16:30:16 install libvirt-daemon-driver-nodedev:amd64 <none> 10.7.0-3
2024-10-07 16:30:16 install libvirt-daemon-driver-secret:amd64 <none> 10.7.0-3
2024-10-07 16:30:16 install libvirt-daemon-driver-storage:amd64 <none> 10.7.0-3
