[systemd-devel] systemd-nspawn image
Hello The title doesn't quite work; # qemu-img --version qemu-img version 2.6.2 (qemu-2.6.2-7.fc24), Copyright (c) 2004-2008 Fabrice Bellard # qemu-img convert fedora25.qcow2 fedora25.raw # file fedora25.* fedora25.qcow2: QEMU QCOW Image (v3), 21474836480 bytes fedora25.raw: DOS/MBR boot sector # fdisk -l fedora25.raw Disk fedora25.raw: 20 GiB, 21474836480 bytes, 41943040 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: dos Disk identifier: 0x13bd2a55 DeviceBoot Start End Sectors Size Id Type fedora25.raw1 * 2048 2099199 2097152 1G 83 Linux fedora25.raw2 2099200 6293503 4194304 2G 82 Linux swap / Solaris fedora25.raw3 6293504 41943039 35649536 17G 83 Linux # systemd-nspawn --version systemd 229 +PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN # systemd-nspawn --machine Fedora25 --image fedora25.raw Spawning container Fedora25 on /indian/cocos/fedora25.raw. Press ^] three times within 1s to kill container. Timezone Indian/Cocos does not exist in container, not updating container timezone. execv() failed: No such file or directory Container Fedora25 failed with error code 1. What could be a brake? ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] My experience with MySQL and systemctl
Am Tue, 11 Apr 2017 15:08:35 +0200 schrieb Lennart Poettering : > > Eventually, after checking all our backups, I decided to issue kill > > -9 to mysqld. I then decided to try restarting the daemon using > > systemctl. It did start up, the log output showed the crash recovery > > procedure, but because it entered into the rollback recovery, > > systemctl never considered that the process had finished starting > > up, and then tried to kill it again, which failed (only kill -9 > > would work in this case). Again, log output was closed. > > Again, we really don't do that. Logging is actually pretty independent > from service management when it comes to keeping the connections open: > all systemd does is set up the connection, fork off the process and > that's it. The forked processes can keep the connections open as long > as they wish, or even pass them on to other processes if they like... > > That said, there's one long-standing race: if a process logs through > syslog(), and very quickly after that exits we under some > circumstances have trouble matching up the log output with the > originating service. That's because the SCM_CREDENTIAL data we get for > the logged messages might reference a PID which already disappeared > from /proc/$PID which we hence cannot read the cgroup membership info > from. > > Hence: maybe the logs are there, but the filtering didn't work the way > you expect? Chances are that he's using the mysqld_safe script which does all kinds of stuff in a bash script, like detaching the daemon, pipe the logging, do process monitoring. Essentially, what systemd can already do just better. It's also doing the "kill -9" stuff. I'd like to know first if the daemon is properly started by systemd itself and not some intermediate script. I'm using the following drop-in as a starter: $ cat /etc/systemd/system/mysqld.service.d/override.conf [Service] Type=simple Restart=on-failure ExecStart= ExecStart=/usr/sbin/mysqld --defaults-file=/etc/mysql/my.cnf --pid-file=/run/mysqld/mysqld.pid RuntimeDirectory=mysqld RuntimeDirectoryMode=0755 WorkingDirectory=/var/lib/mysql -- Regards, Kai Replies to list-only preferred. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] My experience with MySQL and systemctl
On Tue, 11.04.17 13:41, Samuel Williams (space.ship.travel...@gmail.com) wrote: > I had an accident last night. I tried to delete a lot of rows from a > production database in one transaction. I killed the transaction, and > I didn't realise it was still rolling back an hour later when I tried > to reboot the system for updates. > > I might be wrong about exactly who is doing what, but I'll do my best > to explain what happened, and then give a few notes about how I think > the process could be improved. I don't specifically know if it's > systemctl or the service file, or mysqld causing the issues. > > systemctl tried to shutdown mysqld, but due to the rollback, it > wouldn't exit with anything less than kill -9. So, systemctl entered > the failed state, however, in addition to that, it appears it closed > the log output of the mysqld process so I couldn't see any of the > output from the daemon. IIRC mysql actually uses its own log-file, and neither logs to stderr nor to syslog? Note that systemd don't actually close stdout/stderr or syslog at all, this is always done by the logging client, not the recipient (journald). > Eventually, after checking all our backups, I decided to issue kill -9 > to mysqld. I then decided to try restarting the daemon using > systemctl. It did start up, the log output showed the crash recovery > procedure, but because it entered into the rollback recovery, > systemctl never considered that the process had finished starting up, > and then tried to kill it again, which failed (only kill -9 would work > in this case). Again, log output was closed. Again, we really don't do that. Logging is actually pretty independent from service management when it comes to keeping the connections open: all systemd does is set up the connection, fork off the process and that's it. The forked processes can keep the connections open as long as they wish, or even pass them on to other processes if they like... That said, there's one long-standing race: if a process logs through syslog(), and very quickly after that exits we under some circumstances have trouble matching up the log output with the originating service. That's because the SCM_CREDENTIAL data we get for the logged messages might reference a PID which already disappeared from /proc/$PID which we hence cannot read the cgroup membership info from. Hence: maybe the logs are there, but the filtering didn't work the way you expect? > I issued kill -9 again, and started up the process manually `sudo -u > mysql mysqld` and let it go through the entire recovery process, and > then rebooted the system. Normalcy restored. > > I think that the following things could be investigated/improved: > > - Log output shouldn't be closed if the daemon can't be killed. I'm > not sure who is doing this - perhaps it's mysqld, or perhaps it's the > service file, or perhaps its systemctl? I doubt that it's systemd. > - If a daemon fails to start up, trying to kill it.. may not be the > best option. It's probably a matter of the systemctl service file > detecting that a rollback is in progress and accepting that as a valid > startup state, but I'm not really sure. In any case, I ended up having > to do this process manually. The timeouts for killing services that don't start up correctly are configurable per service, and by setting them to "infinity" you may even turn them off entirely. This is configurable precisely so that services that can take ages to start-up in real-life can increase or turn off the timeout the way they need. Or in other words: please make sure that your mysql.service carries the right StartTimeoutSec=/StopTimeoutSec= settings, and please contact your unit file vendor to fix this. Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] Why journald has NotifyAccess=all set in the unit file?
Hi everyone, I was asked today about $subject. I quickly skimmed trough the relevant parts of the code and current default looks like an oversight. I think there are no processes other than journald involved in notification handling. I think it would be nice if drop the setting and rely on default NotifyAccess=main. Michal ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel