** Description changed: Because "PIDFile=" directive is missing in the systemd unit file, keepalived sometimes fails to kill all old processes. The old processes remain with old settings and cause unexpected behaviors. The detail of this bug is described in this ticket in upstream: https://github.com/acassen/keepalived/issues/443. The official systemd unit file is available since version 1.2.24 by this commit: https://github.com/acassen/keepalived/commit/635ab69afb44cd8573663e62f292c6bb84b44f15 This includes "PIDFile" directive correctly: PIDFile=/var/run/keepalived.pid We should go the same way. I am using Ubuntu 16.04.1, kernel 4.4.0-45-generic. Package: keepalived Version: 1.2.19-1 ======================================================================= How to reproduce: I used the two instances of Ubuntu 16.04.2 on DigitalOcean: Configurations -------------- MASTER server's /etc/keepalived/keepalived.conf: - vrrp_script chk_nothing { - script "/bin/true" - interval 2 - } - - vrrp_instance G1 { - interface eth1 - state BACKUP - priority 100 - - virtual_router_id 123 - unicast_src_ip <primal IP> - unicast_peer { - <secondal IP> - } - track_script { - chk_nothing - } - } + vrrp_script chk_nothing { + script "/bin/true" + interval 2 + } + vrrp_instance G1 { + interface eth1 + state BACKUP + priority 100 + + virtual_router_id 123 + unicast_src_ip <primal IP> + unicast_peer { + <secondal IP> + } + track_script { + chk_nothing + } + } BACKUP server's /etc/keepalived/keepalived.conf: - vrrp_script chk_nothing { - script "/bin/true" - interval 2 - } - - vrrp_instance G1 { - interface eth1 - state MASTER - priority 200 - - virtual_router_id 123 - unicast_src_ip <secondal IP> - unicast_peer { - <primal IP> - } - track_script { - chk_nothing - } - } + vrrp_script chk_nothing { + script "/bin/true" + interval 2 + } + vrrp_instance G1 { + interface eth1 + state MASTER + priority 200 + + virtual_router_id 123 + unicast_src_ip <secondal IP> + unicast_peer { + <primal IP> + } + track_script { + chk_nothing + } + } Procedures ---------- 1) Start keepalived on both servers - $ sudo systemctl start keepalived.service + $ sudo systemctl start keepalived.service 2) Restart keepalived on either one - $ sudo systemctl restart keepalived.service + $ sudo systemctl restart keepalived.service 3) Check status and PID - $ systemctl status -n0 keepalived.service - + $ systemctl status -n0 keepalived.service Result ------ - The results are bellow. As you can easily notice, on even numbers of restarts, systemd takes wrong PIDs as Main PID; the process is actually exited but is considered active. - root@ubuntu-2gb-sgp1-01:~# systemctl status -n0 keepalived - ● keepalived.service - Keepalive Daemon (LVS and VRRP) - Loaded: loaded (/lib/systemd/system/keepalived.service; enabled; vendor preset: enabled) - Active: active (running) since Sat 2017-03-04 01:37:12 UTC; 14min ago - Process: 3402 ExecStart=/usr/sbin/keepalived $DAEMON_ARGS (code=exited, status=0/SUCCESS) - Main PID: 3403 (keepalived) - Tasks: 3 - Memory: 1.7M - CPU: 1.900s - CGroup: /system.slice/keepalived.service - ├─3403 /usr/sbin/keepalived - ├─3405 /usr/sbin/keepalived - └─3406 /usr/sbin/keepalived - root@ubuntu-2gb-sgp1-01:~# systemctl restart keepalived - root@ubuntu-2gb-sgp1-01:~# systemctl status -n0 keepalived - ● keepalived.service - Keepalive Daemon (LVS and VRRP) - Loaded: loaded (/lib/systemd/system/keepalived.service; enabled; vendor preset: enabled) - Active: active (running) since Sat 2017-03-04 01:51:45 UTC; 1s ago - Process: 4782 ExecStart=/usr/sbin/keepalived $DAEMON_ARGS (code=exited, status=0/SUCCESS) - Main PID: 3403 (code=exited, status=0/SUCCESS) - Tasks: 3 - Memory: 1.7M - CPU: 11ms - CGroup: /system.slice/keepalived.service - ├─4783 /usr/sbin/keepalived - ├─4784 /usr/sbin/keepalived - └─4785 /usr/sbin/keepalived - root@ubuntu-2gb-sgp1-01:~# systemctl restart keepalived - root@ubuntu-2gb-sgp1-01:~# systemctl status -n0 keepalived - ● keepalived.service - Keepalive Daemon (LVS and VRRP) - Loaded: loaded (/lib/systemd/system/keepalived.service; enabled; vendor preset: enabled) - Active: active (running) since Sat 2017-03-04 01:51:49 UTC; 1s ago - Process: 4796 ExecStart=/usr/sbin/keepalived $DAEMON_ARGS (code=exited, status=0/SUCCESS) - Main PID: 4783 (keepalived) - Tasks: 3 - Memory: 1.7M - CPU: 6ms - CGroup: /system.slice/keepalived.service - ├─4783 /usr/sbin/keepalived - ├─4784 /usr/sbin/keepalived - └─4785 /usr/sbin/keepalived - root@ubuntu-2gb-sgp1-01:~# systemctl restart keepalived - root@ubuntu-2gb-sgp1-01:~# systemctl status -n0 keepalived - ● keepalived.service - Keepalive Daemon (LVS and VRRP) - Loaded: loaded (/lib/systemd/system/keepalived.service; enabled; vendor preset: enabled) - Active: active (running) since Sat 2017-03-04 01:51:53 UTC; 2s ago - Process: 4810 ExecStart=/usr/sbin/keepalived $DAEMON_ARGS (code=exited, status=0/SUCCESS) - Main PID: 4783 (code=exited, status=0/SUCCESS) - Tasks: 3 - Memory: 1.7M - CPU: 17ms - CGroup: /system.slice/keepalived.service - ├─4811 /usr/sbin/keepalived - ├─4812 /usr/sbin/keepalived - └─4813 /usr/sbin/keepalived + 0) Before restart + + Main PID is 3402 and the subprocesses' PIDs are 3403-3406. So far so + good. + + root@ubuntu-2gb-sgp1-01:~# systemctl status -n0 keepalived + ● keepalived.service - Keepalive Daemon (LVS and VRRP) + Loaded: loaded (/lib/systemd/system/keepalived.service; enabled; vendor preset: enabled) + Active: active (running) since Sat 2017-03-04 01:37:12 UTC; 14min ago + Process: 3402 ExecStart=/usr/sbin/keepalived $DAEMON_ARGS (code=exited, status=0/SUCCESS) + Main PID: 3403 (keepalived) + Tasks: 3 + Memory: 1.7M + CPU: 1.900s + CGroup: /system.slice/keepalived.service + ├─3403 /usr/sbin/keepalived + ├─3405 /usr/sbin/keepalived + └─3406 /usr/sbin/keepalived + + 1) First restart + + Now Main PID is 3403, which was one of the previous subprocesses and is + actually exited. Something is wrong. Yet, the previous processes are all + exited; we are not likely to see no weird behaviors here. + + root@ubuntu-2gb-sgp1-01:~# systemctl restart keepalived + root@ubuntu-2gb-sgp1-01:~# systemctl status -n0 keepalived + ● keepalived.service - Keepalive Daemon (LVS and VRRP) + Loaded: loaded (/lib/systemd/system/keepalived.service; enabled; vendor preset: enabled) + Active: active (running) since Sat 2017-03-04 01:51:45 UTC; 1s ago + Process: 4782 ExecStart=/usr/sbin/keepalived $DAEMON_ARGS (code=exited, status=0/SUCCESS) + Main PID: 3403 (code=exited, status=0/SUCCESS) + Tasks: 3 + Memory: 1.7M + CPU: 11ms + CGroup: /system.slice/keepalived.service + ├─4783 /usr/sbin/keepalived + ├─4784 /usr/sbin/keepalived + └─4785 /usr/sbin/keepalived + + 2) Second restart + + Now Main PID is 4783 and subprocesses' PIDs are 4783-4785. This is + problematic as 4783 is the old process, which should have exited before + new processes arose. Therefore, keepalived remains in old settings while + users believe it uses the new setting. + + root@ubuntu-2gb-sgp1-01:~# systemctl restart keepalived + root@ubuntu-2gb-sgp1-01:~# systemctl status -n0 keepalived + ● keepalived.service - Keepalive Daemon (LVS and VRRP) + Loaded: loaded (/lib/systemd/system/keepalived.service; enabled; vendor preset: enabled) + Active: active (running) since Sat 2017-03-04 01:51:49 UTC; 1s ago + Process: 4796 ExecStart=/usr/sbin/keepalived $DAEMON_ARGS (code=exited, status=0/SUCCESS) + Main PID: 4783 (keepalived) + Tasks: 3 + Memory: 1.7M + CPU: 6ms + CGroup: /system.slice/keepalived.service + ├─4783 /usr/sbin/keepalived + ├─4784 /usr/sbin/keepalived + └─4785 /usr/sbin/keepalived
-- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1644530 Title: keepalived fails to restart cleanly due to the wrong systemd settings To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/keepalived/+bug/1644530/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
