** Description changed: [Impact] - TBD + + The systemd file rabbitmq-server.service on Bionic uses "Type=simple" + when defining the service, but unfortunately this doesn't work very well + for rabbitmq-server. In certain situations, systemd will fail to keep + track of a start/stop/restart event, and will hang for 90 seconds before + giving the prompt back to the user. Another problem is that rabbitmq- + server must start after the epmd service, so we need to explicitly + declare this dependency in the service file. [Test Case] - TBD + + Although I was able to reproduce this almost 100% of the time, there + were rare occasions when the restart procedure finished normally. I was + also only able to reproduce it using a bionic VM, not a container. If + you have multipass or lxd configured to launch VMs, that should be easy. + + The steps are: + + $ lxc launch ubuntu-daily:bionic --vm bug1784757-rabbitmq-server # or use multipass + $ lxc shell bug1784757-rabbitmq-server + # apt update + # apt install rabbitmq-server -y + # systemctl restart rabbitmq-server.service + + In a normal scenario, the restart should take around 3 seconds or less. + With the bug, it takes around 90 seconds. If you can't reproduce it, try + running "systemctl restart" again. A quick way to trigger it is to run a + for loop like: + + # for i in $(seq 10); do time systemctl restart rabbitmq-server.service + ; done [Regression Potential] - TBD - [Fix] - TBD + * Because rabbitmq-server implements systemd's "Type=notify" using socat + to communicate with systemd-notify over a socket, we will be introducing + another point of failure (socat) in the mix. - The fix is available upstream in Debian as of 3.7.6, so would need - backported for bionic and cosmic: - - rabbitmq-server | 3.5.7-1 | xenial | source, all - rabbitmq-server | 3.6.10-1 | bionic | source, all - rabbitmq-server | 3.6.10-1 | cosmic | source, all - rabbitmq-server | 3.7.8-4ubuntu2 | disco | source, all - rabbitmq-server | 3.7.8-4ubuntu2 | eoan | source, all - - [Discussion] - TBD + * So, albeit unlikely, there may be a problem when using socat. That + would not be a regression, though, since the outcome would be the same + as we have today: "systemctl restart" would not properly work, even + though the service did restart. [Original Report] - When I run `systemctl restart rabbitmq-server` it waits for 90 seconds then systemd sends SIGKILL to it. + + When I run `systemctl restart rabbitmq-server` it waits for 90 seconds + then systemd sends SIGKILL to it. Presumably the `epmd` process does not receive SIGTERM, since if I run `kill 1493` (or whatever pid it currently is) then restart happens straight after that successfully ● rabbitmq-server.service - RabbitMQ Messaging Server Loaded: loaded (/lib/systemd/system/rabbitmq-server.service; enabled; vendor preset: enabled) Active: deactivating (final-sigterm) since Wed 2018-08-01 01:17:04 UTC; 7s ago Process: 1183 ExecStop=/usr/sbin/rabbitmqctl stop (code=exited, status=0/SUCCESS) Process: 178 ExecStartPost=/usr/lib/rabbitmq/bin/rabbitmq-server-wait (code=exited, status=0/SUCCESS) Process: 177 ExecStart=/usr/sbin/rabbitmq-server (code=killed, signal=TERM) Main PID: 177 (code=killed, signal=TERM) Tasks: 1 (limit: 4915) CGroup: /system.slice/rabbitmq-server.service └─1493 /usr/lib/erlang/erts-9.2/bin/epmd -daemon Aug 01 01:11:20 rmq-1 systemd[1]: rabbitmq-server.service: Failed to reset devices.list: Operation not permitted Aug 01 01:11:20 rmq-1 systemd[1]: Starting RabbitMQ Messaging Server... Aug 01 01:11:25 rmq-1 rabbitmq[178]: Waiting for 'rabbit@rmq-1' Aug 01 01:11:25 rmq-1 rabbitmq[178]: pid is 204 Aug 01 01:11:30 rmq-1 systemd[1]: Started RabbitMQ Messaging Server. Aug 01 01:17:04 rmq-1 systemd[1]: Stopping RabbitMQ Messaging Server... Aug 01 01:17:06 rmq-1 rabbitmq[1183]: Stopping and halting node 'rabbit@rmq-1'
-- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1784757 Title: [SRU] rabbitmq-server does not properly shutdown To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/rabbitmq-server/+bug/1784757/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
