Ladsgroup created this task.
Ladsgroup added projects: Wikidata-Termbox, Wikidata, Wikidata-Campsite.
Restricted Application added a subscriber: Aklapper.
TASK DESCRIPTION
Termbox in labs was down for 218 hours, 53 minutes and 43 seconds. As
Wikidata's lovely incident manager this week, I brought it back by googling
output of systemd service error which said you need to restart the docker
daemon before restarting the termbox daemon otherwise the internal networking
of docker server gets confused and errors things like:
ladsgroup@wikidata-misc:~$ sudo /usr/bin/docker run --restart=always
--name=systemd_termbox_test -e STATSD_HOST=labmon1001.eqiad.wmnet -e
LOGSTASH_HOST=deployment-logstash2.eqiad.wmflabs -e
WIKIBASE_REPO=https://wikidata.beta.wmflabs.org/w -e
WIKIBASE_REPO_HOSTNAME_ALIAS=wikidata.beta.wmflabs.org -e SSR_PORT=3030
-p=3030:3031 wmde/wikibase-termbox-production:latest
/usr/bin/docker: Error response from daemon: driver failed programming
external connectivity on endpoint systemd_termbox_test
(13ec853e0d2772b8c7ca414ff56b39a2ca698bc3b6c9688134c31a3550db5a40): (iptables
failed: iptables --wait -t nat -A DOCKER -p tcp -d 0/0 --dport 3030 -j DNAT
--to-destination 172.17.0.2:3031 ! -i docker0: iptables: No chain/target/match
by that name.
(exit status 1)).
You need to add something like `sudo service docker restart` before the
updater trying to restart `termbox` daemon in updater or ansible settings.
TASK DETAIL
https://phabricator.wikimedia.org/T235069
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Ladsgroup
Cc: Addshore, alaa_wmde, Tarrow, Jakob_WMDE, Pablo-WMDE, Aklapper, Ladsgroup,
darthmon_wmde, DannyS712, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden,
LawExplorer, _jensen, rosalieper, Jonas, Wikidata-bugs, aude, Lydia_Pintscher,
Mbch331
_______________________________________________
Wikidata-bugs mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
