Bug#798935: Starting dnsmasq deadlock with /etc/resolvconf/update-libc.d/squid3
On Thu, 01 Dec 2016 20:41:57 -0500 Stefan Monnier wrote: > > On a system with systemd, dnsmasq and resolvconf installed, the > > /etc/resolvconf/update-libc.d/squid3 hook prevents dnsmasq from > > starting. As far as I can see, the problem is caused by a deadlock of > > "systemctl start dnsmasq.service" in conjunction with "systemctl > > reload squid3.service". The latter one is finally called by the > > /etc/resolvconf/update-libc.d/squid3 hook which is invoked by dnsmasq > > on startup. > With squid now at version 4 the situation leading to this issue has changed quite significantly. Can someone encountering this issue please check and confirm if it is resolved or present with squid 4.2-2 or later. Older versions are already known broken in ways that may affect the test result. Cheers Amos
Bug#798935: Starting dnsmasq deadlock with /etc/resolvconf/update-libc.d/squid3
> On a system with systemd, dnsmasq and resolvconf installed, the > /etc/resolvconf/update-libc.d/squid3 hook prevents dnsmasq from > starting. As far as I can see, the problem is caused by a deadlock of > "systemctl start dnsmasq.service" in conjunction with "systemctl > reload squid3.service". The latter one is finally called by the > /etc/resolvconf/update-libc.d/squid3 hook which is invoked by dnsmasq > on startup. Indeed, I just bumped into this problem. I recently decided to move from Polipo to Squid (on my home router), so I installed Squid. Everything seemed to be dandy until I rebooted, at which point dnsmasq failed to start, preventing access to the outside world to all other machines. > A possible solution might be, to check if systemd is running in the > hook and calling > > systemctl --no-block reload squid3.service > > directly instead of > > invoke-rc.d squid3 reload How 'bout just adding an & at the end of this reload line? That's the workaround I'm using right now, Stefan
Bug#798935: Starting dnsmasq deadlock with /etc/resolvconf/update-libc.d/squid3
Just ran into this bug. The resolvconf hook is indeed run after dnsmasq is up and running. When I run `sudo systemctl start dnsmasq` from a terminal, it doesn't return, and running `systemctl` from another terminal reveals dnsmasq.service loaded activating start-post start dnsmasq - A lightweight DHCP and caching DNS server i.e. the ExecStartPost task of dnsmasq.service is running. In fact, at this moment dnsmasq is already working. If I look at running processes: $ ps aux | grep -e dns -e resolv root 2462 0.0 0.6 40540 3496 pts/1 S+ 19:50 0:00 sudo systemctl start dnsmasq root 2463 0.0 0.4 22488 2512 pts/1 S+ 19:50 0:00 systemctl start dnsmasq dnsmasq 2474 0.0 0.5 36744 2852 ? S 19:50 0:00 /usr/sbin/dnsmasq -x /var/run/dnsmasq/dnsmasq.pid -u dnsmasq -r /var/run/dnsmasq/resolv.conf -7 /etc/dnsmasq.d,.dpkg-dist,.dpkg-old,.dpkg-new --local-service --trust-anchor=.,19036,8,2,49AAC11D7B6F6446702E54A1607371607A1A41855200FD2CE1CDDE32F24E8FB5 root 2475 0.0 0.3 4336 1640 ? Ss 19:50 0:00 /bin/sh /etc/init.d/dnsmasq systemd-start-resolvconf root 2481 0.0 0.1 4224 652 ? S 19:50 0:00 run-parts --arg=-a --arg=lo.dnsmasq /etc/resolvconf/update.d root 2497 0.0 0.1 4224 636 ? S 19:50 0:00 run-parts /etc/resolvconf/update-libc.d root 2539 0.0 0.1 4336 756 ? S 19:50 0:00 /bin/sh /etc/resolvconf/update-libc.d/squid3 So it's /etc/resolvconf/update-libc.d/squid3 that's hanging. After a few minutes, the status of dnsmasq.service changes to: dnsmasq.service loaded deactivating stop start dnsmasq - A lightweight DHCP and caching DNS server And: $ ps aux | grep -e dns -e resolv root 2462 0.0 0.6 40540 3496 pts/1 S+ 19:50 0:00 sudo systemctl start dnsmasq root 2463 0.0 0.4 22488 2512 pts/1 S+ 19:50 0:00 systemctl start dnsmasq dnsmasq 2474 0.0 0.5 36744 2856 ? S 19:50 0:00 /usr/sbin/dnsmasq -x /var/run/dnsmasq/dnsmasq.pid -u dnsmasq -r /var/run/dnsmasq/resolv.conf -7 /etc/dnsmasq.d,.dpkg-dist,.dpkg-old,.dpkg-new --local-service --trust-anchor=.,19036,8,2,49AAC11D7B6F6446702E54A1607371607A1A41855200FD2CE1CDDE32F24E8FB5 root 2475 0.0 0.3 4336 1640 ? Ss 19:50 0:00 /bin/sh /etc/init.d/dnsmasq systemd-start-resolvconf root 2481 0.0 0.1 4224 652 ? S 19:50 0:00 run-parts --arg=-a --arg=lo.dnsmasq /etc/resolvconf/update.d root 2497 0.0 0.1 4224 636 ? S 19:50 0:00 run-parts /etc/resolvconf/update-libc.d root 2539 0.0 0.1 4336 756 ? S 19:50 0:00 /bin/sh /etc/resolvconf/update-libc.d/squid3 root 2581 0.0 0.3 4336 1668 ? Ss 19:51 0:00 /bin/sh /etc/init.d/dnsmasq systemd-stop-resolvconf root 2586 0.0 0.1 4224 660 ? S 19:51 0:00 run-parts --arg=-d --arg=lo.dnsmasq /etc/resolvconf/update.d root 2598 0.0 0.1 4224 652 ? S 19:51 0:00 run-parts /etc/resolvconf/update-libc.d root 2640 0.0 0.1 4336 792 ? S 19:51 0:00 /bin/sh /etc/resolvconf/update-libc.d/squid3 So systemd decides that the starting of dnsmasq.service has failed since its ExecStartPost task timed out, and tries to stop dnsmasq.service. This triggers resolvconf again in dnsmasq.service's ExecStop task, and another instance of /etc/resolvconf/update-libc.d/squid3 is run, while the previous copy hasn't even exited yet. After a few more minutes, the ExecStop task also times out, and the original terminal finally returns with Job for dnsmasq.service failed. See 'systemctl status dnsmasq.service' and 'journalctl -xn' for details.
Bug#798935: Starting dnsmasq deadlock with /etc/resolvconf/update-libc.d/squid3
tags severity important On Mon, 14 Sep 2015 11:11:45 +0200 Patrick Cernko wrote: > > On a system with systemd, dnsmasq and resolvconf installed, the > /etc/resolvconf/update-libc.d/squid3 hook prevents dnsmasq from > starting. That hints that the dnsmasq startup sequence is being done the wrong way around. Squid and other processes using /etc/resolvconf/update-libc.d/ hooks rely on the listed NS providing working service *before* the hook is called. Because the key activity done during the hook is to re-resolve configuration details using the new NS listings. Start dnsmasq running, *then* update resolv.conf settings. > As far as I can see, the problem is caused by a deadlock of > "systemctl start dnsmasq.service" in conjunction with "systemctl > reload squid3.service". The latter one is finally called by the > /etc/resolvconf/update-libc.d/squid3 hook which is invoked by dnsmasq > on startup. Why is dnsmasq deadlocking at all? it should not be depending on things those hooks do. > > A possible solution might be, to check if systemd is running in the > hook and calling > > systemctl --no-block reload squid3.service We are working on a squid.service upstream and finding all the many, many broken and missing "features" in systemd. Making the squid3 package depend on systemd .service files would be completely wrong for production use. Amos (Squid upstream)
Bug#798935: Starting dnsmasq deadlock with /etc/resolvconf/update-libc.d/squid3
Package: squid3 Version: 3.4.8-6+deb8u1 Severity: critical On a system with systemd, dnsmasq and resolvconf installed, the /etc/resolvconf/update-libc.d/squid3 hook prevents dnsmasq from starting. As far as I can see, the problem is caused by a deadlock of "systemctl start dnsmasq.service" in conjunction with "systemctl reload squid3.service". The latter one is finally called by the /etc/resolvconf/update-libc.d/squid3 hook which is invoked by dnsmasq on startup. A possible solution might be, to check if systemd is running in the hook and calling systemctl --no-block reload squid3.service directly instead of invoke-rc.d squid3 reload Best regards, -- System Information: Debian Release: 8.2 APT prefers proposed-updates APT policy: (500, 'proposed-updates'), (500, 'stable') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 3.18.20.1.amd64-smp (SMP w/1 CPU core) Locale: LANG=de_DE, LC_CTYPE=de_DE (charmap=ISO-8859-1) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) Versions of packages squid3 depends on: ii adduser 3.113+nmu3 ii libc62.19-18+deb8u1 ii libcap2 1:2.24-8 ii libcomerr2 1.42.12-1.1 ii libdb5.3 5.3.28-9 ii libecap2 0.2.0-3 ii libexpat12.1.0-6+deb8u1 ii libgcc1 1:4.9.2-10 ii libgssapi-krb5-2 1.12.1+dfsg-19 ii libk5crypto3 1.12.1+dfsg-19 ii libkrb5-31.12.1+dfsg-19 ii libldap-2.4-22.4.40+dfsg-1 ii libltdl7 2.4.2-1.11 ii libnetfilter-conntrack3 1.0.4-1 ii libnettle4 2.7.1-5 ii libpam0g 1.1.8-3.1 ii libsasl2-2 2.1.26.dfsg1-13 ii libstdc++6 4.9.2-10 ii libxml2 2.9.1+dfsg1-5 ii logrotate3.8.7-1+b1 ii lsb-base 4.1+Debian13+nmu1 ii netbase 5.3 ii squid3-common3.4.8-6+deb8u1 squid3 recommends no packages. Versions of packages squid3 suggests: ii resolvconf 1.76.1 pn smbclient pn squid-cgi pn squid-purge pn squidclient pn ufw pn winbindd -- Configuration Files: /etc/resolvconf/update-libc.d/squid3 changed [not included] -- no debconf information
