Re: boot ordering and resolvconf
On Friday and Saturday I sent messages to this list but they
appear to have been filtered out. I try again and CC:
Helmut Grohne this time so at least he will receive this
message and we can continue the discussion over e-mail.
My Friday message contained a C program and its output
when run on my own machine. The purpose was to illustrate
a program re-reading resolv.conf when it changes. Since then
Helmut has posted a long message[0] which among other
things makes the same point, so there's no need for me to
belabor it here. However, there is one remaining question
where we still have different findings and I think we should
get to the bottom of this difference since it is relevant for
how we think about ntpd bug report #683061.
Helmut Grohne wrote:
On Sun, Jun 30, 2013 at 10:40:03PM +0200, Thomas Hood wrote:
A problem is that getaddrinfo() doesn't distinguish in its return
status between couldn't reach a nameserver and nameserver
says the name doesn't exist. Either way it returns EAI_SYSTEM
with errno=2 (ENOENT).
I cannot confirm this observation and assume it to be wrong.
Helmut Grohne wrote:
So what happens when you getaddrinfo ...
... a non-existent name? - EAI_SYSTEM ENOENT
... something when nothing is bound to port 53?
- EAI_SYSTEM ECONNREFUSED
... something when port 53 is bound, but does not answer?
- EAI_SYSTEM ETIMEDOUT
I don't get these results.
I attach a program x.c and append the output below, run on
Ubuntu 13.04. It shows that when resolv.conf is an empty file
and getaddrinfo() is called (... I have no local nameserver
running, so this causes failure to find a nameserver...) the
return value and errno are -11 and 2: EAI_SYSTEM and ENOENT.
The return values are the same when getaddrinfo() is called
with a nonexistent domain name. It was on the basis of a quick
test like this that I wrote what Helmut quoted.
But that was on an Ubuntu 13.04 system. I next ran the program
on Debian 7.0 and got the same results except that the return
value was -2 (EAI_NONAME) instead of -11, both in the case of
the bogus domain name and in the case of the unreachable
nameserver. Ubuntu and Debian thus differ on the return
value, but neither of them distinguishes between
name-doesn't-exist and nameserver-not-found.
Comparing my program with Helmut's the only significant
difference I saw was that he gave getaddrinfo() NULL as the
hints argument. I changed my program to give NULL hints
but the results were the same.
I wonder why Helmut and I are getting different results.
=== x.c run on Ubuntu 13.04 ===
$ sudo ./a.out
Making resolv.conf empty
Results of looking up www.google.com: status = -11, errno = 2
Results of looking up a bogus name: status = -11, errno = 2
Writing nameserver option to resolv.conf
Results of looking up www.google.com: status = 0, errno = 101
Results of looking up a bogus name: status = -11, errno = 2
Making resolv.conf empty
Results of looking up www.google.com: status = -11, errno = 2
Results of looking up a bogus name: status = -11, errno = 2
Writing nameserver option to resolv.conf
Results of looking up www.google.com: status = 0, errno = 101
Results of looking up a bogus name: status = -11, errno = 2
=== x.c run on Debian 7.0 ===
$ su
# ./a.out
Making resolv.conf empty
Results of looking up www.google.com: status = -2, errno = 2
Results of looking up a bogus name: status = -2, errno = 2
Writing nameserver option to resolv.conf
Results of looking up www.google.com: status = 0, errno = 101
Results of looking up a bogus name: status = -2, errno = 2
Making resolv.conf empty
Results of looking up www.google.com: status = -2, errno = 2
Results of looking up a bogus name: status = -2, errno = 2
Writing nameserver option to resolv.conf
Results of looking up www.google.com: status = 0, errno = 101
Results of looking up a bogus name: status = -2, errno = 2
[0] http://lists.debian.org/debian-devel/2013/07/msg00201.html
--
Thomas Hood
#include sys/types.h
#include sys/socket.h
#include netdb.h
#include errno.h
#include stdio.h
struct addrinfo hints;
struct addrinfo *res;
void check_google()
{
int status;
status = getaddrinfo(www.google.com, 0, NULL, res);
printf(Results of looking up www.google.com: status = %d, errno = %d\n, status, errno);
status = getaddrinfo(sjfkdsjfswfloo0f02938sjf28398sd.com, 0, NULL, res);
printf(Results of looking up a bogus name: status = %d, errno = %d\n, status, errno);
}
Re: boot ordering and resolvconf
On 03/07/13 14:30, Ian Jackson wrote: Ian Jackson writes (Re: boot ordering and resolvconf): 4. Therefore in most installations there should be a local proxy or cache. It should use DHCP-provided, PPP-provided or similar, as a forwarder. The local DNS provider address should be owned by whatever proxy or cache is installed. Is there some reason not to use dnsmasq for this ? To do this we would have to: * install dnsmasq by default * teach resolvconf to update dnsmasq's config rather than resolv.conf (but apparently Ubuntu have done this work) * make sure that the full-on DNS servers all conflict with dnsmasq and listen on 127.0.0.1 Please don't make DNS servers conflict with each other. I have Dnsmasq, Unbound and NSD all installed. Unbound provides recursive DNSSEC enabled resolution for the local network, it forwards queries for the LAN domain to an authoritative Dnsmasq DNS/DHCP servicer running on a different port and NSD is running on an external interface serving an external domain. I don't think this is an unreasonable configuration to want. Thanks, Roger -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/fnalaa-qlq@silverstone.rilynn.me.uk
Re: boot ordering and resolvconf
On Sun, Jun 30, 2013 at 12:03:47PM +0200, Thomas Hood wrote: That resolvconf leaves resolv.conf empty of nameserver entries when no nameservers are available is intentional. The idea is to advertise only addresses where something is actually listening. Even though intentional it might be a bad decision to do so for other reasons. By the way, inconsistent with the aforementioned idea is the fact that the resolver defaults to using 127.0.0.1 if there are no nameserver lines in resolv.conf. Were you aware of that? I were aware, but only because I triggered this case while debugging ntp. Unfortunately there is a subtle difference. When resolv.conf is empty and 127.0.0.1. does not responds, it is EAI_SYSTEM, but when it is listed and not responding it is EAI_AGAIN. I am therefore suggesting to change resolvconf to make this inconsistency explicit and always list 127.0.0.1 when there is no other name server. It should probably be accompanied with a warning comment, because this case should never occur during normal operation. On Sun, Jun 30, 2013 at 10:40:03PM +0200, Thomas Hood wrote: (slightly reordered) Helmut Grohne wrote: It is true, that ntpd can work around this issue. Still it appears like a bad design to treat EAI_SYSTEM as a temporary error. Can you explain why you think that? The obvious reason here is that having two error codes to be reacted to in the same way is prone to errors. Forget one? But you already gave an even better reason yourself: A problem is that getaddrinfo() doesn't distinguish in its return status between couldn't reach a nameserver and nameserver says the name doesn't exist. Either way it returns EAI_SYSTEM with errno=2 (ENOENT). It could result in retrying non-existent domains forever. Helmut -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130705172846.ga9...@alf.mars
Re: boot ordering and resolvconf
Sorry for the noise, I finally verified some more claims and discovered that my other recent mail is wrong in multiple places. I guess this discussion really is moot. So what happens when you getaddrinfo ... ... a non-existent name? - EAI_SYSTEM ENOENT ... something when nothing is bound to port 53? - EAI_SYSTEM ECONNREFUSED ... something when port 53 is bound, but does not answer? - EAI_SYSTEM ETIMEDOUT Contrary to what I said earlier it does not make a difference whether resolv.conf lists 127.0.0.1 or is empty in all three cases. So my other mail claiming to add 127.0.0.1 to resolv.conf when there is no nameserver is just wrong. On Sun, Jun 30, 2013 at 10:40:03PM +0200, Thomas Hood wrote: A problem is that getaddrinfo() doesn't distinguish in its return status between couldn't reach a nameserver and nameserver says the name doesn't exist. Either way it returns EAI_SYSTEM with errno=2 (ENOENT). I cannot confirm this observation and assume it to be wrong. For clarity I include my own method[1]. I therefore withdraw my comment about EAI_SYSTEM appearing like a bad design. I just didn't understand it. On Fri, Jul 05, 2013 at 07:21:12AM +0200, Thomas Hood wrote: The only underlying problem I know of is that ntpd is broken (#683061). Other applications work fine in mode B. And the reason that ntpd doesn't work properly is NOT that resolv.conf is dynamic. Ntpd doesn't work properly because it treats any name resolution failure as equivalent to host does not exist and treats host does not exist as a fatal error. So ntpd will also fail in mode A unless you manage to get name service fully operational before ntpd starts. Merely pointing /etc/resolv.conf at a local nameserver does not rule out the possibility of name service failures; it just ensures that applications can access some nameserver once the local nameserver has started. That's nice but it doesn't fix ntpd. Lookups will still fail, e.g., if ntpd starts before the nameserver or if the nameserver doesn't have a forwarding address yet. Applications have to be able to deal with temporary name service failures. Given the above, I fully agree. Helmut Grohne wrote: Usually any program reads /etc/resolv.conf once on the first DNS lookup. So all daemons started before the local DNS cache will either use a different server, or fail DNS resolution in all cases. A minority of services (avahi-daemon, fetchmail, postfix, sendmail, squid, and squid3) hook into resolvconf to reload their daemons when /etc/resolv.conf is changed by resolvconf. These daemons will not be affected by this problem. Many other services on the other hand will. This is an as-yet unsubstantiated claim. Libc resolver clients generally re-read resolv.conf every time it changes. Are there known examples of programs that fail to do so? This aspect of my initial claim was wrong. I was not aware that libc always stats resolv.conf. Thanks for pointing out. Given that systemd will bind 127.0.0.1:53 and the libc uses 127.0.0.1 in the presence of an empty resolv.conf, I see that daemons will have no issues unless they also fail on an unresponsive name server during normal operation. There is no issue. Revisiting the two issues I stated in my initial mail: | 1) If /etc/resolv.conf is initially empty, getaddrinfo returns |EAI_SYSTEM. Some daemons (e.g. ntp) currently treat this as a |permanent error. This usually results in a user visible issue. It is |nasty to track down, because it is usually unreproducible once the |system is started (i.e. a simple daemon restart will fix it). The specific meaning of EAI_SYSTEM is defined by errno and ntp was not handling ECONNREFUSED. Maybe it can be found in other daemons, but systemd will not exhibit this, because there you will only see ETIMEDOUT. The issue is not general. If it can be found anywhere, then in services under sysv using $named without declaring an ordering requirement on it. | 2) If /etc/resolv.conf initially contains some other DNS server (e.g. |from dhcp or from /etc/network/interfaces), some services will bypass |the local DNS cache resulting in a higher load on remote DNS servers. |This problem in general is mostly invisible. Fixed in libc by stating resolv.conf. So we basically are in mode B and it works most of the time. Thanks for clearing things up. I guess this is EOD. Helmut [1] code below #include sys/types.h
Re: boot ordering and resolvconf
On Wed, Jul 03, 2013 at 10:24:42AM +0200, Tollef Fog Heen wrote: What do you do when you are on a network that blocks DNS lookups that don't go via the DNS servers for that network? Or for networks that do that until you visit a web page and press a button on a form? Manually reconfigure unbound to use those DNS servers as forwarders. I can propose a viable alternative from experience: Install resolvconf. You can drop the /etc/resolv.conf managing part by changing the symlink. Nevertheless resolvconf will feed servers obtained from dhcp or ppp to unbound. Only downside: Some announced forwarders break DNSSEC. Helmut -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130705200252.ga29...@alf.mars
Re: boot ordering and resolvconf
Ian Jackson wrote: I think the first thing to do is recognise the underlying problem. To fix this problem properly we need a coherent system design. The two designs lead to different sets of fixes. A. resolv.conf is a static file which changes only very rarely. B. resolv.conf is not static and may change due to network environment changes. The only underlying problem I know of is that ntpd is broken (#683061). Other applications work fine in mode B. And the reason that ntpd doesn't work properly is NOT that resolv.conf is dynamic. Ntpd doesn't work properly because it treats any name resolution failure as equivalent to host does not exist and treats host does not exist as a fatal error. So ntpd will also fail in mode A unless you manage to get name service fully operational before ntpd starts. Merely pointing /etc/resolv.conf at a local nameserver does not rule out the possibility of name service failures; it just ensures that applications can access some nameserver once the local nameserver has started. That's nice but it doesn't fix ntpd. Lookups will still fail, e.g., if ntpd starts before the nameserver or if the nameserver doesn't have a forwarding address yet. Applications have to be able to deal with temporary name service failures. The assumptions that got this thread started were made here: Helmut Grohne wrote: Usually any program reads /etc/resolv.conf once on the first DNS lookup. So all daemons started before the local DNS cache will either use a different server, or fail DNS resolution in all cases. A minority of services (avahi-daemon, fetchmail, postfix, sendmail, squid, and squid3) hook into resolvconf to reload their daemons when /etc/resolv.conf is changed by resolvconf. These daemons will not be affected by this problem. Many other services on the other hand will. This is an as-yet unsubstantiated claim. Libc resolver clients generally re-read resolv.conf every time it changes. Are there known examples of programs that fail to do so? Ian Jackson wrote: Is there some reason not to use dnsmasq for this ? To do this we would have to: * install dnsmasq by default * teach resolvconf to update dnsmasq's config rather than resolv.conf (but apparently Ubuntu have done this work) * make sure that the full-on DNS servers all conflict with dnsmasq and listen on 127.0.0.1 Policy would say: * By default resolv.conf should contain only 127.0.0.1 * No package may update resolv.conf's nameserver settings * All nameserver packages should conflict with dnsmasq (perhaps via a specified virtual-package) and must provide general nameservice on 127.0.0.1. * Nameserver packages and network configuration packages should use resolvconf to communicate the actual nameservers, if they wish to do this. (Support for this is not required by policy, but if support is provided this is how it should be done.) It's worth discussing what Debian should install and run by default. Ubuntu Server installs resolvconf by default. Ubuntu Desktop installs resolvconf, NetworkManager and dnsmasq-base by default: NetworkManager runs an instance of dnsmasq as a forwarding nameserver. But there is no need to introduce new restrictions and policy unless, perhaps, it does turn out that large numbers of libc resolver clients need to be modified, which so far as I know is not the case. -- Thomas Hood
Re: boot ordering and resolvconf
]] Bob Proulx Tollef Fog Heen wrote: ]] Don Armstrong Tollef Fog Heen wrote: It seems resolvconf wants to get its name servers from /etc/network/interfaces? Resolvconf can get its nameservers from anywhere that calls echo 'namserver information'|resolvconf -a interface.program; If I do that by hand, that information will never ever be overwritten by dhclient, NM, openvpn or tools, and it persists through reboots? Now that we have so many mobile devices the network may come and go dynamically. Therefore we need a dynamic method to update nameservers. When a network is activated it would call the above to configure the nameservers found for that network interface. When the network interface is deactivated then those nameservers will need to be removed. Having them fixed statically would defeat the purpose. No, in my setup it should not do that without admin intervention. Doing that means it's likely to cause security-related problems. I use DNSSEC and don't want to trust random resolvers, I want to trust the one that I've set up myself and that I know verifies signatures and that lives on ::1. -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/m2zju45evz@rahvafeir.err.no
Re: boot ordering and resolvconf
On Wed, Jul 3, 2013 at 2:36 PM, Tollef Fog Heen wrote: No, in my setup it should not do that without admin intervention. Doing that means it's likely to cause security-related problems. I use DNSSEC and don't want to trust random resolvers, I want to trust the one that I've set up myself and that I know verifies signatures and that lives on ::1. What do you do when you are on a network that blocks DNS lookups that don't go via the DNS servers for that network? Or for networks that do that until you visit a web page and press a button on a form? -- bye, pabs http://wiki.debian.org/PaulWise -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/caktje6ggmjtgsuj6ecyvm-m7+hxrdvlr2fqsfyq9sxrplnk...@mail.gmail.com
Re: boot ordering and resolvconf
]] Paul Wise On Wed, Jul 3, 2013 at 2:36 PM, Tollef Fog Heen wrote: No, in my setup it should not do that without admin intervention. Doing that means it's likely to cause security-related problems. I use DNSSEC and don't want to trust random resolvers, I want to trust the one that I've set up myself and that I know verifies signatures and that lives on ::1. What do you do when you are on a network that blocks DNS lookups that don't go via the DNS servers for that network? Or for networks that do that until you visit a web page and press a button on a form? Manually reconfigure unbound to use those DNS servers as forwarders. -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87y59onj91@qurzaw.varnish-software.com
Re: boot ordering and resolvconf
On Wed, Jul 3, 2013 at 4:24 PM, Tollef Fog Heen wrote: Manually reconfigure unbound to use those DNS servers as forwarders. ...and then manually reconfigure it to do its own lookups when you go to another network and find they aren't reachable from there? -- bye, pabs http://wiki.debian.org/PaulWise -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAKTje6FUdbKjypZvYqZO0qbDdt=2f7sbepga1itrq7-jdrq...@mail.gmail.com
Re: boot ordering and resolvconf
Thomas Hood writes (Re: boot ordering and resolvconf): Well, you are right that resolvconf was designed to support mode B, and only supports mode A as a trivial case of mode B where a local forwarding nameserver is always running. But mode A involves /etc/resolf.conf not changing during the boot process. As I wrote in my original description: A. resolv.conf is a static file which changes only very rarely. So your statement that resolvconf supports mode A is simply incorrect. Ian. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20948.7942.730522.165...@chiark.greenend.org.uk
Re: boot ordering and resolvconf
Ian Jackson writes (Re: boot ordering and resolvconf): ... I think the first thing to do is recognise the underlying problem. To fix this problem properly we need a coherent system design. The two designs lead to different sets of fixes. A. resolv.conf is a static file which changes only very rarely. ... B. resolv.conf is not static and may change due to network environment changes. ... The difficulty with plan A is probably [this requirement:] ... 4. Therefore in most installations there should be a local proxy or cache. It should use DHCP-provided, PPP-provided or similar, as a forwarder. The local DNS provider address should be owned by whatever proxy or cache is installed. Is there some reason not to use dnsmasq for this ? To do this we would have to: * install dnsmasq by default * teach resolvconf to update dnsmasq's config rather than resolv.conf (but apparently Ubuntu have done this work) * make sure that the full-on DNS servers all conflict with dnsmasq and listen on 127.0.0.1 Policy would say: * By default resolv.conf should contain only 127.0.0.1 * No package may update resolv.conf's nameserver settings * All nameserver packages should conflict with dnsmasq (perhaps via a specified virtual-package) and must provide general nameservice on 127.0.0.1. * Nameserver packages and network configuration packages should use resolvconf to communicate the actual nameservers, if they wish to do this. (Support for this is not required by policy, but if support is provided this is how it should be done.) I think this is probably easier than updating every dns consumer. Ian. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20948.9448.430861.233...@chiark.greenend.org.uk
Re: boot ordering and resolvconf
To those who aren't yet familiar with resolvconf, I'd suggest reading the resolvconf package README and the resolvconf(8) manpage. http://anonscm.debian.org/gitweb/?p=resolvconf/resolvconf.git;a=blob;f=README http://anonscm.debian.org/gitweb/?p=resolvconf/resolvconf.git;a=blob;f=man/resolvconf.8 -- Thomas Hood -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/51d43cc2.4030...@gmail.com
Re: boot ordering and resolvconf
]] Ian Jackson Ian Jackson writes (Re: boot ordering and resolvconf): ... I think the first thing to do is recognise the underlying problem. To fix this problem properly we need a coherent system design. The two designs lead to different sets of fixes. A. resolv.conf is a static file which changes only very rarely. ... B. resolv.conf is not static and may change due to network environment changes. ... The difficulty with plan A is probably [this requirement:] ... 4. Therefore in most installations there should be a local proxy or cache. It should use DHCP-provided, PPP-provided or similar, as a forwarder. The local DNS provider address should be owned by whatever proxy or cache is installed. Is there some reason not to use dnsmasq for this ? It doesn't support DNSSEC. If we're going to put a local resolver on all machines, it should support DNSSEC. -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/m2k3l763up@rahvafeir.err.no
Re: boot ordering and resolvconf
On Wed, 03 Jul 2013, Tollef Fog Heen wrote: Does that mean it's an RC bug for any non-manual process to overwrite [/etc/resolv.conf]? I'd be happy to file bugs. I don't believe it's RC, but it's certainly a bug. [The fact that dhclient does this when it's not called with appropriate arguments is pretty annoying.] If I do [invocation of resolvconf] by hand, that information will never ever be overwritten by dhclient, NM, openvpn or tools, and it persists through reboots? If you put that in /etc/rc.local and use an interface with high enough priority to override everything else, sure. Ok, good, that's different from the behaviour I've seen in the past, but if that's fixd, that's great. It's been updating files in a memory-backed filesystem for a very long time. Not sure what you mean by «resolver»? Same thing you do. -- Don Armstrong http://www.donarmstrong.com There's nothing remarkable about it. All one has to do is hit the right keys at the right time and the instrument plays itself. -- Bach -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130703200902.go12...@teltox.donarmstrong.com
Re: boot ordering and resolvconf
On Wed, 03 Jul 2013, Ian Jackson wrote: But mode A involves /etc/resolf.conf not changing during the boot process. As I wrote in my original description: A. resolv.conf is a static file which changes only very rarely. So your statement that resolvconf supports mode A is simply incorrect. It supports it by either having the statements specified in /e/n/i (or something else which causes resolvconf to be called) or it supports it by having /etc/resolv.conf being a file instead of a symlink. -- Don Armstrong http://www.donarmstrong.com If it jams, force it. If it breaks, it needed replacing anyway. -- Lowery's Law -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130703201140.gp12...@teltox.donarmstrong.com
Re: boot ordering and resolvconf
Tollef Fog Heen wrote: I think changing /etc/resolv.conf automatically is broken at all. What's the malfunction? We should have a generated /run/resolv.conf that's overridden by the settings in /etc/resolv.conf (if it exists). This allows you to have a consistent set of domains searched for matching hosts even when roaming to other networks. It also allows me to express the preference «I want to use localhost as a nameserver, always» without resorting to chattr to prevent dhclient, NetworkManager and other tools from auto-updating the resolv.conf file. Those features are available in resolvconf. Let's consider your idea. I like the aspect that /etc/resolv.conf remains a static file and doesn't have to be symlinked to a generated resolv.conf. However, your idea requires that the glibc resolver be enhanced. And once it is enhanced the logic is cast in binary stone: /etc/resolv.conf always overrides /run/resolv.conf, period. With resolvconf the logic is under the control of the administrator. If the admin puts a static file at /etc/resolv.conf then resolv.conf is static. If the admin puts a symlink at /etc/resolv.conf then the target of that symlink is used. With the resolvconf approach the resolver configuration is readable in one file which has the familiar semantics, resolv.conf(5). If there are two files then questions arise about how the one overrides the other. If /run/resolv.conf contains nameserver options and so does /etc/resolv.conf, then are both addresses tried, or just the latter, or what? Also, you will need a third file, also static, to provide defaults which get overridden by /run/resolv.conf. This will need to be documented. If it's the case that the mere existence of a file at /etc/resolv.conf causes /run/resolv.conf to be completely ignored then there is not so great a difference between your idea and resolvconf. With resolvconf, the absence of a symlink at /etc/resolv.conf is what causes the dynamic resolv.conf to be ignored. What is missing from your idea so far is functionality to handle multiple sources of nameserver information. What if the machine runs both dhclient and a VPN client, or both ifup and pppd? In the past each of these sorts of programs updated /etc/resolv.conf and (sometimes) restored the file to the preceding state on exit which left the file in an incorrect state if programs were stopped in other than LIFO order. Some packages make use of resolvconf hook scripts, a mechanism whereby they get notified when the resolver configuration changes. You could think about whether or not you want to implement that too, and how. Cheers, - - Thomas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/51d2c629.30...@gmail.com
Re: boot ordering and resolvconf
]] Thomas Hood Tollef Fog Heen wrote: I think changing /etc/resolv.conf automatically is broken at all. What's the malfunction? Automatic processes overwrite explicit admin setups. We should have a generated /run/resolv.conf that's overridden by the settings in /etc/resolv.conf (if it exists). This allows you to have a consistent set of domains searched for matching hosts even when roaming to other networks. It also allows me to express the preference «I want to use localhost as a nameserver, always» without resorting to chattr to prevent dhclient, NetworkManager and other tools from auto-updating the resolv.conf file. Those features are available in resolvconf. Let's consider your idea. I like the aspect that /etc/resolv.conf remains a static file and doesn't have to be symlinked to a generated resolv.conf. However, your idea requires that the glibc resolver be enhanced. And once it is enhanced the logic is cast in binary stone: /etc/resolv.conf always overrides /run/resolv.conf, period. With resolvconf the logic is under the control of the administrator. If the admin puts a static file at /etc/resolv.conf then resolv.conf is static. If the admin puts a symlink at /etc/resolv.conf then the target of that symlink is used. It seems resolvconf wants to get its name servers from /etc/network/interfaces? If so, that won't work particularly well with NetworkManager, unless I'm mistaken. Also, I don't think updating files in /etc at runtime is a sensible idea, it should be possible to run with / read-only if you want to. With the resolvconf approach the resolver configuration is readable in one file which has the familiar semantics, resolv.conf(5). I find resolvconf's behaviour confusing enough that I generally purge it from all my machines. It is, to me, an abstraction layer that seems to randomly overwrite my settings. Having it write to /run and having apps fall back to settings there if there's no setting in /etc/resolv.conf would ease that confusion, I think. If there are two files then questions arise about how the one overrides the other. If /run/resolv.conf contains nameserver options and so does /etc/resolv.conf, then are both addresses tried, or just the latter, or what? I specified that: settings are overridden, the file in /run is not masked. Whether you want to append the nameserver list or override the one from /run should probably be specified. I'd say override, since appending can easily lead to security breaches. Also, you will need a third file, also static, to provide defaults which get overridden by /run/resolv.conf. This will need to be documented. You don't need a third file the defaults are «an empty file», just like today. [...] What is missing from your idea so far is functionality to handle multiple sources of nameserver information. What if the machine runs both dhclient and a VPN client, or both ifup and pppd? In the past each of these sorts of programs updated /etc/resolv.conf and (sometimes) restored the file to the preceding state on exit which left the file in an incorrect state if programs were stopped in other than LIFO order. In that case, feel free to provide a framework for packages to coordinate updates to /run/resolv.conf and have stacking and whatnot. (They could write to /run/resolv.conf.d/$num_$basename and resolvconf or a similar tool could build a /run/resolv.conf from that.) Some packages make use of resolvconf hook scripts, a mechanism whereby they get notified when the resolver configuration changes. You could think about whether or not you want to implement that too, and how. I think any packages needing such hooks should just be fixed, since they're buggy. Adding a lot of infrastructure to work around bugginess is, IMO, not a good idea. -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87mwq59i3f@xoog.err.no
Re: boot ordering and resolvconf
On Tue, 02 Jul 2013, Tollef Fog Heen wrote: Automatic processes overwrite explicit admin setups. If /etc/resolv.conf is a symlink to somewhere else, then it's appropriate for automatic processes to override it by writing to somewhere else. If it's not a symlink, then it shouldn't be overridden. It seems resolvconf wants to get its name servers from /etc/network/interfaces? Resolvconf can get its nameservers from anywhere that calls echo 'namserver information'|resolvconf -a interface.program; Also, I don't think updating files in /etc at runtime is a sensible idea, it should be possible to run with / read-only if you want to. Yes, which is exactly why resolvconf doesn't update /etc during normal operation. I specified that: settings are overridden, the file in /run is not masked. Whether you want to append the nameserver list or override the one from /run should probably be specified. I'd say override, since appending can easily lead to security breaches. The only difference here between using resolvconf and this setup is that instead of having the configuration be specified in the /etc/resolv.conf file or symlink, it's specified in the resolver. In that case, feel free to provide a framework for packages to coordinate updates to /run/resolv.conf and have stacking and whatnot. (They could write to /run/resolv.conf.d/$num_$basename and resolvconf or a similar tool could build a /run/resolv.conf from that.) This is already what resolvconf does. It has information on interface primacy (/etc/resolvconf/interface order), knows how many nameservers it is useful to have in /etc/resolv.conf. -- Don Armstrong http://www.donarmstrong.com This message brought to you by weapons of mass destruction related program activities, and the letter G. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130702182805.ga12...@teltox.donarmstrong.com
Re: boot ordering and resolvconf
Don Armstrong wrote: On Tue, 02 Jul 2013, Tollef Fog Heen wrote: Automatic processes overwrite explicit admin setups. If /etc/resolv.conf is a symlink to somewhere else, then it's appropriate for automatic processes to override it by writing to somewhere else. If it's not a symlink, then it shouldn't be overridden. That seems pretty plain, but Ubuntu/Networkmanager routinely ignore this fact. I'm a bit surprised that this conversation still needs to happen today. It was a constant annoyance when I was using Ubuntu. (I finally switched to Arch back in January this year.) Networkmanager seems to have been the root of most of my complaints; its authors seem to believe that their code knows better than a machine's sysadmin how the machine should be operated. https://mail.gnome.org/archives/networkmanager-list/2008-September/msg00189.html The sensible thing to do is let something like dnsmasq do all resolver management. I first proposed this change ~5 years ago https://mail.gnome.org/archives/networkmanager-list/2008-September/msg00042.html It's a topic that comes up perennially. https://mail.gnome.org/archives/networkmanager-list/2009-April/msg00157.html https://mail.gnome.org/archives/networkmanager-list/2011-January/msg00020.html https://mail.gnome.org/archives/networkmanager-list/2012-April/msg00072.html Providing solutions for other tools, e.g. dhclient or pppd is pretty trivial too. http://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/2009q2/003009.html We're well into the 2nd decade of the 21st century. These problems shouldn't exist any more. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/51d32c30.1030...@symas.com
Re: boot ordering and resolvconf
On Tue, Jul 02, 2013 at 12:38:24PM -0700, Howard Chu wrote: Don Armstrong wrote: On Tue, 02 Jul 2013, Tollef Fog Heen wrote: Automatic processes overwrite explicit admin setups. If /etc/resolv.conf is a symlink to somewhere else, then it's appropriate for automatic processes to override it by writing to somewhere else. If it's not a symlink, then it shouldn't be overridden. That seems pretty plain, but Ubuntu/Networkmanager routinely ignore this fact. I'm not sure why you group Ubuntu and NetworkManager together here. Ubuntu has been using dnsmasq and resolvconf on the desktop (and resolvconf alone on the server) since the 12.04 release specifically to provide a coherent architecture and get away from NM's past bad behavior of changing files in /etc out from under the admin. http://www.stgraber.org/2012/02/24/dns-in-ubuntu-12-04/ *By default*, all Ubuntu systems running a newer release will have /etc/resolv.conf as a symlink to /run to ensure the system behaves sensibly (correct DNS server settings with or without dnsmasq running; honoring DNS server settings configured in /etc/network/interfaces; and so on). But if this symlink is really bothering you, you can make /etc/resolv.conf a real file and resolvconf won't mess with it any further. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developerhttp://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: Digital signature
Re: boot ordering and resolvconf
Steve Langasek wrote: On Tue, Jul 02, 2013 at 12:38:24PM -0700, Howard Chu wrote: Don Armstrong wrote: On Tue, 02 Jul 2013, Tollef Fog Heen wrote: Automatic processes overwrite explicit admin setups. If /etc/resolv.conf is a symlink to somewhere else, then it's appropriate for automatic processes to override it by writing to somewhere else. If it's not a symlink, then it shouldn't be overridden. That seems pretty plain, but Ubuntu/Networkmanager routinely ignore this fact. I'm not sure why you group Ubuntu and NetworkManager together here. Ubuntu has been using dnsmasq and resolvconf on the desktop (and resolvconf alone on the server) since the 12.04 release specifically to provide a coherent architecture and get away from NM's past bad behavior of changing files in /etc out from under the admin. http://www.stgraber.org/2012/02/24/dns-in-ubuntu-12-04/ *By default*, all Ubuntu systems running a newer release will have /etc/resolv.conf as a symlink to /run to ensure the system behaves sensibly (correct DNS server settings with or without dnsmasq running; honoring DNS server settings configured in /etc/network/interfaces; and so on). But if this symlink is really bothering you, you can make /etc/resolv.conf a real file and resolvconf won't mess with it any further. That sounds nice, but my old laptop running 12.04 is still getting its /etc/resolv.conf overwritten by networkmanager, I just took a look and the timestamp on it is from a few hours ago. Anyway, having just now read the launchpad blueprint, I'm sorry I missed this discussion when I was at that UDS. resolvconf is an unnecessary layer here, dnsmasq handles all of the interface-specific knowledge already. Patching dhclient's interface scripts to talk to dnsmasq over DBUS is trivial (see link in my previous email) and anything else that wants to touch it can be fixed easily as well. virtmanager's use of dnsmasq is a non-issue since it binds its instances to its specific virtual networks already. They all just need to know to forward to the main dnsmasq on 127.0.0.1 when handling their clients' requests. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/51d34b3d@symas.com
Re: boot ordering and resolvconf
]] Don Armstrong On Tue, 02 Jul 2013, Tollef Fog Heen wrote: Automatic processes overwrite explicit admin setups. If /etc/resolv.conf is a symlink to somewhere else, then it's appropriate for automatic processes to override it by writing to somewhere else. If it's not a symlink, then it shouldn't be overridden. Does that mean it's an RC bug for any non-manual process to overwrite it? I'd be happy to file bugs. It seems resolvconf wants to get its name servers from /etc/network/interfaces? Resolvconf can get its nameservers from anywhere that calls echo 'namserver information'|resolvconf -a interface.program; If I do that by hand, that information will never ever be overwritten by dhclient, NM, openvpn or tools, and it persists through reboots? Also, I don't think updating files in /etc at runtime is a sensible idea, it should be possible to run with / read-only if you want to. Yes, which is exactly why resolvconf doesn't update /etc during normal operation. Ok, good, that's different from the behaviour I've seen in the past, but if that's fixd, that's great. I specified that: settings are overridden, the file in /run is not masked. Whether you want to append the nameserver list or override the one from /run should probably be specified. I'd say override, since appending can easily lead to security breaches. The only difference here between using resolvconf and this setup is that instead of having the configuration be specified in the /etc/resolv.conf file or symlink, it's specified in the resolver. Not sure what you eman by «resolver»? (To me, that's the C code inside glibc that does the actual lookup, which doesn't really fit what you're describing.) -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/m27gh87bmk@rahvafeir.err.no
Re: boot ordering and resolvconf
Tollef Fog Heen wrote: ]] Don Armstrong Tollef Fog Heen wrote: It seems resolvconf wants to get its name servers from /etc/network/interfaces? Resolvconf can get its nameservers from anywhere that calls echo 'namserver information'|resolvconf -a interface.program; If I do that by hand, that information will never ever be overwritten by dhclient, NM, openvpn or tools, and it persists through reboots? Now that we have so many mobile devices the network may come and go dynamically. Therefore we need a dynamic method to update nameservers. When a network is activated it would call the above to configure the nameservers found for that network interface. When the network interface is deactivated then those nameservers will need to be removed. Having them fixed statically would defeat the purpose. And also there is the need to change nameservers when VPNs are established. A vpn comes online will often mean that the nameservers from that vpn need to be used to access private names on that network. I miss the old days when we could simply edit the files in /etc once and be done with it. That is still the way I do things on servers. But on servers I don't install resolvconf. However on mobile devices that roam from network to network resolvconf rocks! Bob signature.asc Description: Digital signature
Re: boot ordering and resolvconf
]] Helmut Grohne * Treating an empty /etc/resolv.conf as a permanent error or failing to discover changes in /etc/resolv.conf later on. * Changing /etc/resolv.conf during startup of a local dns cache. So if /etc/resolv.conf is declared to be static (A), then the second practise is broken. And if dynamic modifications are to be supported (B), then the first practise is broken. I think changing /etc/resolv.conf automatically is broken at all. We should have a generated /run/resolv.conf that's overridden by the settings in /etc/resolv.conf (if it exists). This allows you to have a consistent set of domains searched for matching hosts even when roaming to other networks. It also allows me to express the preference «I want to use localhost as a nameserver, always» without resorting to chattr to prevent dhclient, NetworkManager and other tools from auto-updating the resolv.conf file. -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/m2ppv27pdv@rahvafeir.err.no
Re: boot ordering and resolvconf
In addition resolvconf does not properly support mode (A), due to the local forwarding nameserver is running requirement. It can leave /etc/resolv.conf empty until the name server is actually started. Well, you are right that resolvconf was designed to support mode B, and only supports mode A as a trivial case of mode B where a local forwarding nameserver is always running. That resolvconf leaves resolv.conf empty of nameserver entries when no nameservers are available is intentional. The idea is to advertise only addresses where something is actually listening. I understand, though, (vaguely) that this is not how things work in the systemd world, so I expect that resolvconf, or the use of resolvconf, will have to be adapted for the systemd world. Help and advice will be appreciated: see bug report #700846. By the way, inconsistent with the aforementioned idea is the fact that the resolver defaults to using 127.0.0.1 if there are no nameserver lines in resolv.conf. Were you aware of that? As far as I can tell the unbound part works very well on wheezy already. Well except for ntpd not noticing. Thanks for pointing that out. I don't use unbound so I hadn't noticed that resolvconf hook scripts have been added to the unbound package. I will update the resolvconf README. -- Thomas
Re: boot ordering and resolvconf
Helmut Grohne wrote: All that I am aiming for is to declare one of the following practises as broken: * Treating an empty /etc/resolv.conf as a permanent error or failing to discover changes in /etc/resolv.conf later on. * Changing /etc/resolv.conf during startup of a local dns cache. I think that the first one is broken. If name service is unavailable just after boot then that should not be treated as a permanent error. An answer from a nameserver that a domain name does not exist might more reasonably be treated as a permanent error, but that is disputable. In a world where computers move among LANs, DNS does not always look the same, so it's not unreasonable to retry periodically even after being told NXDOMAIN. A problem is that getaddrinfo() doesn't distinguish in its return status between couldn't reach a nameserver and nameserver says the name doesn't exist. Either way it returns EAI_SYSTEM with errno=2 (ENOENT). I am not aware that there is anything (except perhaps ntpd) that needs to be fixed. It is true, that ntpd can work around this issue. Still it appears like a bad design to treat EAI_SYSTEM as a temporary error. Can you explain why you think that? -- Thomas
Re: boot ordering and resolvconf
On Fri, Jun 28, 2013 at 05:02:51PM +0200, Thomas Hood wrote: Resolvconf supports both mode A and mode B and allows switching between them. With resolvconf installed, (A) so long as a local forwarding nameserver is running, resolv.conf points to this nameserver and thus rarely changes; but (B) if no local forwarding nameserver is running and the network environment changes then resolv.conf is updated and client applications are notified via resolvconf hook scripts in /etc/resolvconf/update.d/ and /etc/resolvconf/update-libc.d/. Even clients without resolvconf hook scripts make use of the latest resolv.conf information because eglibc was enhanced two years ago to watch the mtime of /etc/resolv.conf and re-initialize the resolver state when it changes. Because of this, The issue here is not resolvconf being inflexible. The issue at hand is that different packages expect different and incompatible modes of operation. In addition resolvconf does not properly support mode (A), due to the local forwarding nameserver is running requirement. It can leave /etc/resolv.conf empty until the name server is actually started. In the context of systemd this will break socket activation as a means of avoiding dependencies and either make Debian broken (like ntpd) or incompatible (due to added dependencies) with the rest of the world. I am not aware that there is anything (except perhaps ntpd) that needs to be fixed. It is true, that ntpd can work around this issue. Still it appears like a bad design to treat EAI_SYSTEM as a temporary error. Sometimes it is, other times it is not. How many upstreams will handle this correctly and try again? Searching codesearch.d.n for EAI_SYSTEM and EAI_AGAIN can give you a rough idea. Resolvconf supports the use of dnsmasq, pdnsd and djbdns dnscache as local forwarding nameservers and reconfigures them based on input from DHCP clients, ifup and NetworkManager. Support for bind9 and unbound could easily be added (see bug report #483098). As far as I can tell the unbound part works very well on wheezy already. Well except for ntpd not noticing. Helmut -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130629060332.GA7898@localhost
Re: boot ordering and resolvconf
Ian Jackson wrote: The two designs lead to different sets of fixes. A. resolv.conf is a static file which changes only very rarely. Implications: 1. Existing DNS client applications do not need to change. 2. DNS service should always be provided at a fixed address 3. Therefore in the default installation this address should refer to localhost. Otherwise it cannot be redirected later as the network environment or installed package set changes. 4. Therefore in most installations there should be a local proxy or cache. It should use DHCP-provided, PPP-provided or similar, as a forwarder. The local DNS provider address should be owned by whatever proxy or cache is installed. B. resolv.conf is not static and may change due to network environment changes. Implications: 1. All existing DNS applications must be modified to notice changes to resolv.conf. 2. Corollary: all existing DNS resolver libraries must be so modified. 3. This will be impractical unless a common mechanism with a convenient interface (and low impact on the rest of a program) is provided. Hopefully resolvconf fits this bill. I don't know exactly how impractical B2 is. The libc's resolver is probably a hard case because of the libc's low level in the protocol stack. Can we make it aware of resolvconf ? Resolvconf supports both mode A and mode B and allows switching between them. With resolvconf installed, (A) so long as a local forwarding nameserver is running, resolv.conf points to this nameserver and thus rarely changes; but (B) if no local forwarding nameserver is running and the network environment changes then resolv.conf is updated and client applications are notified via resolvconf hook scripts in /etc/resolvconf/update.d/ and /etc/resolvconf/update-libc.d/. Even clients without resolvconf hook scripts make use of the latest resolv.conf information because eglibc was enhanced two years ago to watch the mtime of /etc/resolv.conf and re-initialize the resolver state when it changes. Because of this, I am not aware that there is anything (except perhaps ntpd) that needs to be fixed. The difficulty with plan A is probably B4. Do we have a suitable minimal proxy we can install, a way to reconfigure it based on dhcp etc., and a means to displace it when something more substantial like unbound is installed ? Resolvconf supports the use of dnsmasq, pdnsd and djbdns dnscache as local forwarding nameservers and reconfigures them based on input from DHCP clients, ifup and NetworkManager. Support for bind9 and unbound could easily be added (see bug report #483098). -- Thomas Hood resolvconf maintainers -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/51cda59b.1000...@gmail.com
Re: boot ordering and resolvconf
On Wed, 12 Jun 2013, Wouter Verhelst wrote: On 10-06-13 18:36, Ian Jackson wrote: B. resolv.conf is not static and may change due to network environment changes. Implications: 1. All existing DNS applications must be modified to notice changes to resolv.conf. 2. Corollary: all existing DNS resolver libraries must be so modified. 3. This will be impractical unless a common mechanism with a convenient interface (and low impact on the rest of a program) is provided. Hopefully resolvconf fits this bill. I don't know exactly how impractical B2 is. The libc's resolver is probably a hard case because of the libc's low level in the protocol stack. Can we make it aware of resolvconf ? You can replace it (through nsswitch.conf) by lwresd, which can easily be restarted when resolv.conf is updated. I've been using lwresd for a long time (2+ years), and it isn't very reliable. I don't think it is widely used at all, we'd need to talk to upstream about it and probably hit lwresd hard to flush out the worst bugs. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130615220446.ga10...@khazad-dum.debian.net
Re: boot ordering and resolvconf
Thanks for all the suggestions on how to implement either On Mon, Jun 10, 2013 at 05:36:46PM +0100, Ian Jackson wrote: A. resolv.conf is a static file which changes only very rarely. Implications: or B. resolv.conf is not static and may change due to network environment changes. Just how do we move from these suggestions to a coherent system design? Fact is that some applications don't cope with /etc/resolv.conf changing (i.e. A). Also fact is that in the presence of resolvconf, /etc/resolv.conf can be a dynamic file (i.e. B). The current situation is not coherent and this causes breakage (and will cause more breakage in a systemd world). Again, either option is better than leaving this undecided. There is no obviously correct choice here. Trying to file a policy bug to settle on either option will likely not work out due to disagreement later. Maybe this question should be answered by ctte? Helmut -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130612110152.GA11187@localhost
Re: boot ordering and resolvconf
]] Helmut Grohne Just how do we move from these suggestions to a coherent system design? Fact is that some applications don't cope with /etc/resolv.conf changing (i.e. A). Also fact is that in the presence of resolvconf, /etc/resolv.conf can be a dynamic file (i.e. B). The current situation is not coherent and this causes breakage (and will cause more breakage in a systemd world). I'm not sure why you think systemd changes anything here? I think we should get rid of resolvconf and other automatic processes touching /etc/resolv.conf and have a /run/resolv.conf for the case of DHCP and other automatic editing of resolv.conf, leaving settings in /etc/resolv.conf to the admin. (This is slightly tangential to the original question in the thread, but if we are going to go mucking about with resolvers, I think we should fix this wart too.) -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87k3lzzha5@qurzaw.varnish-software.com
Re: boot ordering and resolvconf
On Wed, Jun 12, 2013 at 01:42:58PM +0200, Tollef Fog Heen wrote: I'm not sure why you think systemd changes anything here? One of the main purposes of systemd is to eliminate dependencies and fulfil them with socket activation. When converting init scripts to .service files, this will likely mean dropping $named for a number services. With the current resolvconf integration of some dns caches, the /etc/resolv.conf will only be usable after the dns cache has been started. This exposes those services formerly depending on $named to the dynamic nature of /etc/resolv.conf. All that I am aiming for is to declare one of the following practises as broken: * Treating an empty /etc/resolv.conf as a permanent error or failing to discover changes in /etc/resolv.conf later on. * Changing /etc/resolv.conf during startup of a local dns cache. So if /etc/resolv.conf is declared to be static (A), then the second practise is broken. And if dynamic modifications are to be supported (B), then the first practise is broken. Helmut -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130612123436.GA12811@localhost
Re: boot ordering and resolvconf
Ian Jackson ijackson at chiark.greenend.org.uk writes: B. resolv.conf is not static and may change due to network environment changes. Implications: 1. All existing DNS applications must be modified to notice changes to resolv.conf. OpenBSD implements that: http://www.openbsd.org/cgi-bin/cvsweb/src/lib/libc/net/res_init.c#rev1.32 bye, //mirabilos -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/loom.20130611t154752-...@post.gmane.org
Re: boot ordering and resolvconf
On Tue, Jun 11, 2013 at 09:39:40AM +0800, Chow Loong Jin wrote: Shuffled the quotes: The difficulty with plan A is probably B4. Do we have a suitable minimal proxy we can install, a way to reconfigure it based on dhcp etc., and a means to displace it when something more substantial like unbound is installed ? I think that unbound actually might be that minimal proxy. When you install it along with resolvconf, it will currently place itself in /etc/resolv.conf (via resolvconf) as the sole nameserver and it will use the name servers that were reported to resolvconf from dhcp or ppp as upstream name servers. Another cache with similar capabilities in terms of resolvconf integration is pdnsd. (Note that pdnsd doesn't do DNSSEC.) For chroots we would likely need a i-know-that-there-is-a-cache package. If it helps any, NetworkManager in Ubuntu currently sets the hostname in /etc/resolv.conf to 127.0.0.1 and runs dnsmasq locally. This /etc/resolv.conf is copied into new schroots, allowing for approach A while allowing for dynamic manipulation of where the DNS queries go to. Yes. I think the experience with dnsmasq (specifically, NM + dnsmasq-base) in Ubuntu shows that this approach is a good one. Note that this is explicitly *NOT* a caching proxy. Using a local caching proxy without a per-user cache is a security issue. Strange thought: Would it be possible (on Linux) to add an iptables rule to DNAT the local dns port? That would avoid the need for a proxy altogether. Yes. A significant number of Linux-based consumer routers do this already, but I'd rather we avoid this route if possible. This makes it rather hard to do stuff like `dig @$other_server $hostname` for debugging purposes -- you'd need root access to disengage the DNAT rule first. Not for what he's described. dig @$other_server would still work just fine, you would merely have /etc/resolv.conf pointing at 127.0.0.1 and have the *kernel* handle the DNS forwarding instead of using dnsmasq or another proxy. It does have the advantage of simplicity, but there are some important cases (such as split-DNS over a VPN) that couldn't be handled in the kernel. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developerhttp://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: Digital signature
Re: boot ordering and resolvconf
On Tue, Jun 11, 2013 at 10:38:52AM -0700, Steve Langasek wrote: [...] Not for what he's described. dig @$other_server would still work just fine, you would merely have /etc/resolv.conf pointing at 127.0.0.1 and have the *kernel* handle the DNS forwarding instead of using dnsmasq or another proxy. It does have the advantage of simplicity, but there are some important cases (such as split-DNS over a VPN) that couldn't be handled in the kernel. Oh, I misunderstood what he meant. I thought he meant to hijack all outgoing DNS packets and redirect them to a server of choice. -- Kind regards, Loong Jin signature.asc Description: Digital signature
Re: boot ordering and resolvconf
On 10-06-13 18:36, Ian Jackson wrote: B. resolv.conf is not static and may change due to network environment changes. Implications: 1. All existing DNS applications must be modified to notice changes to resolv.conf. 2. Corollary: all existing DNS resolver libraries must be so modified. 3. This will be impractical unless a common mechanism with a convenient interface (and low impact on the rest of a program) is provided. Hopefully resolvconf fits this bill. I don't know exactly how impractical B2 is. The libc's resolver is probably a hard case because of the libc's low level in the protocol stack. Can we make it aware of resolvconf ? You can replace it (through nsswitch.conf) by lwresd, which can easily be restarted when resolv.conf is updated. -- This end should point toward the ground if you want to go to space. If it starts pointing toward space you are having a bad problem and you will not go to space today. -- http://xkcd.com/1133/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/51b80d06.3020...@debian.org
Re: boot ordering and resolvconf
Helmut Grohne writes (boot ordering and resolvconf): Why is this assumption problematic? A number of DNS caches (dnsmasq, pdns-server, pdnsd, and unbound) employ a technique to update /etc/resolv.conf with themselves. This updating happens after the respective cache is started. Usually any program reads /etc/resolv.conf once on the first DNS lookup. So all daemons started before the local DNS cache will either use a different server, or fail DNS resolution in all cases. A minority of services (avahi-daemon, fetchmail, postfix, sendmail, squid, and squid3) hook into resolvconf to reload their daemons when /etc/resolv.conf is changed by resolvconf. These daemons will not be affected by this problem. Many other services on the other hand will. I think this is a somewhat different problem to the one you originally state. The real problem here is that resolv.conf is changing and programs don't have the means to cope. This is the result of a conflict in the system design. Either (A) resolv.conf is a static file, changeable only rarely and with much restarting of services or rebooting. Or (B) it is a dynamic file and everyone who reads it needs the ability to detect when it changes and perhaps some kind of push-update protocol. The design used to be (A). For convenience, some programs providing DNS service have retrospectively decided that the design will be (B). What options do exist, besides adding update-libc.d scripts for each and every service? I think the first thing to do is recognise the underlying problem. To fix this problem properly we need a coherent system design. The two designs lead to different sets of fixes. A. resolv.conf is a static file which changes only very rarely. Implications: 1. Existing DNS client applications do not need to change. 2. DNS service should always be provided at a fixed address 3. Therefore in the default installation this address should refer to localhost. Otherwise it cannot be redirected later as the network environment or installed package set changes. 4. Therefore in most installations there should be a local proxy or cache. It should use DHCP-provided, PPP-provided or similar, as a forwarder. The local DNS provider address should be owned by whatever proxy or cache is installed. B. resolv.conf is not static and may change due to network environment changes. Implications: 1. All existing DNS applications must be modified to notice changes to resolv.conf. 2. Corollary: all existing DNS resolver libraries must be so modified. 3. This will be impractical unless a common mechanism with a convenient interface (and low impact on the rest of a program) is provided. Hopefully resolvconf fits this bill. I don't know exactly how impractical B2 is. The libc's resolver is probably a hard case because of the libc's low level in the protocol stack. Can we make it aware of resolvconf ? The difficulty with plan A is probably B4. Do we have a suitable minimal proxy we can install, a way to reconfigure it based on dhcp etc., and a means to displace it when something more substantial like unbound is installed ? Ian. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20918.158.894717.79...@chiark.greenend.org.uk
Re: boot ordering and resolvconf
No need to CC me here, see Mail-Followup-To. On Mon, Jun 10, 2013 at 05:36:46PM +0100, Ian Jackson wrote: I think this is a somewhat different problem to the one you originally state. The real problem here is that resolv.conf is changing and programs don't have the means to cope. Thanks for your analysis and pointing to the actual issue underneath. This is the result of a conflict in the system design. Either (A) resolv.conf is a static file, changeable only rarely and with much restarting of services or rebooting. Or (B) it is a dynamic file and everyone who reads it needs the ability to detect when it changes and perhaps some kind of push-update protocol. As far as I can see, most of what resolvconf does is assuming (B) and turning /etc/resolv.conf into a dynamic file. What would be left from resolvconf when dropping assumption (B)? A. resolv.conf is a static file which changes only very rarely. Implications: 1. Existing DNS client applications do not need to change. 2. DNS service should always be provided at a fixed address 3. Therefore in the default installation this address should refer to localhost. Otherwise it cannot be redirected later as the network environment or installed package set changes. 4. Therefore in most installations there should be a local proxy or cache. It should use DHCP-provided, PPP-provided or similar, as a forwarder. The local DNS provider address should be owned by whatever proxy or cache is installed. So here we would likely introduce a virtual package, local-dns-resolver. Likely resolvconf would depend on it. It would be provided by name servers, that bind to localhost by default. Shuffled the quotes: The difficulty with plan A is probably B4. Do we have a suitable minimal proxy we can install, a way to reconfigure it based on dhcp etc., and a means to displace it when something more substantial like unbound is installed ? I think that unbound actually might be that minimal proxy. When you install it along with resolvconf, it will currently place itself in /etc/resolv.conf (via resolvconf) as the sole nameserver and it will use the name servers that were reported to resolvconf from dhcp or ppp as upstream name servers. Another cache with similar capabilities in terms of resolvconf integration is pdnsd. (Note that pdnsd doesn't do DNSSEC.) For chroots we would likely need a i-know-that-there-is-a-cache package. Strange thought: Would it be possible (on Linux) to add an iptables rule to DNAT the local dns port? That would avoid the need for a proxy altogether. B. resolv.conf is not static and may change due to network environment changes. Implications: 1. All existing DNS applications must be modified to notice changes to resolv.conf. 2. Corollary: all existing DNS resolver libraries must be so modified. 3. This will be impractical unless a common mechanism with a convenient interface (and low impact on the rest of a program) is provided. Hopefully resolvconf fits this bill. I don't know exactly how impractical B2 is. The libc's resolver is probably a hard case because of the libc's low level in the protocol stack. Can we make it aware of resolvconf ? Actually the backend of getaddrinfo (from eglibc) already does a stat on /etc/resolv.conf for every name resolution attemptd. (Correct me if I'm wrong.) So programs will already use changed DNS servers in some cases. It should be a matter of fixing the other resolvers here. This would not solve the issue observed with ntp, because ntp treats the EAI_SYSTEM returned by getaddrinfo as a permanent error. Kurt Roeckx already indicated, that probably ntp should treat it as a temporary error. Maybe resolvconf should mention 127.0.0.1 in the absence of any other name server to avoid this situation? As far as I can tell, we mostly are in the (B) situation. Anyway clarification on which of these describes the system design to be implemented would help. Where would we document this? Would it be part of the policy? How can we test programs to determine their compliance with the chosen system design? Helmut -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130610194534.ga14...@alf.mars
Re: boot ordering and resolvconf
On Mon, Jun 10, 2013 at 09:45:35PM +0200, Helmut Grohne wrote: [...] A. resolv.conf is a static file which changes only very rarely. Implications: 1. Existing DNS client applications do not need to change. 2. DNS service should always be provided at a fixed address 3. Therefore in the default installation this address should refer to localhost. Otherwise it cannot be redirected later as the network environment or installed package set changes. 4. Therefore in most installations there should be a local proxy or cache. It should use DHCP-provided, PPP-provided or similar, as a forwarder. The local DNS provider address should be owned by whatever proxy or cache is installed. So here we would likely introduce a virtual package, local-dns-resolver. Likely resolvconf would depend on it. It would be provided by name servers, that bind to localhost by default. Shuffled the quotes: The difficulty with plan A is probably B4. Do we have a suitable minimal proxy we can install, a way to reconfigure it based on dhcp etc., and a means to displace it when something more substantial like unbound is installed ? I think that unbound actually might be that minimal proxy. When you install it along with resolvconf, it will currently place itself in /etc/resolv.conf (via resolvconf) as the sole nameserver and it will use the name servers that were reported to resolvconf from dhcp or ppp as upstream name servers. Another cache with similar capabilities in terms of resolvconf integration is pdnsd. (Note that pdnsd doesn't do DNSSEC.) For chroots we would likely need a i-know-that-there-is-a-cache package. If it helps any, NetworkManager in Ubuntu currently sets the hostname in /etc/resolv.conf to 127.0.0.1 and runs dnsmasq locally. This /etc/resolv.conf is copied into new schroots, allowing for approach A while allowing for dynamic manipulation of where the DNS queries go to. Strange thought: Would it be possible (on Linux) to add an iptables rule to DNAT the local dns port? That would avoid the need for a proxy altogether. Yes. A significant number of Linux-based consumer routers do this already, but I'd rather we avoid this route if possible. This makes it rather hard to do stuff like `dig @$other_server $hostname` for debugging purposes -- you'd need root access to disengage the DNAT rule first. This setup should only be used for administrators who want to completely forbid their users from querying other DNS servers (short of starting a tunnel somewhere else). [...] -- Kind regards, Loong Jin signature.asc Description: Digital signature
