Re: [Dnsmasq-discuss] Make dnsmasq distinguish local IPs
Yes, basically this is what's happening. The amount of addresses vary on the lo0 interface, depending on how many jails are fired up with a respective IP address, so I can't tell. Right now I have around 30 of them, but since I manage them with the aforementioned ansible scripts, they can be more or less, depending on the current situation. Adjusting the config with every jail redeployment seems overkill to me, to be honest. I'd be glad if an option that tells dnsmasq it's allowed to use a specific IP existed, against the wildcard listening socket. Cheers, -- László Károlyi http://linkedin.com/in/karolyi On 2020-07-22 14:01, Simon Kelley wrote: > I think this is the crux. > > dnsmasq is listening on the wildcard address and accepting packets which > arrive from lo. lo has address 127.0.0.20 (amongst others) and > therefore dnsmasq is deciding that queries is sends to 127.0.0.20 will > end up back at itself, and refusing to do that because it's a bad thing > to do. It doesn't know that you are gaming obscure kernel behaviour to > send 127.0.0.20 somewhere else. > > How many addresses are on lo? If it's a reasonable number, can you just > enumerate all of them _apart_ from 127.0.0.20 as listen_address configs, > and miss out the interface=lo from the config. That should do what you > want. Failing that, an except-address config, analogous to > except-interface would do the trick, but doesn't exist. :( > > Cheers, > > Simon. > > > > > On 21/07/2020 18:15, László Károlyi wrote: >> dnsmasq needs to listen on all IPs on the lo0 interface _except_ for the >> one unbound also listens on (in this case, 127.0.0.20), so that the >> jailed processes have dnsmasq to communicate with, and then dnsmasq can >> query unbound for 'outside' DNS resolution on its own jail IP. The >> latter happens via IPv6 only now, as dnsmasq refuses to use 127.0.0.20 >> with its current config, however according to sockstat, it listens on >> the wildcard interface despite its log message: >> >> USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS >> nobody dnsmasq 99396 4 udp4 *:53 *:* >> nobody dnsmasq 99396 5 tcp4 *:53 *:* >> nobody dnsmasq 99396 6 udp6 *:53 *:* >> nobody dnsmasq 99396 7 tcp6 *:53 *:* >> nobody dnsmasq 99396 10 dgram (not connected) >> >> Unbound listens on 127.0.0.20: >> >> USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS >> unbound unbound 29892 3 udp6 2a01:4f8:241:15df::32:53 *:* >> unbound unbound 29892 4 tcp6 2a01:4f8:241:15df::32:53 *:* >> unbound unbound 29892 5 udp4 127.0.0.20:53 *:* >> unbound unbound 29892 6 tcp4 127.0.0.20:53 *:* >> >> When testing, dnsmasq responds to all internal hostname queries on >> 127.0.0.x except for 127.0.0.20, so it seems to listen on all >> interfaces. FreeBSD kernel gives preference to the IP-bound >> (non-wildcard) socket when connecting to that socket for querying, see >> querying an inner jail name, jail-mariadb: >> >> # host jail-mariadb 127.0.0.1 >> Using domain server: >> Name: 127.0.0.1 >> Address: 127.0.0.1#53 >> Aliases: >> >> jail-mariadb has address 127.0.0.24 >> jail-mariadb has IPv6 address 2a01:4f8:241:15df::21 >> >> # host jail-mariadb 127.0.0.5 >> Using domain server: >> Name: 127.0.0.5 >> Address: 127.0.0.5#53 >> Aliases: >> >> jail-mariadb has address 127.0.0.24 >> jail-mariadb has IPv6 address 2a01:4f8:241:15df::21 >> >> # host jail-mariadb 127.0.0.20 >> Using domain server: >> Name: 127.0.0.20 >> Address: 127.0.0.20#53 >> Aliases: >> >> Host jail-mariadb not found: 3(NXDOMAIN) >> >> Both 127.0.0.1 and 127.0.0.5 is a response from dnsmasq, but 127.0.0.20 >> is a response from unbound. This is desired, in order for the jailed >> processes to be able to use DNS resolution from within. >> >> What I'm trying to achieve is to make dnsmasq query 127.0.0.20 knowing >> the facts above, as specified in the /usr/local/etc/dnsmasq-resolv.conf: >> >> nameserver 127.0.0.20 >> nameserver 2a01:4f8:241:15df::32 >> >> Basically, the jails talk to their own assigned internal IPs when >> querying (not 127.0.0.1, that won't work because the DNS response gets >> dropped as the response comes from the jail's internal IP and not >> 127.0.0.1), it's why dnsmasq has to listen on them. Then dnsmasq will >> talk to the unbound jail's IP address (127.0.0.20), when querying for >> outside DNS. >> >
Re: [Dnsmasq-discuss] Make dnsmasq distinguish local IPs
dnsmasq needs to listen on all IPs on the lo0 interface _except_ for the one unbound also listens on (in this case, 127.0.0.20), so that the jailed processes have dnsmasq to communicate with, and then dnsmasq can query unbound for 'outside' DNS resolution on its own jail IP. The latter happens via IPv6 only now, as dnsmasq refuses to use 127.0.0.20 with its current config, however according to sockstat, it listens on the wildcard interface despite its log message: USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS nobody dnsmasq 99396 4 udp4 *:53 *:* nobody dnsmasq 99396 5 tcp4 *:53 *:* nobody dnsmasq 99396 6 udp6 *:53 *:* nobody dnsmasq 99396 7 tcp6 *:53 *:* nobody dnsmasq 99396 10 dgram (not connected) Unbound listens on 127.0.0.20: USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS unbound unbound 29892 3 udp6 2a01:4f8:241:15df::32:53 *:* unbound unbound 29892 4 tcp6 2a01:4f8:241:15df::32:53 *:* unbound unbound 29892 5 udp4 127.0.0.20:53 *:* unbound unbound 29892 6 tcp4 127.0.0.20:53 *:* When testing, dnsmasq responds to all internal hostname queries on 127.0.0.x except for 127.0.0.20, so it seems to listen on all interfaces. FreeBSD kernel gives preference to the IP-bound (non-wildcard) socket when connecting to that socket for querying, see querying an inner jail name, jail-mariadb: # host jail-mariadb 127.0.0.1 Using domain server: Name: 127.0.0.1 Address: 127.0.0.1#53 Aliases: jail-mariadb has address 127.0.0.24 jail-mariadb has IPv6 address 2a01:4f8:241:15df::21 # host jail-mariadb 127.0.0.5 Using domain server: Name: 127.0.0.5 Address: 127.0.0.5#53 Aliases: jail-mariadb has address 127.0.0.24 jail-mariadb has IPv6 address 2a01:4f8:241:15df::21 # host jail-mariadb 127.0.0.20 Using domain server: Name: 127.0.0.20 Address: 127.0.0.20#53 Aliases: Host jail-mariadb not found: 3(NXDOMAIN) Both 127.0.0.1 and 127.0.0.5 is a response from dnsmasq, but 127.0.0.20 is a response from unbound. This is desired, in order for the jailed processes to be able to use DNS resolution from within. What I'm trying to achieve is to make dnsmasq query 127.0.0.20 knowing the facts above, as specified in the /usr/local/etc/dnsmasq-resolv.conf: nameserver 127.0.0.20 nameserver 2a01:4f8:241:15df::32 Basically, the jails talk to their own assigned internal IPs when querying (not 127.0.0.1, that won't work because the DNS response gets dropped as the response comes from the jail's internal IP and not 127.0.0.1), it's why dnsmasq has to listen on them. Then dnsmasq will talk to the unbound jail's IP address (127.0.0.20), when querying for outside DNS. Sounds complicated, but this is what I'd like to get done, so it would work with both IPv6 AND IPv4. Cheers, -- László Károlyi http://linkedin.com/in/karolyi On 2020-07-21 17:00, Petr Menšík wrote: > How should unbound listen on lo0 if dnsmasq is already listening there? > I do not know BSD. Linux would not permit dnsmasq listening on wildcard > socket and unbound listening on the same port. > > I think listen-address would listen just on 127.0.0.1. interface=lo0 > should not be necessary. At least on Linux kernel, it means listening on > ANY IPv4/IPv6 address assigned to lo0. That would mean unbound needs > different port to listen on or different interface. I think that is not > what you want. > > What is contents of /usr/local/etc/dnsmasq-resolv.conf? > I think no-resolv should be used as well to prevent reading > /etc/resolv.conf. signature.asc Description: OpenPGP digital signature ___ Dnsmasq-discuss mailing list Dnsmasq-discuss@lists.thekelleys.org.uk http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
Re: [Dnsmasq-discuss] Make dnsmasq distinguish local IPs
I've already added listen-address=127.0.0.1 to it, as it's the host env's IP address. bind-interfaces has to be commented out, otherwise the jails will have problems resolving (it's a FreeBSD host-jail resolution specific thing) Why would you want me to use except-interface=lo0? I _want_ it to listen on lo0. For the sake of clarity, here't my cleaned dnsmasq.conf: domain-needed conf-file=/usr/local/share/dnsmasq/trust-anchors.conf dnssec dnssec-check-unsigned resolv-file=/usr/local/etc/dnsmasq-resolv.conf interface=lo0 listen-address=127.0.0.1 no-dhcp-interface=lo0 local-ttl=5 dhcp-name-match=set:wpad-ignore,wpad dhcp-ignore-names=tag:wpad-ignore rebind-domain-ok=/rfc-ignorant.org/sorbs.net/uribl.com/surbl.org/dnswl.org/njabl.org/spamhaus.org/spamcop.net/barracudacentral.org/ Cheers, -- László Károlyi http://linkedin.com/in/karolyi On 2020-07-21 14:42, Petr Menšík wrote: > I would check what addresses it is listening on. I think it considers > all loopback addresses its own. Probably because it would accept queries > to that address if you stop unbound. > > It might help, if you configured it with this: > bind-interfaces > except-interface=lo0 > listen-address=127.0.0.21 > > It would listen only on 127.0.0.21 and consider all other addresses not > its own. I think it should send queries there. It should then accept: > server=127.0.0.20 > without ignoring it this way. > > On 7/20/20 4:35 PM, László Károlyi wrote: >> Hi Petr, >> >> as you have seen in the original email, it is dnsmasq that refuses to >> use the lo0 interface to communicate with the IP 127.0.0.20: >> >> Jul 20 13:33:23 ksol dnsmasq[99396]: ignoring nameserver 127.0.0.20 - >> local interface >> >> When querying manually from the host env to the jailed unbound, I get >> proper DNS responses. This was something I did pay extra attention to >> get it working from the get-go. See: >> >> Citing my configs here makes no sense as you can see it's working already. >> >> Cheers, >> -- >> László Károlyi >> http://linkedin.com/in/karolyi >> >> On 2020-07-20 16:12, Petr Menšík wrote: >>> Hi László, >>> >>> are you sure it is dnsmasq, who is rejecting the communication? >>> Unbound has by default disabled commuinication on localhost. If you have >>> any other servers running along it, you have to use: >>> >>> do-not-query-localhost: no >>> >>> to override defaults. But that has to be done on unbound side. AFAIK >>> dnsmasq does not have any such limitation. It does limit only >>> per-interface, all required is to configure interface=lo, which is >>> enabled by default. >>> >>> How many interface= statements do you have in configuration? Is >>> localhost included? >> ___ >> Dnsmasq-discuss mailing list >> Dnsmasq-discuss@lists.thekelleys.org.uk >> http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss >> > > ___ > Dnsmasq-discuss mailing list > Dnsmasq-discuss@lists.thekelleys.org.uk > http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss signature.asc Description: OpenPGP digital signature ___ Dnsmasq-discuss mailing list Dnsmasq-discuss@lists.thekelleys.org.uk http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
Re: [Dnsmasq-discuss] Make dnsmasq distinguish local IPs
Hi Petr, as you have seen in the original email, it is dnsmasq that refuses to use the lo0 interface to communicate with the IP 127.0.0.20: Jul 20 13:33:23 ksol dnsmasq[99396]: ignoring nameserver 127.0.0.20 - local interface When querying manually from the host env to the jailed unbound, I get proper DNS responses. This was something I did pay extra attention to get it working from the get-go. See: Citing my configs here makes no sense as you can see it's working already. Cheers, -- László Károlyi http://linkedin.com/in/karolyi On 2020-07-20 16:12, Petr Menšík wrote: > Hi László, > > are you sure it is dnsmasq, who is rejecting the communication? > Unbound has by default disabled commuinication on localhost. If you have > any other servers running along it, you have to use: > > do-not-query-localhost: no > > to override defaults. But that has to be done on unbound side. AFAIK > dnsmasq does not have any such limitation. It does limit only > per-interface, all required is to configure interface=lo, which is > enabled by default. > > How many interface= statements do you have in configuration? Is > localhost included? signature.asc Description: OpenPGP digital signature ___ Dnsmasq-discuss mailing list Dnsmasq-discuss@lists.thekelleys.org.uk http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
[Dnsmasq-discuss] Make dnsmasq distinguish local IPs
Hey Simon, First of all, thanks again for fixing my DNSSEC issue. So as I said before, here's my feature request. I have a FreeBSD box that has multiple local IP addresses on the local 'lo0' interface, used by jails as their IPv4 interface address. Those IPs vary somewhere in the 127.0.0.x range. The jails use those addresses as their IPv4 addresses to communicate with the outside world, while being NAT-ed on the only available external IPv4 address. IPv6-wise, I have a bridge0 interface that handles the many different addresses assigned to my box, each assigned to one jail each. Unbound runs in a jail and thus I've told dnsmasq to communicate with either the IPv4 127.0.0.x address, or the IPv6 address of the jail when looking up DNS records. When starting dnsmasq on the 'host' environment (it's the only service other than syslog I run in the host environment), dnsmasq refuses to communicate with the IPv4 address of the jailed unbound, claiming it's a 'local' address: Jul 20 13:33:23 ksol dnsmasq[99396]: ignoring nameserver 127.0.0.20 - local interface Whereas it's indeed a 'local' interface, it could be used for IPv4 communication because of the mentioned reasons above. Because of this, dnsmasq is now only able to communicate through IPv6 with unbound, but should I lose IPv6 support (unlikely but one never knows), I'd lose dnsmasq and the internal name resolution between the jails, which now it is able to support. So my request would be to fix this functionality and make dnsmasq able to differentiate between local IPs, in order to be able to use them for DNS resolution. Cheers, -- László Károlyi http://linkedin.com/in/karolyi signature.asc Description: OpenPGP digital signature ___ Dnsmasq-discuss mailing list Dnsmasq-discuss@lists.thekelleys.org.uk http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
Re: [Dnsmasq-discuss] BOGUS DNSSEC responses
Thanks for your response again. I'm not an expert in DNSSEC, so I can't answer you the first point. As for the second point, I attached my (pretty milktoast) unbound.conf, not much changes in there; hoping it could give a clue. Edit: Resending the unbound.conf zipped since the unzipped version it got held up by mailman. Cheers, -- László Károlyi https://linkedin/com/in/karolyi On 06.07.20 23:05, Simon Kelley wrote: > OK, I can see the proximate cause of the problem, but I'm not sure > what's causing it and I'm not sure how behaviour needs to change. > > The proximate cause is that the upstream server (unbound, I think.) is > returning answers to queries for DNSKEY records with time-to-live as > zero. Time-to-live zero means "use this once, but don't cache it" so > dnsmasq doesn't cache it. But the DNSSEC validation process in dnsmasq > depends on data like DNSKEYs being cached: that's the path by which it > gets to the correct place for doing the validation. Hence the validation > failures. > > Two questions arise. > > 1) Is dnsmasq wrong to fail validation with DNSKEYS with TTL zero. I > think that answer to that is probably "yes", if only on grounds of "be > forgiving in what you accept". The fix is fairly simple. > > 2) Why is Unbound returning DNSKEY records with TTL zero, over and over > again? Is there something in your unbound config that causes that? > > > Cheers, > > Simon. <> signature.asc Description: OpenPGP digital signature ___ Dnsmasq-discuss mailing list Dnsmasq-discuss@lists.thekelleys.org.uk http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
Re: [Dnsmasq-discuss] BOGUS DNSSEC responses
So, this was done faster as I thought. I've uploaded the file to wetransfer since it's 3MB and I don't want an outcry from people on here about me sending huge emails: https://we.tl/t-mlLySN7n0f In that dump, you will probably see obsswitcher.com, updates.spamassassin.org and api.foursquare.com failed requests. I didn't look into it in detail, but the sheer size should indicate a lot of failed requests. Cheers, -- László Károlyi http://linkedin.com/in/karolyi On 2020-07-06 00:41, László Károlyi wrote: > Hey Simon, > > thanks for your response. > > Yes, my bad, I should have said at the outset that I use the latest > dsmasq in FreeBSD with the latest official (12.1-RELEASE-p6) release on > the latest patch level. So, dnsmasq is "2.81_2,1" , as defined here: > > https://www.freshports.org/dns/dnsmasq/ > > I use NTP to keep the time in sync on my box, the output of ntpq -n -p is: > > remote refid st t when poll reach delay offset > jitter > == > 0.freebsd.pool. .POOL. 16 p - 64 0 0.000 +0.000 > 0.000 > -162.159.200.1 10.71.10.44 3 u 49 1024 377 5.077 -2.686 > 0.288 > *193.158.22.13 .MBGh. 1 u 217 1024 377 11.921 -1.232 > 0.298 > +85.209.49.104 35.73.197.144 2 u 81 1024 377 2.780 -0.842 > 0.242 > +185.120.22.12 130.149.17.21 2 u 384 1024 377 5.404 -0.482 > 0.384 > -212.18.3.19 212.18.1.106 2 u 122 1024 377 6.106 -1.292 > 0.260 > > Basically as you can see, no egregious time differences (delay is in > milliseconds). As for the domains, my domain is kept in cloudflare, they > provide the DNSSEC records as well. I don't know if that's the case for > github and/or updates.spamassassin.org, which I also see failing. > > I'll set the flags and logfile you provided, and will wait until the > error occurs again, and then I'll touch base again with you. It should > take a day or two at most, the sometimes failing cronjob runs hourly. > > Best Regards, > -- > László Károlyi > https://linkedin/com/in/karolyi > > On 05.07.20 23:17, Simon Kelley wrote: >> Just a stab in the dark: are you sure that the clocks on these machines >> are accurate? DNSSEC signatures have validity periods and when I checked >> obsswitcher.com its start-of-validity time was only an hour or so before >> the time when I checked, so a bad clock would explain what you're seeing. >> >> Failing that, you don't say what version of dnsmasq you're running. >> PLease make sure you upgrade to 2.81 if you're running older code. That >> fixes lots of DNSSEC bugs. >> >> If 2.81 still shows the problem, set the following dnsmasq configuration >> >> dumpfile= >> dumpmask=0x00C0 >> >> run the test again and send me the resulting dumps. >> >> >> Cheers, >> >> Simon. >> >> >> ___ >> Dnsmasq-discuss mailing list >> Dnsmasq-discuss@lists.thekelleys.org.uk >> http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss signature.asc Description: OpenPGP digital signature ___ Dnsmasq-discuss mailing list Dnsmasq-discuss@lists.thekelleys.org.uk http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
Re: [Dnsmasq-discuss] BOGUS DNSSEC responses
Hey Simon, thanks for your response. Yes, my bad, I should have said at the outset that I use the latest dsmasq in FreeBSD with the latest official (12.1-RELEASE-p6) release on the latest patch level. So, dnsmasq is "2.81_2,1" , as defined here: https://www.freshports.org/dns/dnsmasq/ I use NTP to keep the time in sync on my box, the output of ntpq -n -p is: remote refid st t when poll reach delay offset jitter == 0.freebsd.pool. .POOL. 16 p - 64 0 0.000 +0.000 0.000 -162.159.200.1 10.71.10.44 3 u 49 1024 377 5.077 -2.686 0.288 *193.158.22.13 .MBGh. 1 u 217 1024 377 11.921 -1.232 0.298 +85.209.49.104 35.73.197.144 2 u 81 1024 377 2.780 -0.842 0.242 +185.120.22.12 130.149.17.21 2 u 384 1024 377 5.404 -0.482 0.384 -212.18.3.19 212.18.1.106 2 u 122 1024 377 6.106 -1.292 0.260 Basically as you can see, no egregious time differences (delay is in milliseconds). As for the domains, my domain is kept in cloudflare, they provide the DNSSEC records as well. I don't know if that's the case for github and/or updates.spamassassin.org, which I also see failing. I'll set the flags and logfile you provided, and will wait until the error occurs again, and then I'll touch base again with you. It should take a day or two at most, the sometimes failing cronjob runs hourly. Best Regards, -- László Károlyi https://linkedin/com/in/karolyi On 05.07.20 23:17, Simon Kelley wrote: > Just a stab in the dark: are you sure that the clocks on these machines > are accurate? DNSSEC signatures have validity periods and when I checked > obsswitcher.com its start-of-validity time was only an hour or so before > the time when I checked, so a bad clock would explain what you're seeing. > > Failing that, you don't say what version of dnsmasq you're running. > PLease make sure you upgrade to 2.81 if you're running older code. That > fixes lots of DNSSEC bugs. > > If 2.81 still shows the problem, set the following dnsmasq configuration > > dumpfile= > dumpmask=0x00C0 > > run the test again and send me the resulting dumps. > > > Cheers, > > Simon. > signature.asc Description: OpenPGP digital signature ___ Dnsmasq-discuss mailing list Dnsmasq-discuss@lists.thekelleys.org.uk http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
[Dnsmasq-discuss] BOGUS DNSSEC responses
Hey, I have a FreeBSD box where jails communicate with dnsmasq outside to resolve each other's addresses (they get different IPs on redeployments), and dnsmasq communicates with unbound where it needs to resolve outside domains. When running stuff from cron within the jails, sometimes hostnames don't resolve and I started to investigate on the problem by turning debug log on with dnsmasq. As it turns out, it complains about domain DNSSEC errors, where they are properly configured. This happens with my domain (attached in the logs), and outher domains (github, updates.spamassassin.org) as well. I'm somewhat clueless as to why it happens, so please see the log attached, with my own domain, obsswitcher.com. What happens here is, I've set up a cronjob with curl to run until it succeeds, that is: while true; do curl -s 'https://obsswitcher.com/' && break || date; done Sometimes hostname resolution succeeds at first time, sometimes it takes 200+ tries until it succeeds once, and quits. The attached log is the one where it happened 200+ times before succeeding. Any help is appreciated. Cheers, -- László Károlyi http://linkedin.com/in/karolyi <> signature.asc Description: OpenPGP digital signature ___ Dnsmasq-discuss mailing list Dnsmasq-discuss@lists.thekelleys.org.uk http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
