[Dnsmasq-discuss] server forwarding all traffic to parents after a successful PTR query of itself

2010-02-17 Thread Alberto Cuesta-Canada
Hi guys,
 
I saw a weird scenario in one of our dnsmasq servers yesterday. As the logs 
below show, the server was all happy doing its thing, until a set of PTR 
queries came from normal servers in our network. The last of it would ask for 
the hostname of the dns server giving the IP, and from that point dnsmasq would 
route all traffic to the parents. Restarting the dnsmasq service would restore 
the server to normal operations. This has happened 4 times in the last 10 days, 
always with the same pattern.
 
Feb 17 01:35:51 dnsmasq[28538]: query[A] grdvpm3.dselgrid.local from 
172.30.158.98

Feb 17 01:35:51 dnsmasq[28538]: /etc/hosts grdvpm3.dselgrid.local is 
172.30.158.93

Feb 17 01:35:51 dnsmasq[28538]: query[PTR] 93.158.30.172.in-addr.arpa from 
172.30.158.98

Feb 17 01:35:51 dnsmasq[28538]: /etc/hosts 172.30.158.93 is 
grdvpm3.dselgrid.local

Feb 17 01:35:51 dnsmasq[28538]: query[A] grdvpm3.dselgrid.local from 
172.30.158.98

Feb 17 01:35:51 dnsmasq[28538]: /etc/hosts grdvpm3.dselgrid.local is 
172.30.158.93

Feb 17 01:37:16 dnsmasq[28538]: query[MX] smtpmail.daiwaeurope.local from 
127.0.0.1

Feb 17 01:37:16 dnsmasq[28538]: forwarded smtpmail.daiwaeurope.local to 
172.30.48.192

Feb 17 01:37:16 dnsmasq[28538]: query[MX] vsmtpmail.daiwaeurope.local from 
127.0.0.1

Feb 17 01:37:16 dnsmasq[28538]: forwarded vsmtpmail.daiwaeurope.local to 
172.30.48.192

Feb 17 01:37:16 dnsmasq[28538]: query[A] smtpmail.daiwaeurope.local from 
127.0.0.1

Feb 17 01:37:16 dnsmasq[28538]: forwarded smtpmail.daiwaeurope.local to 
172.30.48.192

Feb 17 01:37:16 dnsmasq[28538]: reply smtpmail.daiwaeurope.local is CNAME

Feb 17 01:37:16 dnsmasq[28538]: reply vsmtpmail.daiwaeurope.local is 
172.30.19.221

Feb 17 01:37:52 dnsmasq[28538]: query[PTR] 250.158.30.172.in-addr.arpa from 
172.30.158.94

Feb 17 01:37:52 dnsmasq[28538]: /etc/hosts 172.30.158.250 is 
grdxk-mgmt1.dselgrid.local

Feb 17 01:37:52 dnsmasq[28538]: forwarded query to 172.30.48.192

Feb 17 01:37:52 dnsmasq[28538]: forwarded query to 172.30.48.192

Feb 17 01:37:52 dnsmasq[28538]: forwarded query to 172.30.48.192

Feb 17 01:37:52 dnsmasq[28538]: forwarded query to 172.30.48.192

Feb 17 01:37:52 dnsmasq[28538]: forwarded query to 172.30.48.192

Any idea what would be going on? Is that PTR query a signal that some other 
service could be asking the DNS server to stop reading the hosts file?
 
Many thanks,
 
Alberto Cuesta-Canada
GaaS Team Lead
Excelian Ltd.
+44 (0) 7942633361

The information contained in this email and any attached files are confidential 
and intended solely for the addressee(s). The email may be legally privileged 
or prohibited from disclosure and unauthorised use. If you are not the named 
addressee you may not use, copy, or disclose this information to any other 
person. If you received this message in error please notify the sender 
immediately and delete it from your system. 

Any opinion or views contained in this email message are those of the sender, 
and do not represent those of the Company in any way and reliance should not be 
placed upon its contents. Unless otherwise stated, this email message is not 
intended to be contractually binding. Where an Agreement exists between our 
respective companies and there is conflict between the contents of this email 
message and the Agreement then the terms of that Agreement shall prevail.

Excelian
50 Featherstone Street
London
EC1Y 8RT
Tel: +44 (0) 20 7336 9595
Fax: +44 (0) 20 7336 9596
www.Excelian.com
_
This e-mail has been scanned for viruses by MessageLabs. For further 
information visit http://www.messagelabs.com

Excelian subscribes to cleaner and greener methods of working. Help take 
responsibility for the environment. Please don't print this email unless you 
absolutely have to.

Re: [Dnsmasq-discuss] server forwarding all traffic to parents after a successful PTR query of itself

2010-02-17 Thread Simon Kelley

Alberto Cuesta-Canada wrote:

Hi guys,
 
I saw a weird scenario in one of our dnsmasq servers yesterday. As the 
logs below show, the server was all happy doing its thing, until a set 
of PTR queries came from normal servers in our network. The last of it 
would ask for the hostname of the dns server giving the IP, and from 
that point dnsmasq would route all traffic to the parents. Restarting 
the dnsmasq service would restore the server to normal operations. This 
has happened 4 times in the last 10 days, always with the same pattern.
 

Feb 17 01:35:51 dnsmasq[28538]: query[A] grdvpm3.dselgrid.local from 
172.30.158.98


Feb 17 01:35:51 dnsmasq[28538]: /etc/hosts grdvpm3.dselgrid.local is 
172.30.158.93


Feb 17 01:35:51 dnsmasq[28538]: query[PTR] 93.158.30.172.in-addr.arpa 
from 172.30.158.98


Feb 17 01:35:51 dnsmasq[28538]: /etc/hosts 172.30.158.93 is 
grdvpm3.dselgrid.local


Feb 17 01:35:51 dnsmasq[28538]: query[A] grdvpm3.dselgrid.local from 
172.30.158.98


Feb 17 01:35:51 dnsmasq[28538]: /etc/hosts grdvpm3.dselgrid.local is 
172.30.158.93


Feb 17 01:37:16 dnsmasq[28538]: query[MX] smtpmail.daiwaeurope.local 
from 127.0.0.1


Feb 17 01:37:16 dnsmasq[28538]: forwarded smtpmail.daiwaeurope.local to 
172.30.48.192


Feb 17 01:37:16 dnsmasq[28538]: query[MX] vsmtpmail.daiwaeurope.local 
from 127.0.0.1


Feb 17 01:37:16 dnsmasq[28538]: forwarded vsmtpmail.daiwaeurope.local to 
172.30.48.192


Feb 17 01:37:16 dnsmasq[28538]: query[A] smtpmail.daiwaeurope.local from 
127.0.0.1


Feb 17 01:37:16 dnsmasq[28538]: forwarded smtpmail.daiwaeurope.local to 
172.30.48.192


Feb 17 01:37:16 dnsmasq[28538]: reply smtpmail.daiwaeurope.local is CNAME

Feb 17 01:37:16 dnsmasq[28538]: reply vsmtpmail.daiwaeurope.local is 
172.30.19.221


Feb 17 01:37:52 dnsmasq[28538]: query[PTR] 250.158.30.172.in-addr.arpa 
from 172.30.158.94


Feb 17 01:37:52 dnsmasq[28538]: /etc/hosts 172.30.158.250 is 
grdxk-mgmt1.dselgrid.local


Feb 17 01:37:52 dnsmasq[28538]: forwarded query to 172.30.48.192

Feb 17 01:37:52 dnsmasq[28538]: forwarded query to 172.30.48.192

Feb 17 01:37:52 dnsmasq[28538]: forwarded query to 172.30.48.192

Feb 17 01:37:52 dnsmasq[28538]: forwarded query to 172.30.48.192

Feb 17 01:37:52 dnsmasq[28538]: forwarded query to 172.30.48.192

Any idea what would be going on? Is that PTR query a signal that some 
other service could be asking the DNS server to stop reading the hosts file?
 


Which version of dnsmasq are you using?

Simon.



Re: [Dnsmasq-discuss] server forwarding all traffic to parents after a successful PTR query of itself

2010-02-17 Thread Alberto Cuesta-Canada
Hi Simon,
 
2.47
 
Cheers,
 
Alberto Cuesta-Canada
GaaS Team Lead
Excelian Ltd.
+44 (0) 7942633361



From: Simon Kelley [mailto:si...@thekelleys.org.uk]
Sent: Wed 17/02/2010 09:46
To: Alberto Cuesta-Canada
Cc: dnsmasq-discuss@lists.thekelleys.org.uk; Grid Support
Subject: Re: [Dnsmasq-discuss] server forwarding all traffic to parents after a 
successful PTR query of itself



Alberto Cuesta-Canada wrote:
 Hi guys,
 
 I saw a weird scenario in one of our dnsmasq servers yesterday. As the
 logs below show, the server was all happy doing its thing, until a set
 of PTR queries came from normal servers in our network. The last of it
 would ask for the hostname of the dns server giving the IP, and from
 that point dnsmasq would route all traffic to the parents. Restarting
 the dnsmasq service would restore the server to normal operations. This
 has happened 4 times in the last 10 days, always with the same pattern.
 

 Feb 17 01:35:51 dnsmasq[28538]: query[A] grdvpm3.dselgrid.local from
 172.30.158.98

 Feb 17 01:35:51 dnsmasq[28538]: /etc/hosts grdvpm3.dselgrid.local is
 172.30.158.93

 Feb 17 01:35:51 dnsmasq[28538]: query[PTR] 93.158.30.172.in-addr.arpa
 from 172.30.158.98

 Feb 17 01:35:51 dnsmasq[28538]: /etc/hosts 172.30.158.93 is
 grdvpm3.dselgrid.local

 Feb 17 01:35:51 dnsmasq[28538]: query[A] grdvpm3.dselgrid.local from
 172.30.158.98

 Feb 17 01:35:51 dnsmasq[28538]: /etc/hosts grdvpm3.dselgrid.local is
 172.30.158.93

 Feb 17 01:37:16 dnsmasq[28538]: query[MX] smtpmail.daiwaeurope.local
 from 127.0.0.1

 Feb 17 01:37:16 dnsmasq[28538]: forwarded smtpmail.daiwaeurope.local to
 172.30.48.192

 Feb 17 01:37:16 dnsmasq[28538]: query[MX] vsmtpmail.daiwaeurope.local
 from 127.0.0.1

 Feb 17 01:37:16 dnsmasq[28538]: forwarded vsmtpmail.daiwaeurope.local to
 172.30.48.192

 Feb 17 01:37:16 dnsmasq[28538]: query[A] smtpmail.daiwaeurope.local from
 127.0.0.1

 Feb 17 01:37:16 dnsmasq[28538]: forwarded smtpmail.daiwaeurope.local to
 172.30.48.192

 Feb 17 01:37:16 dnsmasq[28538]: reply smtpmail.daiwaeurope.local is CNAME

 Feb 17 01:37:16 dnsmasq[28538]: reply vsmtpmail.daiwaeurope.local is
 172.30.19.221

 Feb 17 01:37:52 dnsmasq[28538]: query[PTR] 250.158.30.172.in-addr.arpa
 from 172.30.158.94

 Feb 17 01:37:52 dnsmasq[28538]: /etc/hosts 172.30.158.250 is
 grdxk-mgmt1.dselgrid.local

 Feb 17 01:37:52 dnsmasq[28538]: forwarded query to 172.30.48.192

 Feb 17 01:37:52 dnsmasq[28538]: forwarded query to 172.30.48.192

 Feb 17 01:37:52 dnsmasq[28538]: forwarded query to 172.30.48.192

 Feb 17 01:37:52 dnsmasq[28538]: forwarded query to 172.30.48.192

 Feb 17 01:37:52 dnsmasq[28538]: forwarded query to 172.30.48.192

 Any idea what would be going on? Is that PTR query a signal that some
 other service could be asking the DNS server to stop reading the hosts file?
 

Which version of dnsmasq are you using?

Simon.



The information contained in this email and any attached files are confidential 
and intended solely for the addressee(s). The email may be legally privileged 
or prohibited from disclosure and unauthorised use. If you are not the named 
addressee you may not use, copy, or disclose this information to any other 
person. If you received this message in error please notify the sender 
immediately and delete it from your system. 

Any opinion or views contained in this email message are those of the sender, 
and do not represent those of the Company in any way and reliance should not be 
placed upon its contents. Unless otherwise stated, this email message is not 
intended to be contractually binding. Where an Agreement exists between our 
respective companies and there is conflict between the contents of this email 
message and the Agreement then the terms of that Agreement shall prevail.

Excelian
50 Featherstone Street
London
EC1Y 8RT
Tel: +44 (0) 20 7336 9595
Fax: +44 (0) 20 7336 9596
www.Excelian.com
_
This e-mail has been scanned for viruses by MessageLabs. For further 
information visit http://www.messagelabs.com

Excelian subscribes to cleaner and greener methods of working. Help take 
responsibility for the environment. Please don't print this email unless you 
absolutely have to.

Re: [Dnsmasq-discuss] server forwarding all traffic to parents after a successful PTR query of itself

2010-02-17 Thread Simon Kelley

Alberto Cuesta-Canada wrote:

Hi guys,
 
I saw a weird scenario in one of our dnsmasq servers yesterday. As the 
logs below show, the server was all happy doing its thing, until a set 
of PTR queries came from normal servers in our network. The last of it 
would ask for the hostname of the dns server giving the IP, and from 
that point dnsmasq would route all traffic to the parents. Restarting 
the dnsmasq service would restore the server to normal operations. This 
has happened 4 times in the last 10 days, always with the same pattern.
 

Feb 17 01:35:51 dnsmasq[28538]: query[A] grdvpm3.dselgrid.local from 
172.30.158.98


Feb 17 01:35:51 dnsmasq[28538]: /etc/hosts grdvpm3.dselgrid.local is 
172.30.158.93


Feb 17 01:35:51 dnsmasq[28538]: query[PTR] 93.158.30.172.in-addr.arpa 
from 172.30.158.98


Feb 17 01:35:51 dnsmasq[28538]: /etc/hosts 172.30.158.93 is 
grdvpm3.dselgrid.local


Feb 17 01:35:51 dnsmasq[28538]: query[A] grdvpm3.dselgrid.local from 
172.30.158.98


Feb 17 01:35:51 dnsmasq[28538]: /etc/hosts grdvpm3.dselgrid.local is 
172.30.158.93


Feb 17 01:37:16 dnsmasq[28538]: query[MX] smtpmail.daiwaeurope.local 
from 127.0.0.1


Feb 17 01:37:16 dnsmasq[28538]: forwarded smtpmail.daiwaeurope.local to 
172.30.48.192


Feb 17 01:37:16 dnsmasq[28538]: query[MX] vsmtpmail.daiwaeurope.local 
from 127.0.0.1


Feb 17 01:37:16 dnsmasq[28538]: forwarded vsmtpmail.daiwaeurope.local to 
172.30.48.192


Feb 17 01:37:16 dnsmasq[28538]: query[A] smtpmail.daiwaeurope.local from 
127.0.0.1


Feb 17 01:37:16 dnsmasq[28538]: forwarded smtpmail.daiwaeurope.local to 
172.30.48.192


Feb 17 01:37:16 dnsmasq[28538]: reply smtpmail.daiwaeurope.local is CNAME

Feb 17 01:37:16 dnsmasq[28538]: reply vsmtpmail.daiwaeurope.local is 
172.30.19.221


Feb 17 01:37:52 dnsmasq[28538]: query[PTR] 250.158.30.172.in-addr.arpa 
from 172.30.158.94


Feb 17 01:37:52 dnsmasq[28538]: /etc/hosts 172.30.158.250 is 
grdxk-mgmt1.dselgrid.local


Feb 17 01:37:52 dnsmasq[28538]: forwarded query to 172.30.48.192

Feb 17 01:37:52 dnsmasq[28538]: forwarded query to 172.30.48.192

Feb 17 01:37:52 dnsmasq[28538]: forwarded query to 172.30.48.192

Feb 17 01:37:52 dnsmasq[28538]: forwarded query to 172.30.48.192

Feb 17 01:37:52 dnsmasq[28538]: forwarded query to 172.30.48.192

Any idea what would be going on? Is that PTR query a signal that some 
other service could be asking the DNS server to stop reading the hosts file?
 



It's not clear to me what is going on here. How does the pattern 
continue? Do you just see  forwarded query to 172.30.48.192 from now 
on until the server is restarted, or do you still see query[A] and 
query[PTR} lines?


Do queries which get pushed upstream continue to work? How about queries 
which should be answered locally?


What is 172.30.158.94? Is it running anything that may generate odd 
DNS queries? The holy grail would be to able prod that machine to 
reproduce this at will.


What sort of machine are you running dnsmasq on? Does it have a 
reasonable amount of spare storage so that you could tcpdump all traffic 
to/from port 53,UDP for offline analysis?



Simon.



Re: [Dnsmasq-discuss] server forwarding all traffic to parents after a successful PTR query of itself

2010-02-17 Thread Alberto Cuesta-Canada
Hi Simon,
 
the parents of 250 (my dnsmasq server) have forwarding rules for the 
dselgrid.local domain, that I run. So I assumed that the queries pushed 
upstream would be routed down again, and timeout in a loop.
 
That said, in the logs I could still see successful PTR and A queries, 
outnumbered 10 to 1 by forwards. I'm not sure about the behaviour of local 
queries, I don't remember from yesterday, but I think they worked.
 
94 is a Platform Grid Master, that is a W2K3 machine which runs only one 
application. It keeps a cache of machines but it doesn't give DNS services, or 
anything similar. The interesting thing is that the PTR request doesn't always 
produce this effect. I have enterprise support for that software, so I will ask 
them.
 
dnsmasq is running in a quite complicated setup. We have a XenServer host 
running a Ubuntu 9.04 VM. I have just 1GB free on that machine and out of disk 
space scenarios are fatal, so I can't tcpdump. There is a rebuild of it coming 
in the next two weeks that will give me another 50GB.
 
Any idea on what to look for, or any hypothesis of what could be happening 
should be enough, I can keep investigating and contain it with workarounds for 
a while.
 
Many thanks,
 
Alberto Cuesta-Canada
GaaS Team Lead
Excelian Ltd.
+44 (0) 7942633361



From: Simon Kelley [mailto:si...@thekelleys.org.uk]
Sent: Wed 17/02/2010 10:04
To: Alberto Cuesta-Canada
Cc: dnsmasq-discuss@lists.thekelleys.org.uk; Grid Support
Subject: Re: [Dnsmasq-discuss] server forwarding all traffic to parents after a 
successful PTR query of itself



It's not clear to me what is going on here. How does the pattern
continue? Do you just see  forwarded query to 172.30.48.192 from now
on until the server is restarted, or do you still see query[A] and
query[PTR} lines?

Do queries which get pushed upstream continue to work? How about queries
which should be answered locally?

What is 172.30.158.94? Is it running anything that may generate odd
DNS queries? The holy grail would be to able prod that machine to
reproduce this at will.

What sort of machine are you running dnsmasq on? Does it have a
reasonable amount of spare storage so that you could tcpdump all traffic
to/from port 53,UDP for offline analysis?


Simon.



The information contained in this email and any attached files are confidential 
and intended solely for the addressee(s). The email may be legally privileged 
or prohibited from disclosure and unauthorised use. If you are not the named 
addressee you may not use, copy, or disclose this information to any other 
person. If you received this message in error please notify the sender 
immediately and delete it from your system. 

Any opinion or views contained in this email message are those of the sender, 
and do not represent those of the Company in any way and reliance should not be 
placed upon its contents. Unless otherwise stated, this email message is not 
intended to be contractually binding. Where an Agreement exists between our 
respective companies and there is conflict between the contents of this email 
message and the Agreement then the terms of that Agreement shall prevail.

Excelian
50 Featherstone Street
London
EC1Y 8RT
Tel: +44 (0) 20 7336 9595
Fax: +44 (0) 20 7336 9596
www.Excelian.com
_
This e-mail has been scanned for viruses by MessageLabs. For further 
information visit http://www.messagelabs.com

Excelian subscribes to cleaner and greener methods of working. Help take 
responsibility for the environment. Please don't print this email unless you 
absolutely have to.

Re: [Dnsmasq-discuss] server forwarding all traffic to parents after a successful PTR query of itself

2010-02-17 Thread Simon Kelley

Alberto Cuesta-Canada wrote:

Hi Simon,
 
the parents of 250 (my dnsmasq server) have forwarding rules for the 
dselgrid.local domain, that I run. So I assumed that the queries pushed 
upstream would be routed down again, and timeout in a loop.


Ahh, that could easily be the problem. If you generate a loop between 
two DNS servers, each forwarding to the other, then the queries can 
easily bounce back-and-forth forever. Dnsmasq will manage this situation 
reasonably well, and manage to server other traffic, but the circulating 
queries will eat bandwidth and CPU.


The logs would seem to show a strange query of some sort (dnsmasq 
can't parse a domain-name from the query, hence forwarded query rather 
than forwarded domain-name) If such queries can circulate forever 
then you have a problem.



 
That said, in the logs I could still see successful PTR and A queries, 
outnumbered 10 to 1 by forwards. I'm not sure about the behaviour of 
local queries, I don't remember from yesterday, but I think they worked.
 
94 is a Platform Grid Master, that is a W2K3 machine which runs only one 
application. It keeps a cache of machines but it doesn't give DNS 
services, or anything similar. The interesting thing is that the PTR 
request doesn't always produce this effect. I have enterprise support 
for that software, so I will ask them.
 
dnsmasq is running in a quite complicated setup. We have a XenServer 
host running a Ubuntu 9.04 VM. I have just 1GB free on that machine and 
out of disk space scenarios are fatal, so I can't tcpdump. There is a 
rebuild of it coming in the next two weeks that will give me another 50GB.
 
Any idea on what to look for, or any hypothesis of what could be 
happening should be enough, I can keep investigating and contain it with 
workarounds for a while.


See above. a loop, possibly only of odd queries.

Cheers,

Simon.

 
Many thanks,
 
*Alberto Cuesta-Canada*

GaaS Team Lead
Excelian Ltd.
+44 (0) 7942633361


*From:* Simon Kelley [mailto:si...@thekelleys.org.uk]
*Sent:* Wed 17/02/2010 10:04
*To:* Alberto Cuesta-Canada
*Cc:* dnsmasq-discuss@lists.thekelleys.org.uk; Grid Support
*Subject:* Re: [Dnsmasq-discuss] server forwarding all traffic to 
parents after a successful PTR query of itself


It's not clear to me what is going on here. How does the pattern
continue? Do you just see  forwarded query to 172.30.48.192 from now
on until the server is restarted, or do you still see query[A] and
query[PTR} lines?

Do queries which get pushed upstream continue to work? How about queries
which should be answered locally?

What is 172.30.158.94? Is it running anything that may generate odd
DNS queries? The holy grail would be to able prod that machine to
reproduce this at will.

What sort of machine are you running dnsmasq on? Does it have a
reasonable amount of spare storage so that you could tcpdump all traffic
to/from port 53,UDP for offline analysis?


Simon.


The information contained in this email and any attached files are 
confidential and intended solely for the addressee(s). The email may be 
legally privileged or prohibited from disclosure and unauthorised use. 
If you are not the named addressee you may not use, copy, or disclose 
this information to any other person. If you received this message in 
error please notify the sender immediately and delete it from your system.


Any opinion or views contained in this email message are those of the 
sender, and do not represent those of the Company in any way and 
reliance should not be placed upon its contents. Unless otherwise 
stated, this email message is not intended to be contractually binding. 
Where an Agreement exists between our respective companies and there is 
conflict between the contents of this email message and the Agreement 
then the terms of that Agreement shall prevail.


Excelian
50 Featherstone Street
London
EC1Y 8RT
Tel: +44 (0) 20 7336 9595
Fax: +44 (0) 20 7336 9596
www.Excelian.com
_
This e-mail has been scanned for viruses by MessageLabs. For further 
information visit http://www.messagelabs.com


Excelian subscribes to cleaner and greener methods of working. Help take 
responsibility for the environment. Please don't print this email unless 
you absolutely have to.





Re: [Dnsmasq-discuss] server forwarding all traffic to parents after a successful PTR query of itself

2010-02-17 Thread Alberto Cuesta-Canada
Cool, that makes a lot of sense. I'm actually reengineering the DNS 
infrastructure here, so it will be easy to account for and trace that at this 
stage.
 
I'll let you know when I find the rogue queries, many thanks,
 
Alberto Cuesta-Canada
GaaS Team Lead
Excelian Ltd.
+44 (0) 7942633361



From: Simon Kelley [mailto:si...@thekelleys.org.uk]
Sent: Wed 17/02/2010 10:30
To: Alberto Cuesta-Canada
Cc: dnsmasq-discuss@lists.thekelleys.org.uk; Grid Support
Subject: Re: [Dnsmasq-discuss] server forwarding all traffic to parents after a 
successful PTR query of itself



Alberto Cuesta-Canada wrote:
 Hi Simon,
 
 the parents of 250 (my dnsmasq server) have forwarding rules for the
 dselgrid.local domain, that I run. So I assumed that the queries pushed
 upstream would be routed down again, and timeout in a loop.

Ahh, that could easily be the problem. If you generate a loop between
two DNS servers, each forwarding to the other, then the queries can
easily bounce back-and-forth forever. Dnsmasq will manage this situation
reasonably well, and manage to server other traffic, but the circulating
queries will eat bandwidth and CPU.

The logs would seem to show a strange query of some sort (dnsmasq
can't parse a domain-name from the query, hence forwarded query rather
than forwarded domain-name) If such queries can circulate forever
then you have a problem.


 
 That said, in the logs I could still see successful PTR and A queries,
 outnumbered 10 to 1 by forwards. I'm not sure about the behaviour of
 local queries, I don't remember from yesterday, but I think they worked.
 
 94 is a Platform Grid Master, that is a W2K3 machine which runs only one
 application. It keeps a cache of machines but it doesn't give DNS
 services, or anything similar. The interesting thing is that the PTR
 request doesn't always produce this effect. I have enterprise support
 for that software, so I will ask them.
 
 dnsmasq is running in a quite complicated setup. We have a XenServer
 host running a Ubuntu 9.04 VM. I have just 1GB free on that machine and
 out of disk space scenarios are fatal, so I can't tcpdump. There is a
 rebuild of it coming in the next two weeks that will give me another 50GB.
 
 Any idea on what to look for, or any hypothesis of what could be
 happening should be enough, I can keep investigating and contain it with
 workarounds for a while.

See above. a loop, possibly only of odd queries.

Cheers,

Simon.

 
 Many thanks,
 
 *Alberto Cuesta-Canada*
 GaaS Team Lead
 Excelian Ltd.
 +44 (0) 7942633361

 
 *From:* Simon Kelley [mailto:si...@thekelleys.org.uk]
 *Sent:* Wed 17/02/2010 10:04
 *To:* Alberto Cuesta-Canada
 *Cc:* dnsmasq-discuss@lists.thekelleys.org.uk; Grid Support
 *Subject:* Re: [Dnsmasq-discuss] server forwarding all traffic to
 parents after a successful PTR query of itself

 It's not clear to me what is going on here. How does the pattern
 continue? Do you just see  forwarded query to 172.30.48.192 from now
 on until the server is restarted, or do you still see query[A] and
 query[PTR} lines?

 Do queries which get pushed upstream continue to work? How about queries
 which should be answered locally?

 What is 172.30.158.94? Is it running anything that may generate odd
 DNS queries? The holy grail would be to able prod that machine to
 reproduce this at will.

 What sort of machine are you running dnsmasq on? Does it have a
 reasonable amount of spare storage so that you could tcpdump all traffic
 to/from port 53,UDP for offline analysis?


 Simon.


 The information contained in this email and any attached files are
 confidential and intended solely for the addressee(s). The email may be
 legally privileged or prohibited from disclosure and unauthorised use.
 If you are not the named addressee you may not use, copy, or disclose
 this information to any other person. If you received this message in
 error please notify the sender immediately and delete it from your system.

 Any opinion or views contained in this email message are those of the
 sender, and do not represent those of the Company in any way and
 reliance should not be placed upon its contents. Unless otherwise
 stated, this email message is not intended to be contractually binding.
 Where an Agreement exists between our respective companies and there is
 conflict between the contents of this email message and the Agreement
 then the terms of that Agreement shall prevail.

 Excelian
 50 Featherstone Street
 London
 EC1Y 8RT
 Tel: +44 (0) 20 7336 9595
 Fax: +44 (0) 20 7336 9596
 www.Excelian.com
 _
 This e-mail has been scanned for viruses by MessageLabs. For further
 information visit http://www.messagelabs.com http://www.messagelabs.com/ 

 Excelian subscribes to cleaner and greener methods of working. Help take
 responsibility for the environment. Please don't 

[Dnsmasq-discuss] forwarding-loop mitigation.

2010-02-17 Thread Simon Kelley
Alberto's query got me thinking: If dnsmasq were to read the value of 
the IP hop-count on incoming queries, and decrement it when forwarding, 
loops would be squashed in the same way as IP layer-three forwarding.


Can anyone see a problem with this?

Simon.



Re: [Dnsmasq-discuss] IP address based on switch port number (option 82)

2010-02-17 Thread Simon Kelley

ignacio.br...@belden.com wrote:

Simon Kelley si...@thekelleys.org.uk wrote on 16/02/2010 14:27:36:


fakeroot debian/rules binary
I found a problem when fakerooting (sorry for my ignorance) Do I need to 
install additional tools containing this lib?:

Package libidn was not found in the pkg-config search path.
Perhaps you should add the directory containing `libidn.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libidn' found


That's a mismatch between what's needed to compile Ubuntu's current 
dnsmasq package and the latest one. the fix is


sudo apt-get install libidn11-dev



The magic you need is that you can invert tags, so make the last line



dhcp-range=net:#switch1,net:#puerto3,10.10.35.40,10.10.35.42,255.255.255.0

Then it can only be used when none of the port specific tags are in use.


The problem I see here is switch1 tag is common to all switch1-ports. So I 
discard all switch1 ports when I write: net:#switch1


I feel I need something like: #(switch1ANDport3) instead 
#(switch1)AND#(port3)


Is it possible to set something like #(net:switch1,net:port3)? or maybe an 
OR function so that different conditions (switchANDport) apply to the same 
range?


Good point. Your requirements exceed what it's currently possible to 
express using the tag system.


Maybe there needs to be something to calculate and arbitrary boolean 
expression on tags


declare-tag set:newtag, !(switch1  port1)

Joy, the options code finally becomes a full recusive-descent parser.


Let me think about this a bit more.


Simon.



Thanks
Ignacio

DISCLAIMER:

Privileged and/or Confidential information may be contained in this
message. If you are not the addressee of this message, you may not
copy, use or deliver this message to anyone. In such event, you
should destroy the message and kindly notify the sender by reply
e-mail. It is understood that opinions or conclusions that do not
relate to the official business of the company are neither given
nor endorsed by the company.

Thank You.







Re: [Dnsmasq-discuss] forwarding-loop mitigation.

2010-02-17 Thread Jan 'RedBully' Seiffert
Simon Kelley schrieb:
 Alberto's query got me thinking: If dnsmasq were to read the value of 
 the IP hop-count on incoming queries, and decrement it when forwarding, 
 loops would be squashed in the same way as IP layer-three forwarding.
 
 Can anyone see a problem with this?
 

If i'm not mistaken, IP hop-count is always reset since the packet reached its
destination (it is received) even if you or the other end forwards the query(
you do so on a higher protocol level, DNS).
And DNS has no Hop Count AFAIKS.

But maybe i don't get it...
This means you AND the remote and have to fudge with low level IP protocol 
stuff.
If the other end of the loop does not do the same thing, you gained nothing?

Nearby: getting to that info (the HopCount on reception) is ... ugly. But since
you already have to deal with IP_PKTINFO, IP_RECVHOPLIMIT is only an additional
pain.
But this also means you also have to set the hop count on send.

 Simon.
 

Greetings
Jan

-- 
/home
sweet
/home



Re: [Dnsmasq-discuss] IP address based on switch port number (option 82)

2010-02-17 Thread richardvo...@gmail.com
On Wed, Feb 17, 2010 at 5:03 AM, Simon Kelley si...@thekelleys.org.uk wrote:
 ignacio.br...@belden.com wrote:
 Simon Kelley si...@thekelleys.org.uk wrote on 16/02/2010 14:27:36:

 fakeroot debian/rules binary
 I found a problem when fakerooting (sorry for my ignorance) Do I need to
 install additional tools containing this lib?:
 Package libidn was not found in the pkg-config search path.
 Perhaps you should add the directory containing `libidn.pc'
 to the PKG_CONFIG_PATH environment variable
 No package 'libidn' found

 That's a mismatch between what's needed to compile Ubuntu's current
 dnsmasq package and the latest one. the fix is

 sudo apt-get install libidn11-dev


 The magic you need is that you can invert tags, so make the last line


 dhcp-range=net:#switch1,net:#puerto3,10.10.35.40,10.10.35.42,255.255.255.0
 Then it can only be used when none of the port specific tags are in use.

 The problem I see here is switch1 tag is common to all switch1-ports. So I
 discard all switch1 ports when I write: net:#switch1

 I feel I need something like: #(switch1ANDport3) instead
 #(switch1)AND#(port3)

 Is it possible to set something like #(net:switch1,net:port3)? or maybe an
 OR function so that different conditions (switchANDport) apply to the same
 range?


Can't the option-matching lines which set e.g. port1 also set e.g.
nopool, then use #nopool in the dhcp-range line?

I guess there might need to be nosw1pool, nosw2pool, nosw3pool, and
then dhcp-range such as net:switch1,net:#nosw1pool

Or maybe even just

net:switch1,net:#port3,net:#port4 (pool which is valid for all switch1
ports except 3 and 4)

 Good point. Your requirements exceed what it's currently possible to
 express using the tag system.

 Maybe there needs to be something to calculate and arbitrary boolean
 expression on tags

 declare-tag set:newtag, !(switch1  port1)

 Joy, the options code finally becomes a full recusive-descent parser.

Actually, I think you can avoid that without loss of generality.

By DeMorgan's theorem, the AND and NOT operations currently available
are sufficient to define any expression.  You just need a way to do
grouping, which a syntax for setting one tag conditionally based on
another tag would do.

Remember that the example given doesn't need a complement applied, you
can negate it in the match later.

Since if is a very well understood concept, I'd propose

tag-if switch1,port1 newtag

Where newtag gets set if all listed tags are matched.  Using multiple
tag-if lines lets you effectively OR things together (this yields
sum-of-products capability).  Allow a user to unset a tag with

tag-if #switch1,port15 #newtag

for even more flexibility, but now all tag-if commands need to be
processed in order.



 Let me think about this a bit more.


 Simon.


 Thanks
 Ignacio

 DISCLAIMER:

 Privileged and/or Confidential information may be contained in this
 message. If you are not the addressee of this message, you may not
 copy, use or deliver this message to anyone. In such event, you
 should destroy the message and kindly notify the sender by reply
 e-mail. It is understood that opinions or conclusions that do not
 relate to the official business of the company are neither given
 nor endorsed by the company.

 Thank You.




 ___
 Dnsmasq-discuss mailing list
 Dnsmasq-discuss@lists.thekelleys.org.uk
 http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss




Re: [Dnsmasq-discuss] Two questions about the cache and how dnsmasq forwards queries

2010-02-17 Thread /dev/rob0
On Tue, Feb 16, 2010 at 09:42:33AM +0100, SamLT wrote:
 Simon:
  Maybe your ISPs DNS server is playing games?
 
 I think my ISP also REDIRECTs DNS traffic to their nameservers, 
 since, I get the same result using google public dns service. (and 
 this doesn't happen @home with an other ISP).
 
 Well, this is going to be... fun!

snip

 I'd like to collect as much information as I can before I contact 
 my ISP, eventhough I think they'll just ignore me anyway...

What I do, and whilst it seemed ugly at first, I grew to like and
appreciate it: I run ISC BIND named(8) on an alternate port, totally
ignoring ISP upstream resolvers.

True, a redirection would still be a problem, but this could be an
easy way to gather more facts.

BIND as recursion-only is quite simple. And the dnsmasq interface is
also simple:
no-resolv
server=127.0.0.1#1053
# the following to prevent duplicate caching
cache-size=0
And of course, nameserver 127.0.0.1 in resolv.conf (and protect it
from your DHCP client, if applicable.)

The named.conf(5) file:
options {
directory /var/named;
listen-on port 1053 { 127.0.0.1; };
};
controls {
inet 127.0.0.1 port 1035 allow { localhost; };
};
zone . IN {
type hint;
file named.root;
};

(using the root hints file which can be obtained at
ftp://ftp.internic.net/domain/named.root installed as
/var/named/named.root and readable by the named user.)

named can run entirely as a non-root process this way.  You might
want to use a shell alias for rndc(8) to use 1035 rather than 953;
but in this configuration I have little need for rndc. Note, since
only 127.0.0.1 is bound, there is no need for access controls; only
shell users on the same host could query named directly. If that
worries you, get rid of your untrusted shell users. :) In Linux you
could restrict to the dnsmasq user like this:
iptables -vI OUTPUT -d 127.0.0.1 -p tcp --dport 1053 -j REJECT
iptables -vI OUTPUT -d 127.0.0.1 -p udp --dport 1053 -j DROP
iptables -vI OUTPUT -d 127.0.0.1 -p tcp --dport 1053 -m owner \
--uid-owner dnsmasq -j ACCEPT
iptables -vI OUTPUT -d 127.0.0.1 -p udp --dport 1053 -m owner \
--uid-owner dnsmasq -j ACCEPT
(Note the use of -I, the order of these commands is important; they
yield rules at the top of filter/OUTPUT in reverse order. First
ACCEPT from dnsmasq user; then DROP or REJECT from any other.)

The ISC folks and DNS gurus I have met generally recommend keeping
recursion separate from authoritative DNS service, and this does the
job well.

I have not encountered an ISP doing DNS redirection. I'd be very
angry if I did!
-- 
Offlist mail to this address is discarded unless
/dev/rob0 or not-spam is in Subject: header



Re: [Dnsmasq-discuss] IP address based on switch port number (option 82)

2010-02-17 Thread Simon Kelley
richardvo...@gmail.com wrote:

 Actually, I think you can avoid that without loss of generality.
 
 By DeMorgan's theorem, the AND and NOT operations currently available
 are sufficient to define any expression.  You just need a way to do
 grouping, which a syntax for setting one tag conditionally based on
 another tag would do.
 
 Remember that the example given doesn't need a complement applied, you
 can negate it in the match later.
 
 Since if is a very well understood concept, I'd propose
 
 tag-if switch1,port1 newtag
 
 Where newtag gets set if all listed tags are matched.  Using multiple
 tag-if lines lets you effectively OR things together (this yields
 sum-of-products capability).  Allow a user to unset a tag with
 
 tag-if #switch1,port15 #newtag
 
 for even more flexibility, but now all tag-if commands need to be
 processed in order.
 

Genius!

That's perfect. I think the second form might not be sensible, since the
 order of lines in the configuration file(s) is not significant anywhere
else.

The simplest scheme would ban tags defined in tag-if declarations from
use in other tag-if declarations. If that wasn't done, there would be a
need for multiple iterations to find all the tag values, and the
possibility of unresolvable data-loops exemplified by

tag-if black, white
tag-if white, black

I'd modify the syntax to match the changes in 2.53 where testing the
value of a tag is always done with tag:tagname and setting a tag is
always done with set:tagname, and ! is used for NOT. So we end up with

tag-if set:newtag, tag:!switch1, tag:port15

I like. Back soon with a test implementation.


Simon.









Re: [Dnsmasq-discuss] forwarding-loop mitigation.

2010-02-17 Thread Simon Kelley
Jan 'RedBully' Seiffert wrote:
 Simon Kelley schrieb:
 Alberto's query got me thinking: If dnsmasq were to read the value of 
 the IP hop-count on incoming queries, and decrement it when forwarding, 
 loops would be squashed in the same way as IP layer-three forwarding.

 Can anyone see a problem with this?

 
 If i'm not mistaken, IP hop-count is always reset since the packet reached 
 its
 destination (it is received) even if you or the other end forwards the 
 query(
 you do so on a higher protocol level, DNS).
 And DNS has no Hop Count AFAIKS.
 
 But maybe i don't get it...
 This means you AND the remote and have to fudge with low level IP protocol 
 stuff.
 If the other end of the loop does not do the same thing, you gained nothing?

You're right. This wouldn't fix Alberto's problem. Most of the instances
I've seen of this have involved multiple dnsmasq servers, and it would
work there.

Oh well.
 
 Nearby: getting to that info (the HopCount on reception) is ... ugly. But 
 since
 you already have to deal with IP_PKTINFO, IP_RECVHOPLIMIT is only an 
 additional
 pain.
Exactly, my reading is the it's  IP_RECVTTL, inevitably, it's different
for IPv6.

 But this also means you also have to set the hop count on send.
That's easy, just a call to setsockopt.

Cheers,

Simon.