Re: bind unexpectedly quit, how to debug

[G.W. Haywood]

Hi there,

On Tue, 9 May 2017, Paul Seward wrote:


... I'm not so much asking for a fix as asking how I can find more
information. ...


grep '\(released\|security\)' bind-9.10.5/CHANGES | head -n 90

--

73,
Ged.
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: bind unexpectedly quit, how to debug

[Tony Finch]

Paul Seward  wrote:
>
> I thought I might get that sort of response, I'm not so much asking for a
> fix as asking how I can find more information.

It'll be one of the 42 CVEs in the table at the top of this page:
https://kb.isc.org/article/AA-00913/74/BIND-9-Security-Vulnerability-Matrix.html

I think all of them probably apply to the version you are running.

However you are running a version with Red Hat's mystery meat patches,
so the vulnerabilities in what you are running won't match the nominal
ISC version number.

If you are running a service based on Red Hat's code, you should really
be paying for support from Red Hat. If that isn't an option, use Carl
Byington's RPMs instead.

> but until then I need to show management that I've done my due diligence
> into investigating the root cause.

Well the root cause is that your management aren't supporting your
routine security patch process!

Tony.
-- 
f.anthony.n.finch    http://dotat.at/  -  I xn--zr8h punycode
North Shannon, Rockall, Malin, South Hebrides: Variable, mainly easterly at
first, 3 or 4. Slight or moderate. Fair. Good.
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: bind unexpectedly quit, how to debug

[Paul Seward]

Hi Jim,

I thought I might get that sort of response, I'm not so much asking for a
fix as asking how I can find more information.

We're in the process of migrating from this version of bind to something
more recent - and may well use this incident as a lever to speed up some of
the political hurdles involved in doing so - but until then I need to show
management that I've done my due diligence into investigating the root
cause.

So if anyone has any suggestions for how I can get more information about
what's triggering the crash I would still welcome them.

-Paul

On 9 May 2017 at 11:04, Jim Reid  wrote:

>
> > On 9 May 2017, at 10:47, Paul Seward  wrote:
> >
> > We've got some recursive-only servers running bind 9.8.1 on CentOS 6.9
> (using 9.8.2rc1-RedHat-9.8.2-0.62.rc1.el6_9.1 from the CentOS repos)
> >
> > They've unexpectedly quit a couple of times in the last month, leaving
> errors like this in the logs:
>
> Come back when you see the same problem with a current version of BIND (ie
> 9.10 or 9.11). Version 9.8 has been dead for a while and many of its bugs
> have been fixed in newer releases.
>
>


-- 
--
Paul Seward,Senior Systems Administrator,University of Bristol
paul.sew...@bristol.ac.uk  +44 (0)117 39 41148GPG Key ID: E24DA8A2
GPG Fingerprint:7210 4E4A B5FC 7D9C 39F8  5C3C 6759 3937 E24D A8A2
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

bind unexpectedly quit, how to debug

[Paul Seward]

Hi all,

We've got some recursive-only servers running bind 9.8.1 on CentOS 6.9
(using 9.8.2rc1-RedHat-9.8.2-0.62.rc1.el6_9.1 from the CentOS repos)

They've unexpectedly quit a couple of times in the last month, leaving
errors like this in the logs:

09-May-2017 09:12:56.747 dnssec: info: validating @0x7f37dbf852e0:
ntp1.glb.nist.gov A: no valid signature found
09-May-2017 09:12:56.831 dnssec: info: validating @0x7f37d7dd3100:
www.puma.com.cdn.cloudflare.net A: no valid signature found
09-May-2017 09:12:58.172 dnssec: info: validating @0x7f37dbf852e0:
cdnjs.cloudflare.com : no valid signature found
09-May-2017 09:12:59.470 dnssec: info: validating @0x7f37dbf832c0: cdnjs.com
A: no valid signature found
09-May-2017 09:13:02.401 general: critical: validator.c:1861:
INSIST(rdataset->type == ((dns_rdatatype_t)dns_rdatatype_dnskey)) failed,
back trace
09-May-2017 09:13:02.401 general: critical: #0 0x7f3831b5007f in ??
09-May-2017 09:13:02.401 general: critical: #1 0x7f38304afa9a in ??
09-May-2017 09:13:02.401 general: critical: #2 0x7f383145eb4c in ??
09-May-2017 09:13:02.401 general: critical: #3 0x7f3831466620 in ??
09-May-2017 09:13:02.401 general: critical: #4 0x7f38304ce858 in ??
09-May-2017 09:13:02.401 general: critical: #5 0x7f382fe83aa1 in ??
09-May-2017 09:13:02.401 general: critical: #6 0x7f382f3e3bcd in ??
09-May-2017 09:13:02.401 general: critical: exiting (due to assertion
failure)

The DNSSec validation errors which precede the validator.c assertion don't
appear to trigger the bug when tested against an identical resolver.

What's the best way for me to get more information about what's causing
bind to quit?

-Paul
-- 
--
Paul Seward,Senior Systems Administrator,University of Bristol
paul.sew...@bristol.ac.uk  +44 (0)117 39 41148GPG Key ID: E24DA8A2
GPG Fingerprint:7210 4E4A B5FC 7D9C 39F8  5C3C 6759 3937 E24D A8A2
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: How to debug BIND

[Kaouthar Chetioui]

thank very much for your answer,

I have done it, and when I look at log file, I found a lot of information
like this:

29-Nov-2014 18:09:58.121 general: debug 60: sockmgr 0xb77d7008: watcher got
message -3 for socket 514
29-Nov-2014 18:09:58.121 general: debug 60: sockmgr 0xb77d7008: watcher got
message -2 for socket -1
29-Nov-2014 18:09:58.121 general: debug 50: socket 0xb558ce18: socket_recv:
event 0xb558d008 - task 0xb7807550
29-Nov-2014 18:09:58.121 resolver: debug 3: resquery 0xb5103280 (fctx
0xb50fc008(C0010/A)): sent
29-Nov-2014 18:09:58.121 resolver: debug 3: resquery 0xb5103008 (fctx
0xb50fc008(C0010/A)): senddone
29-Nov-2014 18:09:58.121 resolver: debug 3: resquery 0xb5103280 (fctx
0xb50fc008(C0010/A)): udpconnected
29-Nov-2014 18:09:58.121 resolver: debug 3: fctx 0xb50fc008(C0010/A'):
add_bad
29-Nov-2014 18:09:58.121 lame-servers: info: network unreachable resolving
'C0010/A/IN': 192.228.79.201#53
29-Nov-2014 18:09:58.121 resolver: debug 3: fctx 0xb50fc008(C0010/A'):
cancelquery
29-Nov-2014 18:09:58.121 dispatch: debug 90: dispatch 0xb78037f8 response
0xb5591638 192.228.79.201#53: detaching from task 0xb7809ca0

I don't find any name of source file (like message.c or name.c) or name of
function in this log file, so I can't understand excatly the process of
resolution for dig command.
So, I'm asking if we can have more details in log files about BND  source
files and functions involved in dns resolution?

Thanks.

2014-11-30 14:32 GMT+00:00 Tony Finch d...@dotat.at:

 Kaouthar Chetioui kaoutharcheti...@gmail.com wrote:
  I want to know the exact path that follows bind to resolve a DNS query

 Try running

 $ rndc flush
 $ rndc trace 11
 $ dig www.example.ma

 Then look at named's logs which will give you lots of details about
 queries, responses, and the parts of BIND involved in the process.

 Tony.
 --
 f.anthony.n.finch  d...@dotat.at  http://dotat.at/
 South Fitzroy: Northerly 5 to 7, occasionally gale 8 at first. Rough,
 occasionally very rough at first. Showers. Good, occasionally moderate.




-- 
Kaouthar CHETIOUI
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: How to debug BIND

[Tony Finch]

Kaouthar Chetioui kaoutharcheti...@gmail.com wrote:

 I don't find any name of source file (like message.c or name.c) or name of
 function in this log file, so I can't understand excatly the process of
 resolution for dig command.

The log module gives you a rough idea of which part of the system emitted
the log message. I often find I have to grep the source to find the exact
place, which is a bit tiresome especially because messages are often split
across multiple lines in the code.

Tony.
-- 
f.anthony.n.finch  d...@dotat.at  http://dotat.at/
Southeast Iceland: Southwesterly severe gale 9 or storm 10, decreasing 6 to
gale 8. High becoming very rough. Squally wintry showers. Good, occasionally
poor.
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: How to debug BIND

[Kaouthar Chetioui]

Ok , Thank you

2014-12-01 10:49 GMT+00:00 Tony Finch d...@dotat.at:

 Kaouthar Chetioui kaoutharcheti...@gmail.com wrote:
 
  I don't find any name of source file (like message.c or name.c) or name
 of
  function in this log file, so I can't understand excatly the process of
  resolution for dig command.

 The log module gives you a rough idea of which part of the system emitted
 the log message. I often find I have to grep the source to find the exact
 place, which is a bit tiresome especially because messages are often split
 across multiple lines in the code.

 Tony.
 --
 f.anthony.n.finch  d...@dotat.at  http://dotat.at/
 Southeast Iceland: Southwesterly severe gale 9 or storm 10, decreasing 6 to
 gale 8. High becoming very rough. Squally wintry showers. Good,
 occasionally
 poor.




-- 
Kaouthar CHETIOUI
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: How to debug BIND

[Steven Carr]

On 30 November 2014 at 01:22, Kaouthar Chetioui
kaoutharcheti...@gmail.com wrote:
 I want to do full debug for BIND

 I use this command: dig www.example.ma -d

What's the problem you are having?

What are you expecting to see when you perform a debug?

What is the real name you are trying to diagnose?

Steve
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: How to debug BIND

[Steven Carr]

On 30 November 2014 at 11:04, Kaouthar Chetioui
kaoutharcheti...@gmail.com wrote:
 I want to know the exact path that follows bind to resolve a DNS query

Please reply to the list not direct.

The option you are looking for is +trace and needs to be invoked on
the server/system that will be resolving the query for the client.

You might want to try man dig and look at the documentation first in future...

Steve
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: How to debug BIND

[Kaouthar Chetioui]

I have already use +trace it gives me the following answer, like this:
global options: +cmd
. 518400 IN NS E.ROOT-SERVERS.NET.
. 518400 IN NS G.ROOT-SERVERS.NET.
. 518400 IN NS D.ROOT-SERVERS.NET.
. 518400 IN NS H.ROOT-SERVERS.NET.
. 518400 IN NS K.ROOT-SERVERS.NET.
. 518400 IN NS B.ROOT-SERVERS.NET.
. 518400 IN NS I.ROOT-SERVERS.NET.
. 518400 IN NS J.ROOT-SERVERS.NET.
. 518400 IN NS F.ROOT-SERVERS.NET.
. 518400 IN NS C.ROOT-SERVERS.NET.
. 518400 IN NS M.ROOT-SERVERS.NET.
. 518400 IN NS L.ROOT-SERVERS.NET.
. 518400 IN NS A.ROOT-SERVERS.NET.

I also add in 'named.conf' file, the following commands:
logging {
channel debug {
file data/named.log size 10m;
severity debug 99;
print-time yes;
print-severity yes;
print-category yes;
};
category default { debug; };
category general { debug; };
category database { debug; };
category security { debug; };
category config { debug; };
category resolver { debug; };
category xfer-in { debug; };
category xfer-out { debug; };
category notify { debug; };
category client { debug; };
category unmatched { debug; };
category network { debug; };
category update { debug; };
category queries { debug; };
category dispatch { debug; };
category dnssec { debug; };
category lame-servers { debug; };

};
and I used 'dig www.example.ma -d' to debug.
In the file 'named.log', I have the detail of debug but I dont find
functions that are used in Bind source files.

Thanks.

2014-11-30 11:10 GMT+00:00 Steven Carr sjc...@gmail.com:

 On 30 November 2014 at 11:04, Kaouthar Chetioui
 kaoutharcheti...@gmail.com wrote:
  I want to know the exact path that follows bind to resolve a DNS query

 Please reply to the list not direct.

 The option you are looking for is +trace and needs to be invoked on
 the server/system that will be resolving the query for the client.

 You might want to try man dig and look at the documentation first in
 future...

 Steve




Kaouthar.
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: How to debug BIND

[Matus UHLAR - fantomas]

On 30.11.14 11:24, Kaouthar Chetioui wrote:

I have already use +trace it gives me the following answer, like this:


no, it doeas not:


global options: +cmd


you clearly did not use +trace here.

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
How does cat play with mouse? cat /dev/mouse
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: How to debug BIND

[Steven Carr]

DIG is used to test/troubleshoot DNS queries. BIND logging is used to
troubleshoot the BIND server itself. Which are you trying to debug?

Also be mindful that BIND will cache any DNS entries it retrieves for
the defined TTLs, so if you dig a second time chances are it's not
going to go to the Internet, it will answer from cache.

If you are trying to examine exactly what BIND is querying then use
dig against the server for the requested records while running a
packet capture on the server itself. Filter the capture for all DNS
packets to see what's happening. Make sure BIND's cache is flushed
between digs.

If you want to debug the underlying BIND code then you'll need to use
an actual code debugger, BIND's debug logging is for debugging the
running of the program, so if you want to see it jumping through the
various code functions then look at GDB (GNU Project Debugger) - not
quite sure what you're hoping to gain from this though.

Steve
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: How to debug BIND

[Alex]

Try option (+nodnssec):
dig www.example.ma +trace +nodnssec


On 11/30/2014 04:40 PM, Matus UHLAR - fantomas wrote:
 On 30.11.14 11:24, Kaouthar Chetioui wrote:
 I have already use +trace it gives me the following answer, like this:

 no, it doeas not:

 global options: +cmd

 you clearly did not use +trace here.



-- 
Kanogin Alex

___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: How to debug BIND

[Tony Finch]

Kaouthar Chetioui kaoutharcheti...@gmail.com wrote:
 I want to know the exact path that follows bind to resolve a DNS query

Try running

$ rndc flush
$ rndc trace 11
$ dig www.example.ma

Then look at named's logs which will give you lots of details about
queries, responses, and the parts of BIND involved in the process.

Tony.
-- 
f.anthony.n.finch  d...@dotat.at  http://dotat.at/
South Fitzroy: Northerly 5 to 7, occasionally gale 8 at first. Rough,
occasionally very rough at first. Showers. Good, occasionally moderate.
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

How to debug BIND

[Kaouthar Chetioui]

Hi,

I want to do full debug for BIND

I use this command: dig www.example.ma -d

and I have as result:
cancel_lookup()
check_if_done()
list empty
clear_query(0xb77c5010)
sockcount=0
check_next_lookup(0x9915980)
try_clear_lookup(0x9915980)
destroy
freeing server 0xb77c11b0 belonging to 0x9915980
start_lookup()
check_if_done()
list empty
shutting down
unlock_lookup dighost.c:3829
destroy
cancel_all()
lock_lookup dighost.c:3940
success
unlock_lookup dighost.c:3983
destroy_libs()
freeing task
freeing taskmgr
lock_lookup dighost.c:4015
success
flush_server_list()
freeing commctx
freeing socketmgr
freeing timermgr
destroy DST lib
detach from entropy
unlock_lookup dighost.c:4068
Removing log context
Destroy memory

I need more detail so, can you give me the solution please...?


-- 
Kaouthar CHETIOUI
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

bind-9.7.2-P3 linux how to debug/troubleshoot query failures?

[Tory M Blue]

Hey all,

Well I'm reaching out as I'm at a loss. I have a distributed DNS
architecture with 2 bind-9.7.2-P3 servers behind an F5 Loadbalancer. I
then have another 2 behind another F5 at another location.

My app servers are configured with their resolv.conf looking like:
(please ignore the domain and networks, they have been altered)

search gc.domain.net
domain gc.domain.net
nameserver 1.1.1.15
nameserver 1.1.2.56
options timeout:1

What I'm finding out is that there are a ton of requests being made to
the 1.1.2.56 address. In reality the servers at 1.1.1.15 (again behind
the F5) are healthy, no retransmissions, no excessive load nothing
that tells me they are having issues. Yet my servers seem to fail to
connect to them and must failover to the secondary DNS servers (again
I don't understand why, nor can I figure out why).

If I run a script that does a dig I can't seem to get it to failover
to the secondary DNS, but something in code or other that uses
gethostbyname or the host command seem to cause a lookup fail and thus
it fails over to the secondary nodes, across the internet in fact.

Is there a documented method to troubleshoot, debug why a system
believes that they were unable to get an acceptable results from the
primary DNS server?

Doesn't appear to be any health related issues, so I'm at a loss. I
feel the DNS infrastructure is healthy but at this point
I need some assistance proving that it's not and therefore fixing it!

I've added the 1 Second timeout since I was seeing 5 second delays in
our application and again this was due to it waiting for the primary
server to respond before it could failover, now after a second it just
goes to the secondary dns and seems to be happy (most of the time, I'm
getting some hard failures that I'm trying to troubleshoot as well).


Thanks
Tory
___
bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: bind-9.7.2-P3 linux how to debug/troubleshoot query failures?

[Barry Margolin]

In article mailman.1634.1296765859.555.bind-us...@lists.isc.org,
 Tory M Blue tmb...@gmail.com wrote:

 Is there a documented method to troubleshoot, debug why a system
 believes that they were unable to get an acceptable results from the
 primary DNS server?

Capture the DNS packets and see if you're sending to the primary in the 
first place and what the responses look like.

-- 
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE don't copy me on replies, I'll read them in the group ***
___
bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

how to debug

[aihua zhang]

HI,

  I  have already analysis where to add new RR,and how to make it works.
 But i don't contact  automake tool before, so reading so large configure
and makefiles make me feel so bad. I try to understand ,but it just myself
alone to do this , so anyone can give some guide how to debug the source
code 、 how to modify makefile and test result!

 Thanks very much！

-- 
Best regards!
___
bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: how to debug

[Mark Andrews]

In message e1b1ab9e0910281921j612d2982le3170b6dc3d60...@mail.gmail.com, aihua
 zhang writes:
 
 HI,
 
   I  have already analysis where to add new RR,and how to make it works.
  But i don't contact  automake tool before, so reading so large configure
 and makefiles make me feel so bad. I try to understand ,but it just myself
 alone to do this , so anyone can give some guide how to debug the source
 code =A1=A2 how to modify makefile and test result!

I'll repeat what I said before make clean then make.  You don't
need to touch configure or the Makefiles.  You just need to do a
clean build.  The process will look in lib/dns/rdata and find the
files there.

Mark

  Thanks very much=A3=A1
 
 --=20
 Best regards!
 
 --001485354cc2c8f4fa0477099043
 Content-Type: text/html; charset=GB2312
 Content-Transfer-Encoding: quoted-printable
 
 divHI,/div
 divnbsp;nbsp;nbsp; /div
 divnbsp; Inbsp; have already analysis where to add new RR,and how to ma=
 ke it works./div
 divnbsp;But i don#39;t contactnbsp;nbsp;automake tool before, so read=
 ing so large configure and makefiles make me feel so bad.nbsp;I try to und=
 erstand ,but it just myself alone to do this ,nbsp;so anyone can give some=
  guidenbsp;how to debug the source codenbsp;=A1=A2nbsp;hownbsp;to modif=
 y makefilenbsp;and test result!/div
 
 divnbsp;/div
 divnbsp;Thanks very much=A3=A1br clear=3Dallbr-- brBest regards!=
 brbr/div
 
 --001485354cc2c8f4fa0477099043--
 
 --===8156758388202099534==
 Content-Type: text/plain; charset=us-ascii
 MIME-Version: 1.0
 Content-Transfer-Encoding: 7bit
 Content-Disposition: inline
 
 ___
 bind-users mailing list
 bind-users@lists.isc.org
 https://lists.isc.org/mailman/listinfo/bind-users
 --===8156758388202099534==--
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org
___
bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users