Re: [Freeipa-users] dns_tkey_negotiategss: failure GSSAPI error [...] Message stream modified.
On 27.10.2016 21:47, Tyrell Jentink wrote: > Thank you Petr! I found the problem, but quite by accident... There may > be a Best Practice at hand that I wasn't aware of... > > I still have the Windows AD server sitting on the side, serving as DHCP > server and waiting patiently for my Cross Realm Trust; That server will > forward DNS requests to the IPA server, and return a non-authoritative > answer. Occasionally, that server will seemingly loose track of the IPA > server, and stop returning results... And that happened while I was trying > to follow through with your request for info... So as a quick work around, > I simply dropped the AD server from my resolv.conf... > > And then performed your requests, without errors. I ran the DNS Update > from the ipa-server-install script, and that worked without errors. I > added the AD server back into resolv.conf, and everything failed again. I > put the AD server as the SECOND name server in resolv.conf, and the errors > went away. So I've clearly identified the problem. > > I uninstalled the client, and reinstalled the client, and everything went > cleanly. > > To prevent this problem in the future... I will be changing the DHCP > options to list the IPA DNS first for the Linux clients, and the AD DNS > first for Windows clients; I still want the AD DNS server in the list, as a > fallback. Is this plan the best practice here? Well, the ordering of the servers does not matter as long as they can resolve records properly. The key problem is > answer. Occasionally, that server will seemingly loose track of the IPA > server, and stop returning results... And that happened while I was trying ... It should just work if you fix this. I hope it helps. Petr Spacek @ Red Hat > > On Wed, Oct 26, 2016 at 11:36 PM, Petr Spacek wrote: > >> On 27.10.2016 04:43, Tyrell Jentink wrote: 2016-10-26T23:30:40Z DEBUG Writing nsupdate commands to > /etc/ipa/.dns_update.txt: > 2016-10-26T23:30:40Z DEBUG debug > > update delete trainmaster.ipa.rxrhouse.net. IN A > show > send > > update delete trainmaster.ipa.rxrhouse.net. IN > show > send > > update add trainmaster.ipa.rxrhouse.net. 1200 IN A 10.42.0.100 > show > send > > 2016-10-26T23:30:40Z DEBUG Starting external process > 2016-10-26T23:30:40Z DEBUG args=/usr/bin/nsupdate -g > /etc/ipa/.dns_update.txt > 2016-10-26T23:30:40Z DEBUG Process finished, return code=1 > 2016-10-26T23:30:40Z DEBUG stdout=Outgoing update query: > ;; ->>HEADER<<- opcode: UPDATE, status: NOERROR, id: 0 > ;; flags:; ZONE: 0, PREREQ: 0, UPDATE: 0, ADDITIONAL: 0 > ;; UPDATE SECTION: > trainmaster.ipa.rxrhouse.net. 0 ANY A > > Outgoing update query: > ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39562 > ;; flags:; QUESTION: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1 > ;; QUESTION SECTION: > ;3107127915.sig-ipa-pdc.ipa.rxrhouse.net. ANY TKEY > > ;; ADDITIONAL SECTION: > 3107127915.sig-ipa-pdc.ipa.rxrhouse.net. 0 ANY TKEY gss-tsig. >> 1477524640 >> [...] > > 2016-10-26T23:30:40Z DEBUG stderr=Reply from SOA query: > ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 38738 > ;; flags: qr aa rd ra; QUESTION: 1, ANSWER: 0, AUTHORITY: 1, >> ADDITIONAL: 0 > ;; QUESTION SECTION: > ;trainmaster.ipa.rxrhouse.net. IN SOA > > ;; AUTHORITY SECTION: > ipa.rxrhouse.net. 0 IN SOA >> ipa-pdc.ipa.rxrhouse.net. > hostmaster.ipa.rxrhouse.net. 1477524446 3600 900 1209600 3600 > > Found zone name: ipa.rxrhouse.net > The master is: ipa-pdc.ipa.rxrhouse.net > start_gssrequest > Found realm from ticket: IPA.RXRHOUSE.NET > send_gssrequest > recvmsg reply from GSS-TSIG query > ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39562 > ;; flags: qr; QUESTION: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0 > ;; QUESTION SECTION: > ;3107127915.sig-ipa-pdc.ipa.rxrhouse.net. ANY TKEY > > ;; ANSWER SECTION: > 3107127915.sig-ipa-pdc.ipa.rxrhouse.net. 0 ANY TKEY gss-tsig. >> 1466301805 > 1466388205 3 NOERROR 101 > YGMGCSqGSIb3EgECAgMAflQwUqADAgEFoQMCAR6kERgPMjAxNjA2MTkw > MjAzMjVapQUCAwHGkaYDAgEpqREbD0FELlJYUkhPVVNFLk5FVKoUMBKg > AwIBAaELMAkbB2FkLXBkYyQ= > 0 > > dns_tkey_negotiategss: failure GSSAPI error: Major = Unspecified GSS > failure. Minor code may provide more information, Minor = Message >> stream > modified. > > 2016-10-26T23:30:40Z DEBUG nsupdate failed: Command >> '/usr/bin/nsupdate -g > /etc/ipa/.dns_update.txt' returned non-zero exit status 1 > 2016-10-26T23:30:40Z ERROR Failed to update DNS records. > 2016-10-26T23:30:40Z DEBUG DNS resolver: Query: > trainmaster.ipa.rxrhouse.net IN A > 2016-10-26T23:30:40Z DEBUG DNS resolver: No record. > 2016-10-26T23:30:40Z DEBUG DNS resolver: Query: > trainmaster.ipa.rxrhouse.net
Re: [Freeipa-users] dns_tkey_negotiategss: failure GSSAPI error [...] Message stream modified.
Thank you Petr! I found the problem, but quite by accident... There may be a Best Practice at hand that I wasn't aware of... I still have the Windows AD server sitting on the side, serving as DHCP server and waiting patiently for my Cross Realm Trust; That server will forward DNS requests to the IPA server, and return a non-authoritative answer. Occasionally, that server will seemingly loose track of the IPA server, and stop returning results... And that happened while I was trying to follow through with your request for info... So as a quick work around, I simply dropped the AD server from my resolv.conf... And then performed your requests, without errors. I ran the DNS Update from the ipa-server-install script, and that worked without errors. I added the AD server back into resolv.conf, and everything failed again. I put the AD server as the SECOND name server in resolv.conf, and the errors went away. So I've clearly identified the problem. I uninstalled the client, and reinstalled the client, and everything went cleanly. To prevent this problem in the future... I will be changing the DHCP options to list the IPA DNS first for the Linux clients, and the AD DNS first for Windows clients; I still want the AD DNS server in the list, as a fallback. Is this plan the best practice here? On Wed, Oct 26, 2016 at 11:36 PM, Petr Spacek wrote: > On 27.10.2016 04:43, Tyrell Jentink wrote: > >> 2016-10-26T23:30:40Z DEBUG Writing nsupdate commands to > >> > /etc/ipa/.dns_update.txt: > >> > 2016-10-26T23:30:40Z DEBUG debug > >> > > >> > update delete trainmaster.ipa.rxrhouse.net. IN A > >> > show > >> > send > >> > > >> > update delete trainmaster.ipa.rxrhouse.net. IN > >> > show > >> > send > >> > > >> > update add trainmaster.ipa.rxrhouse.net. 1200 IN A 10.42.0.100 > >> > show > >> > send > >> > > >> > 2016-10-26T23:30:40Z DEBUG Starting external process > >> > 2016-10-26T23:30:40Z DEBUG args=/usr/bin/nsupdate -g > >> > /etc/ipa/.dns_update.txt > >> > 2016-10-26T23:30:40Z DEBUG Process finished, return code=1 > >> > 2016-10-26T23:30:40Z DEBUG stdout=Outgoing update query: > >> > ;; ->>HEADER<<- opcode: UPDATE, status: NOERROR, id: 0 > >> > ;; flags:; ZONE: 0, PREREQ: 0, UPDATE: 0, ADDITIONAL: 0 > >> > ;; UPDATE SECTION: > >> > trainmaster.ipa.rxrhouse.net. 0 ANY A > >> > > >> > Outgoing update query: > >> > ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39562 > >> > ;; flags:; QUESTION: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1 > >> > ;; QUESTION SECTION: > >> > ;3107127915.sig-ipa-pdc.ipa.rxrhouse.net. ANY TKEY > >> > > >> > ;; ADDITIONAL SECTION: > >> > 3107127915.sig-ipa-pdc.ipa.rxrhouse.net. 0 ANY TKEY gss-tsig. > 1477524640 > [...] > >> > > >> > 2016-10-26T23:30:40Z DEBUG stderr=Reply from SOA query: > >> > ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 38738 > >> > ;; flags: qr aa rd ra; QUESTION: 1, ANSWER: 0, AUTHORITY: 1, > ADDITIONAL: 0 > >> > ;; QUESTION SECTION: > >> > ;trainmaster.ipa.rxrhouse.net. IN SOA > >> > > >> > ;; AUTHORITY SECTION: > >> > ipa.rxrhouse.net. 0 IN SOA > ipa-pdc.ipa.rxrhouse.net. > >> > hostmaster.ipa.rxrhouse.net. 1477524446 3600 900 1209600 3600 > >> > > >> > Found zone name: ipa.rxrhouse.net > >> > The master is: ipa-pdc.ipa.rxrhouse.net > >> > start_gssrequest > >> > Found realm from ticket: IPA.RXRHOUSE.NET > >> > send_gssrequest > >> > recvmsg reply from GSS-TSIG query > >> > ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39562 > >> > ;; flags: qr; QUESTION: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0 > >> > ;; QUESTION SECTION: > >> > ;3107127915.sig-ipa-pdc.ipa.rxrhouse.net. ANY TKEY > >> > > >> > ;; ANSWER SECTION: > >> > 3107127915.sig-ipa-pdc.ipa.rxrhouse.net. 0 ANY TKEY gss-tsig. > 1466301805 > >> > 1466388205 3 NOERROR 101 > >> > YGMGCSqGSIb3EgECAgMAflQwUqADAgEFoQMCAR6kERgPMjAxNjA2MTkw > >> > MjAzMjVapQUCAwHGkaYDAgEpqREbD0FELlJYUkhPVVNFLk5FVKoUMBKg > >> > AwIBAaELMAkbB2FkLXBkYyQ= > >> > 0 > >> > > >> > dns_tkey_negotiategss: failure GSSAPI error: Major = Unspecified GSS > >> > failure. Minor code may provide more information, Minor = Message > stream > >> > modified. > >> > > >> > 2016-10-26T23:30:40Z DEBUG nsupdate failed: Command > '/usr/bin/nsupdate -g > >> > /etc/ipa/.dns_update.txt' returned non-zero exit status 1 > >> > 2016-10-26T23:30:40Z ERROR Failed to update DNS records. > >> > 2016-10-26T23:30:40Z DEBUG DNS resolver: Query: > >> > trainmaster.ipa.rxrhouse.net IN A > >> > 2016-10-26T23:30:40Z DEBUG DNS resolver: No record. > >> > 2016-10-26T23:30:40Z DEBUG DNS resolver: Query: > >> > trainmaster.ipa.rxrhouse.net IN > >> > 2016-10-26T23:30:40Z DEBUG DNS resolver: No record. > >> > 2016-10-26T23:30:40Z DEBUG DNS resolver: Query: > 100.0.42.10.in-addr.arpa. > >> > IN PTR > >> > 2016-10-26T23:30:40Z DEBUG DNS resolver: No record. > >> > 2016-10-26T23:30:40Z WARNING Missing A/ record(s) for host > >> > trainmaster.ipa.rxrhouse.net: 10.42.0.100. > >> > 2016-10-26T23:30:40Z WARNING Missin
Re: [Freeipa-users] dns_tkey_negotiategss: failure GSSAPI error [...] Message stream modified.
On 27.10.2016 04:43, Tyrell Jentink wrote: >> 2016-10-26T23:30:40Z DEBUG Writing nsupdate commands to >> > /etc/ipa/.dns_update.txt: >> > 2016-10-26T23:30:40Z DEBUG debug >> > >> > update delete trainmaster.ipa.rxrhouse.net. IN A >> > show >> > send >> > >> > update delete trainmaster.ipa.rxrhouse.net. IN >> > show >> > send >> > >> > update add trainmaster.ipa.rxrhouse.net. 1200 IN A 10.42.0.100 >> > show >> > send >> > >> > 2016-10-26T23:30:40Z DEBUG Starting external process >> > 2016-10-26T23:30:40Z DEBUG args=/usr/bin/nsupdate -g >> > /etc/ipa/.dns_update.txt >> > 2016-10-26T23:30:40Z DEBUG Process finished, return code=1 >> > 2016-10-26T23:30:40Z DEBUG stdout=Outgoing update query: >> > ;; ->>HEADER<<- opcode: UPDATE, status: NOERROR, id: 0 >> > ;; flags:; ZONE: 0, PREREQ: 0, UPDATE: 0, ADDITIONAL: 0 >> > ;; UPDATE SECTION: >> > trainmaster.ipa.rxrhouse.net. 0 ANY A >> > >> > Outgoing update query: >> > ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39562 >> > ;; flags:; QUESTION: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1 >> > ;; QUESTION SECTION: >> > ;3107127915.sig-ipa-pdc.ipa.rxrhouse.net. ANY TKEY >> > >> > ;; ADDITIONAL SECTION: >> > 3107127915.sig-ipa-pdc.ipa.rxrhouse.net. 0 ANY TKEY gss-tsig. 1477524640 [...] >> > >> > 2016-10-26T23:30:40Z DEBUG stderr=Reply from SOA query: >> > ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 38738 >> > ;; flags: qr aa rd ra; QUESTION: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0 >> > ;; QUESTION SECTION: >> > ;trainmaster.ipa.rxrhouse.net. IN SOA >> > >> > ;; AUTHORITY SECTION: >> > ipa.rxrhouse.net. 0 IN SOA ipa-pdc.ipa.rxrhouse.net. >> > hostmaster.ipa.rxrhouse.net. 1477524446 3600 900 1209600 3600 >> > >> > Found zone name: ipa.rxrhouse.net >> > The master is: ipa-pdc.ipa.rxrhouse.net >> > start_gssrequest >> > Found realm from ticket: IPA.RXRHOUSE.NET >> > send_gssrequest >> > recvmsg reply from GSS-TSIG query >> > ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39562 >> > ;; flags: qr; QUESTION: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0 >> > ;; QUESTION SECTION: >> > ;3107127915.sig-ipa-pdc.ipa.rxrhouse.net. ANY TKEY >> > >> > ;; ANSWER SECTION: >> > 3107127915.sig-ipa-pdc.ipa.rxrhouse.net. 0 ANY TKEY gss-tsig. 1466301805 >> > 1466388205 3 NOERROR 101 >> > YGMGCSqGSIb3EgECAgMAflQwUqADAgEFoQMCAR6kERgPMjAxNjA2MTkw >> > MjAzMjVapQUCAwHGkaYDAgEpqREbD0FELlJYUkhPVVNFLk5FVKoUMBKg >> > AwIBAaELMAkbB2FkLXBkYyQ= >> > 0 >> > >> > dns_tkey_negotiategss: failure GSSAPI error: Major = Unspecified GSS >> > failure. Minor code may provide more information, Minor = Message stream >> > modified. >> > >> > 2016-10-26T23:30:40Z DEBUG nsupdate failed: Command '/usr/bin/nsupdate -g >> > /etc/ipa/.dns_update.txt' returned non-zero exit status 1 >> > 2016-10-26T23:30:40Z ERROR Failed to update DNS records. >> > 2016-10-26T23:30:40Z DEBUG DNS resolver: Query: >> > trainmaster.ipa.rxrhouse.net IN A >> > 2016-10-26T23:30:40Z DEBUG DNS resolver: No record. >> > 2016-10-26T23:30:40Z DEBUG DNS resolver: Query: >> > trainmaster.ipa.rxrhouse.net IN >> > 2016-10-26T23:30:40Z DEBUG DNS resolver: No record. >> > 2016-10-26T23:30:40Z DEBUG DNS resolver: Query: 100.0.42.10.in-addr.arpa. >> > IN PTR >> > 2016-10-26T23:30:40Z DEBUG DNS resolver: No record. >> > 2016-10-26T23:30:40Z WARNING Missing A/ record(s) for host >> > trainmaster.ipa.rxrhouse.net: 10.42.0.100. >> > 2016-10-26T23:30:40Z WARNING Missing reverse record(s) for address(es): >> > 10.42.0.100. >> > > -- Full logs can be found here: http://pastebin.com/90dG9Ffu > >- For grins, I decided to test: >kinit admin >id admin >getent passwd admin >on the client, and all of those all made valid responses... So >authentication is working, I just can't update DNS records. > > > So that's what I've tried, and where I'm at... My client machines running > modern client software can NOT update DNS records, complaining about GSSAPI > "Message Stream Modified" errors... And I have no idea how to troubleshoot > that... Any ideas? Interesting, I haven't seen this one :-) There is something fishy in GSSAPI negotiation between the client and DNS server. I would try this (and watch out for suspicious messages along the way): 1) To be sure, please double-check that ipa-pdc.ipa.rxrhouse.net. resolves (from the client) to correct IP address of IPA DNS server. 2) Verify that Kerberos ticket for the DNS server can be obtained: $ kinit -k $ kvno DNS/ipa-pdc.ipa.rxrhouse.net $ klist # it should list Kerberos ticket for ipa-pdc.ipa.rxrhouse.net 3) Create a plain text file with update message content: cat > /tmp/dnsupdate <