Re: [Freeipa-devel] DNS improvements: Should we add some sanity checking?
On 09/13/2013 06:17 PM, Simo Sorce wrote: On Fri, 2013-09-13 at 09:29 +0200, Petr Spacek wrote: Hello list, Jan Pazdziora jpazdzi...@redhat.com proposed that 'ipa dns*' commands should do some sanity checking/waiting after the record is added to LDAP. I think that it could be valuable and I would like to get opinions from freeipa-devel list. === The problem === ipa dnsrecord-add and similar commands add the data to LDAP, but it doesn't mean that the data are *immediately* resolvable via DNS protocol. Note that data from LDAP are *asynchronously* read and processed by Named and the time when records are available is not predictable. A mismatch between LDAP can be caused by some connection problem between DNS and LDAP servers, LDAP or DNS server restart, or simply by a bug in DNS-LDAP synchronization code. (This is becomming more and more important if we consider the whole DNSSEC effort and related re-factoring.) My experience is that users are very confused if the ipa dnsrecord-add command says 'record added' but it is still not available via DNS. It is really hard to debug when you see the problem first 10 times :-) === The proposal === 1. Let FreeIPA framework to change DNS data in LDAP as we do now. 2. After each change, do DNS queries for changed record and wait until the new data are available. IMHO it is very cheap operation (in usual cases 1 DNS packet back and forth) and it would save a lot of headaches to users and support. This will naturally catch the case where named crashes after the change etc. === Expected outcome === There will not be any failure like this: $ ipa-adtrust-install $ ipa dnszone-add $AD_DOMAIN --name-server=advm.$AD_DOMAIN --admin-email=hostmaster@$AD_DOMAIN.com --force --forwarder=$AD_IP --forward-policy=only --ip-address=$AD_IP Zone name: dom123.example.com [...] $ ipa trust-add --type=ad DOM123.EXAMPLE.COM --admin Administrator --password Password for ad...@dom123.example.com: ipa: ERROR: Cannot find specified domain or server name Would it make sense to change the code to use dynDNS update to add records ? Wouldn't that force named to be in sync ? Simo. Switching from LDAP modify operation to dynDNS update seems as a too big change to me. If nothing else, it would not fly with our LDAP ACI/permission system and ability to delegate DNS read/write rights to somebody else. Martin ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] DNS improvements: Should we add some sanity checking?
On 16.9.2013 09:06, Martin Kosek wrote: On 09/13/2013 06:17 PM, Simo Sorce wrote: On Fri, 2013-09-13 at 09:29 +0200, Petr Spacek wrote: Hello list, Jan Pazdziora jpazdzi...@redhat.com proposed that 'ipa dns*' commands should do some sanity checking/waiting after the record is added to LDAP. I think that it could be valuable and I would like to get opinions from freeipa-devel list. === The problem === ipa dnsrecord-add and similar commands add the data to LDAP, but it doesn't mean that the data are *immediately* resolvable via DNS protocol. Note that data from LDAP are *asynchronously* read and processed by Named and the time when records are available is not predictable. A mismatch between LDAP can be caused by some connection problem between DNS and LDAP servers, LDAP or DNS server restart, or simply by a bug in DNS-LDAP synchronization code. (This is becomming more and more important if we consider the whole DNSSEC effort and related re-factoring.) My experience is that users are very confused if the ipa dnsrecord-add command says 'record added' but it is still not available via DNS. It is really hard to debug when you see the problem first 10 times :-) === The proposal === 1. Let FreeIPA framework to change DNS data in LDAP as we do now. 2. After each change, do DNS queries for changed record and wait until the new data are available. IMHO it is very cheap operation (in usual cases 1 DNS packet back and forth) and it would save a lot of headaches to users and support. This will naturally catch the case where named crashes after the change etc. === Expected outcome === There will not be any failure like this: $ ipa-adtrust-install $ ipa dnszone-add $AD_DOMAIN --name-server=advm.$AD_DOMAIN --admin-email=hostmaster@$AD_DOMAIN.com --force --forwarder=$AD_IP --forward-policy=only --ip-address=$AD_IP Zone name: dom123.example.com [...] $ ipa trust-add --type=ad DOM123.EXAMPLE.COM --admin Administrator --password Password for ad...@dom123.example.com: ipa: ERROR: Cannot find specified domain or server name Would it make sense to change the code to use dynDNS update to add records ? Wouldn't that force named to be in sync ? Simo. Switching from LDAP modify operation to dynDNS update seems as a too big change to me. If nothing else, it would not fly with our LDAP ACI/permission system and ability to delegate DNS read/write rights to somebody else. I can see pros and cons for both ways: LDAP: + we have the code :-) + ACI magic - works only with bind-dyndb-ldap - can get out of sync (bugs, timeouts etc.) Standard DNS updates: + can work with any DNS server + with AD integration, we could use existing AD DNS infrastructure: i.e. manage DNS records for FreeIPA servers and host without deploying a new DNS server and related 'politics' + bind-dyndb-ldap is not necessary (ouh, my work is useless now :-)) - we don't have the code in FreeIPA framework - ACI magic is not available (in reality, it depends on the DNS server) - reading of current state could be more complex for user interface (On the other hand, current user interface doesn't show real state of thing because LDAP != DNS.) -- Petr^2 Spacek ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] DNS improvements: Should we add some sanity checking?
On Mon, 2013-09-16 at 09:45 +0200, Petr Spacek wrote: On 16.9.2013 09:06, Martin Kosek wrote: On 09/13/2013 06:17 PM, Simo Sorce wrote: On Fri, 2013-09-13 at 09:29 +0200, Petr Spacek wrote: Hello list, Jan Pazdziora jpazdzi...@redhat.com proposed that 'ipa dns*' commands should do some sanity checking/waiting after the record is added to LDAP. I think that it could be valuable and I would like to get opinions from freeipa-devel list. === The problem === ipa dnsrecord-add and similar commands add the data to LDAP, but it doesn't mean that the data are *immediately* resolvable via DNS protocol. Note that data from LDAP are *asynchronously* read and processed by Named and the time when records are available is not predictable. A mismatch between LDAP can be caused by some connection problem between DNS and LDAP servers, LDAP or DNS server restart, or simply by a bug in DNS-LDAP synchronization code. (This is becomming more and more important if we consider the whole DNSSEC effort and related re-factoring.) My experience is that users are very confused if the ipa dnsrecord-add command says 'record added' but it is still not available via DNS. It is really hard to debug when you see the problem first 10 times :-) === The proposal === 1. Let FreeIPA framework to change DNS data in LDAP as we do now. 2. After each change, do DNS queries for changed record and wait until the new data are available. IMHO it is very cheap operation (in usual cases 1 DNS packet back and forth) and it would save a lot of headaches to users and support. This will naturally catch the case where named crashes after the change etc. === Expected outcome === There will not be any failure like this: $ ipa-adtrust-install $ ipa dnszone-add $AD_DOMAIN --name-server=advm.$AD_DOMAIN --admin-email=hostmaster@$AD_DOMAIN.com --force --forwarder=$AD_IP --forward-policy=only --ip-address=$AD_IP Zone name: dom123.example.com [...] $ ipa trust-add --type=ad DOM123.EXAMPLE.COM --admin Administrator --password Password for ad...@dom123.example.com: ipa: ERROR: Cannot find specified domain or server name Would it make sense to change the code to use dynDNS update to add records ? Wouldn't that force named to be in sync ? Simo. Switching from LDAP modify operation to dynDNS update seems as a too big change to me. If nothing else, it would not fly with our LDAP ACI/permission system and ability to delegate DNS read/write rights to somebody else. I can see pros and cons for both ways: LDAP: + we have the code :-) + ACI magic - works only with bind-dyndb-ldap - can get out of sync (bugs, timeouts etc.) Standard DNS updates: + can work with any DNS server + with AD integration, we could use existing AD DNS infrastructure: i.e. manage DNS records for FreeIPA servers and host without deploying a new DNS server and related 'politics' + bind-dyndb-ldap is not necessary (ouh, my work is useless now :-)) - we don't have the code in FreeIPA framework - ACI magic is not available (in reality, it depends on the DNS server) - reading of current state could be more complex for user interface (On the other hand, current user interface doesn't show real state of thing because LDAP != DNS.) I forgot one thing that breaks, we cannot create new zones via dyndns, so we'd still have a mixed set. But I was thinking about your pros too, esp being able to use an AD DNS if necessary (evil but doable). I do not want to insist, because I also agree with Martin, but we should think about it. Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] DNS improvements: Should we add some sanity checking?
On 09/16/2013 09:32 AM, Simo Sorce wrote: On Mon, 2013-09-16 at 09:45 +0200, Petr Spacek wrote: On 16.9.2013 09:06, Martin Kosek wrote: On 09/13/2013 06:17 PM, Simo Sorce wrote: On Fri, 2013-09-13 at 09:29 +0200, Petr Spacek wrote: Hello list, Jan Pazdziora jpazdzi...@redhat.com proposed that 'ipa dns*' commands should do some sanity checking/waiting after the record is added to LDAP. I think that it could be valuable and I would like to get opinions from freeipa-devel list. === The problem === ipa dnsrecord-add and similar commands add the data to LDAP, but it doesn't mean that the data are *immediately* resolvable via DNS protocol. Note that data from LDAP are *asynchronously* read and processed by Named and the time when records are available is not predictable. A mismatch between LDAP can be caused by some connection problem between DNS and LDAP servers, LDAP or DNS server restart, or simply by a bug in DNS-LDAP synchronization code. (This is becomming more and more important if we consider the whole DNSSEC effort and related re-factoring.) My experience is that users are very confused if the ipa dnsrecord-add command says 'record added' but it is still not available via DNS. It is really hard to debug when you see the problem first 10 times :-) === The proposal === 1. Let FreeIPA framework to change DNS data in LDAP as we do now. 2. After each change, do DNS queries for changed record and wait until the new data are available. IMHO it is very cheap operation (in usual cases 1 DNS packet back and forth) and it would save a lot of headaches to users and support. This will naturally catch the case where named crashes after the change etc. === Expected outcome === There will not be any failure like this: $ ipa-adtrust-install $ ipa dnszone-add $AD_DOMAIN --name-server=advm.$AD_DOMAIN --admin-email=hostmaster@$AD_DOMAIN.com --force --forwarder=$AD_IP --forward-policy=only --ip-address=$AD_IP Zone name: dom123.example.com [...] $ ipa trust-add --type=ad DOM123.EXAMPLE.COM --admin Administrator --password Password for ad...@dom123.example.com: ipa: ERROR: Cannot find specified domain or server name Would it make sense to change the code to use dynDNS update to add records ? Wouldn't that force named to be in sync ? Simo. Switching from LDAP modify operation to dynDNS update seems as a too big change to me. If nothing else, it would not fly with our LDAP ACI/permission system and ability to delegate DNS read/write rights to somebody else. I can see pros and cons for both ways: LDAP: + we have the code :-) + ACI magic - works only with bind-dyndb-ldap - can get out of sync (bugs, timeouts etc.) Standard DNS updates: + can work with any DNS server + with AD integration, we could use existing AD DNS infrastructure: i.e. manage DNS records for FreeIPA servers and host without deploying a new DNS server and related 'politics' + bind-dyndb-ldap is not necessary (ouh, my work is useless now :-)) - we don't have the code in FreeIPA framework - ACI magic is not available (in reality, it depends on the DNS server) - reading of current state could be more complex for user interface (On the other hand, current user interface doesn't show real state of thing because LDAP != DNS.) I forgot one thing that breaks, we cannot create new zones via dyndns, so we'd still have a mixed set. But I was thinking about your pros too, esp being able to use an AD DNS if necessary (evil but doable). I do not want to insist, because I also agree with Martin, but we should think about it. ... and not rush Simo. -- Thank you, Dmitri Pal Sr. Engineering Manager for IdM portfolio Red Hat Inc. --- Looking to carve out IT costs? www.redhat.com/carveoutcosts/ ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] DNS improvements: Should we add some sanity checking?
On 09/13/2013 10:18 AM, Tomas Babej wrote: On 09/13/2013 09:29 AM, Petr Spacek wrote: Hello list, Jan Pazdziora jpazdzi...@redhat.com proposed that 'ipa dns*' commands should do some sanity checking/waiting after the record is added to LDAP. I think that it could be valuable and I would like to get opinions from freeipa-devel list. +1! === The problem === ipa dnsrecord-add and similar commands add the data to LDAP, but it doesn't mean that the data are *immediately* resolvable via DNS protocol. Note that data from LDAP are *asynchronously* read and processed by Named and the time when records are available is not predictable. A mismatch between LDAP can be caused by some connection problem between DNS and LDAP servers, LDAP or DNS server restart, or simply by a bug in DNS-LDAP synchronization code. (This is becomming more and more important if we consider the whole DNSSEC effort and related re-factoring.) My experience is that users are very confused if the ipa dnsrecord-add command says 'record added' but it is still not available via DNS. It is really hard to debug when you see the problem first 10 times :-) === The proposal === 1. Let FreeIPA framework to change DNS data in LDAP as we do now. 2. After each change, do DNS queries for changed record and wait until the new data are available. IMHO it is very cheap operation (in usual cases 1 DNS packet back and forth) and it would save a lot of headaches to users and support. We should make sure that we do not wait indefinitely here in case there's something else wrong with the named. We could wait for DNS data to be made available up to small reasonable timeout. If the check succeeds, we can output Verified: Yes along with the usual ipa dns(whatever) command output. Otherwise, we could print out Verified: No I think we should rather add an error message to the output: http://www.freeipa.org/page/V3/Messages However, it would be nice to print out Verified: No in a somewhat emphasized manner. I created the following ticket: https://fedorahosted.org/freeipa/ticket/3930 Messages should already stand out so they won't get lost in the output. (Which doesn't mean we can't also make them red, if someone wants to do contribute that.) -- Petr³ ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] DNS improvements: Should we add some sanity checking?
On Fri, 2013-09-13 at 09:29 +0200, Petr Spacek wrote: Hello list, Jan Pazdziora jpazdzi...@redhat.com proposed that 'ipa dns*' commands should do some sanity checking/waiting after the record is added to LDAP. I think that it could be valuable and I would like to get opinions from freeipa-devel list. === The problem === ipa dnsrecord-add and similar commands add the data to LDAP, but it doesn't mean that the data are *immediately* resolvable via DNS protocol. Note that data from LDAP are *asynchronously* read and processed by Named and the time when records are available is not predictable. A mismatch between LDAP can be caused by some connection problem between DNS and LDAP servers, LDAP or DNS server restart, or simply by a bug in DNS-LDAP synchronization code. (This is becomming more and more important if we consider the whole DNSSEC effort and related re-factoring.) My experience is that users are very confused if the ipa dnsrecord-add command says 'record added' but it is still not available via DNS. It is really hard to debug when you see the problem first 10 times :-) === The proposal === 1. Let FreeIPA framework to change DNS data in LDAP as we do now. 2. After each change, do DNS queries for changed record and wait until the new data are available. IMHO it is very cheap operation (in usual cases 1 DNS packet back and forth) and it would save a lot of headaches to users and support. This will naturally catch the case where named crashes after the change etc. === Expected outcome === There will not be any failure like this: $ ipa-adtrust-install $ ipa dnszone-add $AD_DOMAIN --name-server=advm.$AD_DOMAIN --admin-email=hostmaster@$AD_DOMAIN.com --force --forwarder=$AD_IP --forward-policy=only --ip-address=$AD_IP Zone name: dom123.example.com [...] $ ipa trust-add --type=ad DOM123.EXAMPLE.COM --admin Administrator --password Password for ad...@dom123.example.com: ipa: ERROR: Cannot find specified domain or server name Would it make sense to change the code to use dynDNS update to add records ? Wouldn't that force named to be in sync ? Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] DNS improvements: Should we add some sanity checking?
On 09/13/2013 09:29 AM, Petr Spacek wrote: Hello list, Jan Pazdziora jpazdzi...@redhat.com proposed that 'ipa dns*' commands should do some sanity checking/waiting after the record is added to LDAP. I think that it could be valuable and I would like to get opinions from freeipa-devel list. +1! === The problem === ipa dnsrecord-add and similar commands add the data to LDAP, but it doesn't mean that the data are *immediately* resolvable via DNS protocol. Note that data from LDAP are *asynchronously* read and processed by Named and the time when records are available is not predictable. A mismatch between LDAP can be caused by some connection problem between DNS and LDAP servers, LDAP or DNS server restart, or simply by a bug in DNS-LDAP synchronization code. (This is becomming more and more important if we consider the whole DNSSEC effort and related re-factoring.) My experience is that users are very confused if the ipa dnsrecord-add command says 'record added' but it is still not available via DNS. It is really hard to debug when you see the problem first 10 times :-) === The proposal === 1. Let FreeIPA framework to change DNS data in LDAP as we do now. 2. After each change, do DNS queries for changed record and wait until the new data are available. IMHO it is very cheap operation (in usual cases 1 DNS packet back and forth) and it would save a lot of headaches to users and support. We should make sure that we do not wait indefinitely here in case there's something else wrong with the named. We could wait for DNS data to be made available up to small reasonable timeout. If the check succeeds, we can output Verified: Yes along with the usual ipa dns(whatever) command output. Otherwise, we could print out Verified: No $ ipa dnszone-add $AD_DOMAIN --name-server=advm.$AD_DOMAIN --admin-email=hostmaster@$AD_DOMAIN.com --force --forwarder=$AD_IP --forward-policy=only --ip-address=$AD_IP Zone name: tbad.ipa.com Authoritative nameserver: advm.tbad.ipa.com Administrator e-mail address: hostmaster.tbad.ipa.com.com. SOA serial: 1378285614 SOA refresh: 3600 SOA retry: 900 SOA expire: 1209600 SOA minimum: 3600 BIND update policy: grant DOM007.TBAD.IPA.COM krb5-self * A; grant DOM007.TBAD.IPA.COM krb5-self * ; grant DOM007.TBAD.IPA.COM krb5-self * SSHFP; Active zone: TRUE Dynamic update: FALSE Allow query: any; Allow transfer: none; Zone forwarders: 192.168.122.20 Forward policy: only Verified: Yes However, it would be nice to print out Verified: No in a somewhat emphasized manner. I created the following ticket: https://fedorahosted.org/freeipa/ticket/3930 This will naturally catch the case where named crashes after the change etc. === Expected outcome === There will not be any failure like this: We debugged this with Petr few days ago as part of CI testing for trusts, I'll just provide detailed explanation here: $ ipa-adtrust-install Ipa-adtrust-install restarts Directory Server as one of the installation steps. Named looses connection to the LDAP server and by default reconnects in 60 seconds. $ ipa dnszone-add $AD_DOMAIN --name-server=advm.$AD_DOMAIN --admin-email=hostmaster@$AD_DOMAIN.com --force --forwarder=$AD_IP --forward-policy=only --ip-address=$AD_IP Zone name: dom123.example.com [...] Ipa dnszone-add writes to LDAP and reports success. $ ipa trust-add --type=ad DOM123.EXAMPLE.COM --admin Administrator --password Password for ad...@dom123.example.com: ipa: ERROR: Cannot find specified domain or server name Named is unable to find the domain, since the connection is down. -- Tomas Babej Associate Software Engeneer | Red Hat | Identity Management RHCE | Brno Site | IRC: tbabej | freeipa.org ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
