Re: [Freeipa-users] multi homed environment
On Fri, May 08, 2015 at 05:21:09PM +0300, Alexander Bokovoy wrote: On Fri, 08 May 2015, Andy Thompson wrote: On Fri, 08 May 2015, Andy Thompson wrote: I'm having an issue with adding a trust to the domain with the error below ipa: ERROR: CIFS server communication error: code -1073741801, message Memory allocation error (both may be None) And I'm ashamed to say I tracked down the issue to a fat finger in the resolv.conf file, so it really couldn't look up the needed record :/ In any case, it is mostly a question of correct routing tables and DNS name resolution. Is there anything we can add to the tool on our side to catch the errors earlier and/or make the error messages less scary and more descriptive? -- Jan Pazdziora Senior Principal Software Engineer, Identity Management Engineering, Red Hat -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] multi homed environment
-Original Message- From: freeipa-users-boun...@redhat.com [mailto:freeipa-users- boun...@redhat.com] On Behalf Of Jan Pazdziora Sent: Monday, May 11, 2015 8:14 AM To: Alexander Bokovoy Cc: freeipa-users@redhat.com Subject: Re: [Freeipa-users] multi homed environment On Fri, May 08, 2015 at 05:21:09PM +0300, Alexander Bokovoy wrote: On Fri, 08 May 2015, Andy Thompson wrote: On Fri, 08 May 2015, Andy Thompson wrote: I'm having an issue with adding a trust to the domain with the error below ipa: ERROR: CIFS server communication error: code -1073741801, message Memory allocation error (both may be None) And I'm ashamed to say I tracked down the issue to a fat finger in the resolv.conf file, so it really couldn't look up the needed record :/ In any case, it is mostly a question of correct routing tables and DNS name resolution. Is there anything we can add to the tool on our side to catch the errors earlier and/or make the error messages less scary and more descriptive? Possibly error out and stop the setup entirely if DNS can't be resolved... make it a pre setup check and halt. Currently it allows the install to continue, albeit saying what happened. I just didn't pay close enough attention the first time around to see that it failed that step... I think I started it, went to another screen and came back and noted it was completed and moved forward. -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] multi homed environment
On Fri, 08 May 2015, Andy Thompson wrote: -Original Message- From: Alexander Bokovoy [mailto:aboko...@redhat.com] Sent: Friday, May 8, 2015 9:40 AM To: Andy Thompson Cc: freeipa-users@redhat.com Subject: Re: [Freeipa-users] multi homed environment On Fri, 08 May 2015, Andy Thompson wrote: -Original Message- From: Alexander Bokovoy [mailto:aboko...@redhat.com] Sent: Friday, May 8, 2015 8:17 AM To: Andy Thompson Cc: freeipa-users@redhat.com Subject: Re: [Freeipa-users] multi homed environment On Fri, 08 May 2015, Andy Thompson wrote: I'm trying to roll out IPA in an existing windows environment where everything is multi homed. I did not put my IPA server on all the subnets. I'm having an issue with adding a trust to the domain with the error below ipa: ERROR: CIFS server communication error: code -1073741801, message Memory allocation error (both may be None) DNS I think since it round robins all the existing A records and is returning IPs out of the local subnet. I don't know much about windows dns services but it's got netmask optimization enabled and doing digs against the service returns the local IP first every time, but pings return them in any order. I've considered adding the DCs to the local hosts file but I'm not sure if that will solve the problem or not. Is that a viable fix? Anyone have any experience in an environment like this? Really not sure what additional problems I will run into with all this multi homed nonsense. Stop here and make sure you obtained the debugging information as described in http://www.freeipa.org/page/Active_Directory_trust_setup#Debugging_tr u st Without that information it is hard to tell what is happening. Make also sure to tell exact environment (distribution, version, package versions, etc). Well things got ugly. I enabled debug and pointed in the right direction, smb failed to start. Came down to the cifs service was not added when I did the adtrust-install. I tried adding it and it complained that it could not find the A record for the host even though it was there. Thinking something was hung up in resolver cache possibly I restarted the ipa service and it failed completely. Ipactl start fails starting smb because of the missing service and everything fails from there. Is there any way to recover from this mess I just made? :) I assume you have IPA 4.x, i.e. systemd-based environment. Yes, sorry forgot to include that. 1. Start manually dirsrv@INSTANCE-NAME.service 2. Disable ADTRUST and EXTID services with ipa-ldap-updater. Note that you SHOULD NOT replace $FOO variables below, they should be as specified in the resulting file. For ipa-ldap-updater use see its manual page and my blog: https://vda.li/en/posts/2015/01/02/playing-with-freeipa-ipa-ldap-updater/ # cat END 88-disable-adtrust-extid.update dn: cn=ADTRUST,cn=$FQDN,cn=masters,cn=ipa,cn=etc,$SUFFIX remove:ipaConfigString:enabledService dn: cn=EXTID,cn=$FQDN,cn=masters,cn=ipa,cn=etc,$SUFFIX remove:ipaConfigString:enabledService END # ipa-ldap-updater -l ./88-disable-adtrust-extid.update 3. Restart IPA 4. Re-run ipa-adtrust-install and look at the output, including what it appends to /var/log/ipaserver-install.log. Beautiful, that much is running again, thanks for those pointers. And I'm ashamed to say I tracked down the issue to a fat finger in the resolv.conf file, so it really couldn't look up the needed record :/ So back to the original issue that was in the end because smb wasn't started most likely. I'm still not sure how this will all respond in a multi homed environment like this if the IPA server cannot communicate with all of the interfaces on the DC. Will that cause an issue with the trust or is there anything I need to take into consideration with this? There are few things to consider: 1. IPA master uses DNS SRV records to discover whom to talk to on AD side. Received name from the SRV record is them used by IPA master to connect to the AD DC. 2. AD DCs use DNS SRV records to discover which IPA master to respond to when verifying trust. Received name from the SRV record is then used by AD DC to connect to the IPA master. 3. While right now trust is established using password-based authentication between IPA and AD DCs, actual resolution of identities when trust is in use requires working Kerberos authentication. This might give you a headache in multi-homed environments if the IP returned when resolving AD DC or IPA master would be unreachable. In any case, it is mostly a question of correct routing tables and DNS name resolution. -- / Alexander Bokovoy -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] multi homed environment
-Original Message- From: Alexander Bokovoy [mailto:aboko...@redhat.com] Sent: Friday, May 8, 2015 10:21 AM To: Andy Thompson Cc: freeipa-users@redhat.com Subject: Re: [Freeipa-users] multi homed environment On Fri, 08 May 2015, Andy Thompson wrote: -Original Message- From: Alexander Bokovoy [mailto:aboko...@redhat.com] Sent: Friday, May 8, 2015 9:40 AM To: Andy Thompson Cc: freeipa-users@redhat.com Subject: Re: [Freeipa-users] multi homed environment On Fri, 08 May 2015, Andy Thompson wrote: -Original Message- From: Alexander Bokovoy [mailto:aboko...@redhat.com] Sent: Friday, May 8, 2015 8:17 AM To: Andy Thompson Cc: freeipa-users@redhat.com Subject: Re: [Freeipa-users] multi homed environment On Fri, 08 May 2015, Andy Thompson wrote: I'm trying to roll out IPA in an existing windows environment where everything is multi homed. I did not put my IPA server on all the subnets. I'm having an issue with adding a trust to the domain with the error below ipa: ERROR: CIFS server communication error: code -1073741801, message Memory allocation error (both may be None) DNS I think since it round robins all the existing A records and is returning IPs out of the local subnet. I don't know much about windows dns services but it's got netmask optimization enabled and doing digs against the service returns the local IP first every time, but pings return them in any order. I've considered adding the DCs to the local hosts file but I'm not sure if that will solve the problem or not. Is that a viable fix? Anyone have any experience in an environment like this? Really not sure what additional problems I will run into with all this multi homed nonsense. Stop here and make sure you obtained the debugging information as described in http://www.freeipa.org/page/Active_Directory_trust_setup#Debugging_tr u st Without that information it is hard to tell what is happening. Make also sure to tell exact environment (distribution, version, package versions, etc). Well things got ugly. I enabled debug and pointed in the right direction, smb failed to start. Came down to the cifs service was not added when I did the adtrust-install. I tried adding it and it complained that it could not find the A record for the host even though it was there. Thinking something was hung up in resolver cache possibly I restarted the ipa service and it failed completely. Ipactl start fails starting smb because of the missing service and everything fails from there. Is there any way to recover from this mess I just made? :) I assume you have IPA 4.x, i.e. systemd-based environment. Yes, sorry forgot to include that. 1. Start manually dirsrv@INSTANCE-NAME.service 2. Disable ADTRUST and EXTID services with ipa-ldap-updater. Note that you SHOULD NOT replace $FOO variables below, they should be as specified in the resulting file. For ipa-ldap-updater use see its manual page and my blog: https://vda.li/en/posts/2015/01/02/playing-with-freeipa-ipa-ldap-upda ter/ # cat END 88-disable-adtrust-extid.update dn: cn=ADTRUST,cn=$FQDN,cn=masters,cn=ipa,cn=etc,$SUFFIX remove:ipaConfigString:enabledService dn: cn=EXTID,cn=$FQDN,cn=masters,cn=ipa,cn=etc,$SUFFIX remove:ipaConfigString:enabledService END # ipa-ldap-updater -l ./88-disable-adtrust-extid.update 3. Restart IPA 4. Re-run ipa-adtrust-install and look at the output, including what it appends to /var/log/ipaserver-install.log. Beautiful, that much is running again, thanks for those pointers. And I'm ashamed to say I tracked down the issue to a fat finger in the resolv.conf file, so it really couldn't look up the needed record :/ So back to the original issue that was in the end because smb wasn't started most likely. I'm still not sure how this will all respond in a multi homed environment like this if the IPA server cannot communicate with all of the interfaces on the DC. Will that cause an issue with the trust or is there anything I need to take into consideration with this? There are few things to consider: 1. IPA master uses DNS SRV records to discover whom to talk to on AD side. Received name from the SRV record is them used by IPA master to connect to the AD DC. 2. AD DCs use DNS SRV records to discover which IPA master to respond to when verifying trust. Received name from the SRV record is then used by AD DC to connect to the IPA master. 3. While right now trust is established using password-based authentication between IPA and AD DCs, actual resolution of identities when trust is in use requires working Kerberos authentication. This might give you a headache in multi-homed environments if the IP returned when resolving AD DC or IPA master would be unreachable. In any case
Re: [Freeipa-users] multi homed environment
On Fri, 08 May 2015, Andy Thompson wrote: -Original Message- From: Alexander Bokovoy [mailto:aboko...@redhat.com] Sent: Friday, May 8, 2015 8:17 AM To: Andy Thompson Cc: freeipa-users@redhat.com Subject: Re: [Freeipa-users] multi homed environment On Fri, 08 May 2015, Andy Thompson wrote: I'm trying to roll out IPA in an existing windows environment where everything is multi homed. I did not put my IPA server on all the subnets. I'm having an issue with adding a trust to the domain with the error below ipa: ERROR: CIFS server communication error: code -1073741801, message Memory allocation error (both may be None) DNS I think since it round robins all the existing A records and is returning IPs out of the local subnet. I don't know much about windows dns services but it's got netmask optimization enabled and doing digs against the service returns the local IP first every time, but pings return them in any order. I've considered adding the DCs to the local hosts file but I'm not sure if that will solve the problem or not. Is that a viable fix? Anyone have any experience in an environment like this? Really not sure what additional problems I will run into with all this multi homed nonsense. Stop here and make sure you obtained the debugging information as described in http://www.freeipa.org/page/Active_Directory_trust_setup#Debugging_tru st Without that information it is hard to tell what is happening. Make also sure to tell exact environment (distribution, version, package versions, etc). Well things got ugly. I enabled debug and pointed in the right direction, smb failed to start. Came down to the cifs service was not added when I did the adtrust-install. I tried adding it and it complained that it could not find the A record for the host even though it was there. Thinking something was hung up in resolver cache possibly I restarted the ipa service and it failed completely. Ipactl start fails starting smb because of the missing service and everything fails from there. Is there any way to recover from this mess I just made? :) I assume you have IPA 4.x, i.e. systemd-based environment. 1. Start manually dirsrv@INSTANCE-NAME.service 2. Disable ADTRUST and EXTID services with ipa-ldap-updater. Note that you SHOULD NOT replace $FOO variables below, they should be as specified in the resulting file. For ipa-ldap-updater use see its manual page and my blog: https://vda.li/en/posts/2015/01/02/playing-with-freeipa-ipa-ldap-updater/ # cat END 88-disable-adtrust-extid.update dn: cn=ADTRUST,cn=$FQDN,cn=masters,cn=ipa,cn=etc,$SUFFIX remove:ipaConfigString:enabledService dn: cn=EXTID,cn=$FQDN,cn=masters,cn=ipa,cn=etc,$SUFFIX remove:ipaConfigString:enabledService END # ipa-ldap-updater -l ./88-disable-adtrust-extid.update 3. Restart IPA 4. Re-run ipa-adtrust-install and look at the output, including what it appends to /var/log/ipaserver-install.log. -- / Alexander Bokovoy -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] multi homed environment
-Original Message- From: Alexander Bokovoy [mailto:aboko...@redhat.com] Sent: Friday, May 8, 2015 9:40 AM To: Andy Thompson Cc: freeipa-users@redhat.com Subject: Re: [Freeipa-users] multi homed environment On Fri, 08 May 2015, Andy Thompson wrote: -Original Message- From: Alexander Bokovoy [mailto:aboko...@redhat.com] Sent: Friday, May 8, 2015 8:17 AM To: Andy Thompson Cc: freeipa-users@redhat.com Subject: Re: [Freeipa-users] multi homed environment On Fri, 08 May 2015, Andy Thompson wrote: I'm trying to roll out IPA in an existing windows environment where everything is multi homed. I did not put my IPA server on all the subnets. I'm having an issue with adding a trust to the domain with the error below ipa: ERROR: CIFS server communication error: code -1073741801, message Memory allocation error (both may be None) DNS I think since it round robins all the existing A records and is returning IPs out of the local subnet. I don't know much about windows dns services but it's got netmask optimization enabled and doing digs against the service returns the local IP first every time, but pings return them in any order. I've considered adding the DCs to the local hosts file but I'm not sure if that will solve the problem or not. Is that a viable fix? Anyone have any experience in an environment like this? Really not sure what additional problems I will run into with all this multi homed nonsense. Stop here and make sure you obtained the debugging information as described in http://www.freeipa.org/page/Active_Directory_trust_setup#Debugging_tr u st Without that information it is hard to tell what is happening. Make also sure to tell exact environment (distribution, version, package versions, etc). Well things got ugly. I enabled debug and pointed in the right direction, smb failed to start. Came down to the cifs service was not added when I did the adtrust-install. I tried adding it and it complained that it could not find the A record for the host even though it was there. Thinking something was hung up in resolver cache possibly I restarted the ipa service and it failed completely. Ipactl start fails starting smb because of the missing service and everything fails from there. Is there any way to recover from this mess I just made? :) I assume you have IPA 4.x, i.e. systemd-based environment. Yes, sorry forgot to include that. 1. Start manually dirsrv@INSTANCE-NAME.service 2. Disable ADTRUST and EXTID services with ipa-ldap-updater. Note that you SHOULD NOT replace $FOO variables below, they should be as specified in the resulting file. For ipa-ldap-updater use see its manual page and my blog: https://vda.li/en/posts/2015/01/02/playing-with-freeipa-ipa-ldap-updater/ # cat END 88-disable-adtrust-extid.update dn: cn=ADTRUST,cn=$FQDN,cn=masters,cn=ipa,cn=etc,$SUFFIX remove:ipaConfigString:enabledService dn: cn=EXTID,cn=$FQDN,cn=masters,cn=ipa,cn=etc,$SUFFIX remove:ipaConfigString:enabledService END # ipa-ldap-updater -l ./88-disable-adtrust-extid.update 3. Restart IPA 4. Re-run ipa-adtrust-install and look at the output, including what it appends to /var/log/ipaserver-install.log. Beautiful, that much is running again, thanks for those pointers. And I'm ashamed to say I tracked down the issue to a fat finger in the resolv.conf file, so it really couldn't look up the needed record :/ So back to the original issue that was in the end because smb wasn't started most likely. I'm still not sure how this will all respond in a multi homed environment like this if the IPA server cannot communicate with all of the interfaces on the DC. Will that cause an issue with the trust or is there anything I need to take into consideration with this? Thanks much -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] multi homed environment
-Original Message- From: Alexander Bokovoy [mailto:aboko...@redhat.com] Sent: Friday, May 8, 2015 8:17 AM To: Andy Thompson Cc: freeipa-users@redhat.com Subject: Re: [Freeipa-users] multi homed environment On Fri, 08 May 2015, Andy Thompson wrote: I'm trying to roll out IPA in an existing windows environment where everything is multi homed. I did not put my IPA server on all the subnets. I'm having an issue with adding a trust to the domain with the error below ipa: ERROR: CIFS server communication error: code -1073741801, message Memory allocation error (both may be None) DNS I think since it round robins all the existing A records and is returning IPs out of the local subnet. I don't know much about windows dns services but it's got netmask optimization enabled and doing digs against the service returns the local IP first every time, but pings return them in any order. I've considered adding the DCs to the local hosts file but I'm not sure if that will solve the problem or not. Is that a viable fix? Anyone have any experience in an environment like this? Really not sure what additional problems I will run into with all this multi homed nonsense. Stop here and make sure you obtained the debugging information as described in http://www.freeipa.org/page/Active_Directory_trust_setup#Debugging_tru st Without that information it is hard to tell what is happening. Make also sure to tell exact environment (distribution, version, package versions, etc). Well things got ugly. I enabled debug and pointed in the right direction, smb failed to start. Came down to the cifs service was not added when I did the adtrust-install. I tried adding it and it complained that it could not find the A record for the host even though it was there. Thinking something was hung up in resolver cache possibly I restarted the ipa service and it failed completely. Ipactl start fails starting smb because of the missing service and everything fails from there. Is there any way to recover from this mess I just made? :) thanks -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
Re: [Freeipa-users] multi homed environment
On Fri, 08 May 2015, Andy Thompson wrote: I'm trying to roll out IPA in an existing windows environment where everything is multi homed. I did not put my IPA server on all the subnets. I'm having an issue with adding a trust to the domain with the error below ipa: ERROR: CIFS server communication error: code -1073741801, message Memory allocation error (both may be None) DNS I think since it round robins all the existing A records and is returning IPs out of the local subnet. I don't know much about windows dns services but it's got netmask optimization enabled and doing digs against the service returns the local IP first every time, but pings return them in any order. I've considered adding the DCs to the local hosts file but I'm not sure if that will solve the problem or not. Is that a viable fix? Anyone have any experience in an environment like this? Really not sure what additional problems I will run into with all this multi homed nonsense. Stop here and make sure you obtained the debugging information as described in http://www.freeipa.org/page/Active_Directory_trust_setup#Debugging_trust Without that information it is hard to tell what is happening. Make also sure to tell exact environment (distribution, version, package versions, etc). -- / Alexander Bokovoy -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project
