Re: [Freeipa-users] Unable to start the krb5kdc
On Thu, 27 Jan 2011 19:20:02 -0500 James Roman james.ro...@ssaihq.com wrote: On 1/27/11 12:58 PM, Simo Sorce wrote: On Wed, 2011-01-26 at 13:59 -0500, James Roman wrote: So it looks like the replication password issue was a red herring as far as the kerberos is concerned. I issued the command ipa-replica-manage synch ipaserver1.domain.com from the working ldap replica and no longer get password expiration errors in the error logs. However, I still can not get the krb5kdc process on ipaserver1 to start when it uses the local (ldap://127.0.0.1/) LDAP database. If I perform an LDAP search of the kdc account using the Directory Manager account, both kdc entries are identical, so it does not seem to be the password for the KDC account that is preventing the krb5kdc service from starting. Could it be the service or host principals? Should I init from ipaserver2 - ipaserver1 (Note: ipaserver1 is the winsync server)? ipaserver1: FC 11 ipa-server-1.2.2-2.fc11.i586 ipaserver2: FC10 ipa-server-1.2.2-1.fc10.i386 I am surprised you get back INVALID CREDENTIALS as an error when the KDC tries to log in using the data in ldappwd, given it works against the other server ... If you search with directory manager the accounts on both servers, do you get back an identical userPassword field ? Simo. Yes, when I check the passwords are also identical. Odd. Have you ever played with DS password policies by chance ? Can you search explicitly for the paswwordExpirationTime on both uid=kdc accounts and see if it set by chance ? You need to search explicitly for the attribute as it is not returned by default. Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Unable to start the krb5kdc
On 1/28/11 8:28 AM, Simo Sorce wrote: On Thu, 27 Jan 2011 19:20:02 -0500 James Romanjames.ro...@ssaihq.com wrote: On 1/27/11 12:58 PM, Simo Sorce wrote: On Wed, 2011-01-26 at 13:59 -0500, James Roman wrote: So it looks like the replication password issue was a red herring as far as the kerberos is concerned. I issued the command ipa-replica-manage synch ipaserver1.domain.com from the working ldap replica and no longer get password expiration errors in the error logs. However, I still can not get the krb5kdc process on ipaserver1 to start when it uses the local (ldap://127.0.0.1/) LDAP database. If I perform an LDAP search of the kdc account using the Directory Manager account, both kdc entries are identical, so it does not seem to be the password for the KDC account that is preventing the krb5kdc service from starting. Could it be the service or host principals? Should I init from ipaserver2 - ipaserver1 (Note: ipaserver1 is the winsync server)? ipaserver1: FC 11 ipa-server-1.2.2-2.fc11.i586 ipaserver2: FC10 ipa-server-1.2.2-1.fc10.i386 I am surprised you get back INVALID CREDENTIALS as an error when the KDC tries to log in using the data in ldappwd, given it works against the other server ... If you search with directory manager the accounts on both servers, do you get back an identical userPassword field ? Simo. Yes, when I check the passwords are also identical. Odd. Have you ever played with DS password policies by chance ? Can you search explicitly for the paswwordExpirationTime on both uid=kdc accounts and see if it set by chance ? You need to search explicitly for the attribute as it is not returned by default. Simo. OK. Now I feel like an idiot. I swear that was the first thing I checked. It seems the password policy on this server was set at the base, instead of cn=users. We have a script that reports on expiring accounts in the cn=accounts branch, but not under cn=etc. I now know what to fix. Thanks. ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Unable to start the krb5kdc
On Fri, 28 Jan 2011 09:20:37 -0500 James Roman james.ro...@ssaihq.com wrote: OK. Now I feel like an idiot. I swear that was the first thing I checked. It seems the password policy on this server was set at the base, instead of cn=users. We have a script that reports on expiring accounts in the cn=accounts branch, but not under cn=etc. I now know what to fix. Thanks. Rirst of all. I am glad this was resolved, it looked puzzling indeed. I just want to note that we do not support using the DS password policy in ipa as we already have the kerberos pw policy, that's why the uid=kdc was not protected against it. In v2 we perfected the pw policies check so that the kerberos policies covers also binds done against DS directly. I also am adding a patch so that uid=kdc is protected in case DS policy is enabled nonetheless for whatever reason. Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Unable to start the krb5kdc
On 01/28/2011 10:39 AM, Simo Sorce wrote: Rirst of all. I am glad this was resolved, it looked puzzling indeed. I just want to note that we do not support using the DS password policy in ipa as we already have the kerberos pw policy, that's why the uid=kdc was not protected against it. In v2 we perfected the pw policies check so that the kerberos policies covers also binds done against DS directly. Just to clarify, in v2 Kerberos password policies also cover ldap binds? I also am adding a patch so that uid=kdc is protected in case DS policy is enabled nonetheless for whatever reason. Simo. ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Unable to start the krb5kdc
On Fri, 28 Jan 2011 17:39:14 -0500 James Roman james.ro...@ssaihq.com wrote: On 01/28/2011 10:39 AM, Simo Sorce wrote: Rirst of all. I am glad this was resolved, it looked puzzling indeed. I just want to note that we do not support using the DS password policy in ipa as we already have the kerberos pw policy, that's why the uid=kdc was not protected against it. In v2 we perfected the pw policies check so that the kerberos policies covers also binds done against DS directly. Just to clarify, in v2 Kerberos password policies also cover ldap binds? Yes with have a bind pre/post op plugin that enforces the same account/password policies for ldap binds too. Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Unable to start the krb5kdc
On 1/27/11 12:58 PM, Simo Sorce wrote: On Wed, 2011-01-26 at 13:59 -0500, James Roman wrote: So it looks like the replication password issue was a red herring as far as the kerberos is concerned. I issued the command ipa-replica-manage synch ipaserver1.domain.com from the working ldap replica and no longer get password expiration errors in the error logs. However, I still can not get the krb5kdc process on ipaserver1 to start when it uses the local (ldap://127.0.0.1/) LDAP database. If I perform an LDAP search of the kdc account using the Directory Manager account, both kdc entries are identical, so it does not seem to be the password for the KDC account that is preventing the krb5kdc service from starting. Could it be the service or host principals? Should I init from ipaserver2 - ipaserver1 (Note: ipaserver1 is the winsync server)? ipaserver1: FC 11 ipa-server-1.2.2-2.fc11.i586 ipaserver2: FC10 ipa-server-1.2.2-1.fc10.i386 I am surprised you get back INVALID CREDENTIALS as an error when the KDC tries to log in using the data in ldappwd, given it works against the other server ... If you search with directory manager the accounts on both servers, do you get back an identical userPassword field ? Simo. Yes, when I check the passwords are also identical. ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Unable to start the krb5kdc
On 01/26/2011 09:32 AM, James Roman wrote: Simo Sorce wrote: On Tue, 25 Jan 2011 15:58:35 -0500 James Romanjames.ro...@ssaihq.com wrote: On 1/25/11 2:44 PM, Simo Sorce wrote: On Tue, 25 Jan 2011 14:33:14 -0500 James Romanjames.ro...@ssaihq.com wrote: On 01/25/2011 12:42 PM, Simo Sorce wrote: On Tue, 25 Jan 2011 12:04:25 -0500 James Romanjames.ro...@ssaihq.comwrote: I noticed today that one of our FreeIPA 1.2.2 servers has stopped issuing tickets. When I attempt to restart all the IPA services the krb5kdc service failed to restart with the following error: krb5kdc: Unable to access Kerberos database - while initializing database for realm DOMAIN.COM I don't see any issues with the local LDAP database, or the kdc account in the LDAP database. I suspect the problem is with the ticket granting ticket on the problem server, but am unsure how to go about validating this assertion. I have not tried to restart the ipa services on the working server for fera that it might stop working. Do you see errors in /var/log/krb5kdc.log ? Simo. The error above is the only one that repeats in the krb5kdc.log when I attempt to restart the krb5kdc service. The actual error that is shown in standard out is: Starting Kerberos 5 KDC: krb5kdc: cannot initialize realm DOMAIN.COM - see log file for details Ok can you check the dirsrv logs and see if the KDC is actually trying (and perhaps getting auth refused) at all ? /var/log/dirsrv/slapd-DOMAIN-COM/access should show your KDC attempts to access the LDAP server and bind as the uid=kdc. user. Simo. Looks like an authentication failure: [25/Jan/2011:15:11:29 -0500] conn=391 op=0 BIND dn=uid=kdc,cn=sysaccounts,cn=etc,dc=domain,dc=com method=128 version=3 [25/Jan/2011:15:11:29 -0500] conn=391 op=0 RESULT err=49 tag=97 nentries=0 etime=0 [25/Jan/2011:15:11:29 -0500] conn=391 op=-1 fd=73 closed - B1 The ldappwd file on both systems look identical. I don't think that the SSL certificate comes into the equation, but I have no way of knowing whether it initiates TLS or not. No in ipa 1.2.x the kdc is configured to use ldap://127.0.0.1 with no auth. I wonder if your local DS is having problems. Can you change krb5.conf to point to the other server (maybe using ldaps:// so as to not expose the password in the clear) and see if the krb5kdc will start that way ? Don't use this in production, just as a test to identify where the problem lies. if it turns out it is the local DS that is having issues, then we can try to force sync it again. Ah btw, on what distribution version is this? what 389-ds base version are you using ? Simo. So if I switch the kdc.conf to point to the other FreeIPA ldap server the krb5kdc service starts up without any problems. I was just about to force a sync when I noticed this in the error log on the working ldap server (lets call it ipserver2): [17/Jan/2011:10:24:33 -0500] NSMMReplicationPlugin - agmt=cn=meToipaserver1.domain.com636 (ipaserver1:636): Succesfully bound cn=replication manager,cn=config to consumer, but password has expired on consumer. This is the earliest record I have on the ldap replica without going to tape. So it appears that the replica password has expired. So I have this problem. ipaserver1 is used as my winsync server, but I can not use it to start krb5kdc. ipaserver2 has a working ldap server, but is not synchronizing with the winsync master. If I fix the password expiration issue, is it going to break ipaserver2?\ See here for information about how to make the repl manager password not expire - http://docs.redhat.com/docs/en-US/Red_Hat_Directory_Server/8.2/html-single/Administration_Guide/index.html#Creating_the_Supplier_Bind_DN_Entry if you fix the password expiration issue, it should not break anything ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Unable to start the krb5kdc
On Tue, 25 Jan 2011 12:04:25 -0500 James Roman james.ro...@ssaihq.com wrote: I noticed today that one of our FreeIPA 1.2.2 servers has stopped issuing tickets. When I attempt to restart all the IPA services the krb5kdc service failed to restart with the following error: krb5kdc: Unable to access Kerberos database - while initializing database for realm DOMAIN.COM I don't see any issues with the local LDAP database, or the kdc account in the LDAP database. I suspect the problem is with the ticket granting ticket on the problem server, but am unsure how to go about validating this assertion. I have not tried to restart the ipa services on the working server for fera that it might stop working. Do you see errors in /var/log/krb5kdc.log ? Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Unable to start the krb5kdc
On 01/25/2011 01:58 PM, James Roman wrote: On 1/25/11 2:44 PM, Simo Sorce wrote: On Tue, 25 Jan 2011 14:33:14 -0500 James Romanjames.ro...@ssaihq.com wrote: On 01/25/2011 12:42 PM, Simo Sorce wrote: On Tue, 25 Jan 2011 12:04:25 -0500 James Romanjames.ro...@ssaihq.com wrote: I noticed today that one of our FreeIPA 1.2.2 servers has stopped issuing tickets. When I attempt to restart all the IPA services the krb5kdc service failed to restart with the following error: krb5kdc: Unable to access Kerberos database - while initializing database for realm DOMAIN.COM I don't see any issues with the local LDAP database, or the kdc account in the LDAP database. I suspect the problem is with the ticket granting ticket on the problem server, but am unsure how to go about validating this assertion. I have not tried to restart the ipa services on the working server for fera that it might stop working. Do you see errors in /var/log/krb5kdc.log ? Simo. The error above is the only one that repeats in the krb5kdc.log when I attempt to restart the krb5kdc service. The actual error that is shown in standard out is: Starting Kerberos 5 KDC: krb5kdc: cannot initialize realm DOMAIN.COM - see log file for details Ok can you check the dirsrv logs and see if the KDC is actually trying (and perhaps getting auth refused) at all ? /var/log/dirsrv/slapd-DOMAIN-COM/access should show your KDC attempts to access the LDAP server and bind as the uid=kdc. user. Simo. Looks like an authentication failure: [25/Jan/2011:15:11:29 -0500] conn=391 op=0 BIND dn=uid=kdc,cn=sysaccounts,cn=etc,dc=domain,dc=com method=128 version=3 [25/Jan/2011:15:11:29 -0500] conn=391 op=0 RESULT err=49 tag=97 nentries=0 etime=0 [25/Jan/2011:15:11:29 -0500] conn=391 op=-1 fd=73 closed - B1 The ldappwd file on both systems look identical. I don't think that the SSL certificate comes into the equation, but I have no way of knowing whether it initiates TLS or not. You can tell if the connection is using TLS/SSL because when the connection is opened you should see a log line that says what cipher suite is being used You can tell if client cert auth is being used because there will be a line for that too. Look for conn=391 lines before this one ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] Unable to start the krb5kdc
On Tue, 25 Jan 2011 15:58:35 -0500 James Roman james.ro...@ssaihq.com wrote: On 1/25/11 2:44 PM, Simo Sorce wrote: On Tue, 25 Jan 2011 14:33:14 -0500 James Romanjames.ro...@ssaihq.com wrote: On 01/25/2011 12:42 PM, Simo Sorce wrote: On Tue, 25 Jan 2011 12:04:25 -0500 James Romanjames.ro...@ssaihq.com wrote: I noticed today that one of our FreeIPA 1.2.2 servers has stopped issuing tickets. When I attempt to restart all the IPA services the krb5kdc service failed to restart with the following error: krb5kdc: Unable to access Kerberos database - while initializing database for realm DOMAIN.COM I don't see any issues with the local LDAP database, or the kdc account in the LDAP database. I suspect the problem is with the ticket granting ticket on the problem server, but am unsure how to go about validating this assertion. I have not tried to restart the ipa services on the working server for fera that it might stop working. Do you see errors in /var/log/krb5kdc.log ? Simo. The error above is the only one that repeats in the krb5kdc.log when I attempt to restart the krb5kdc service. The actual error that is shown in standard out is: Starting Kerberos 5 KDC: krb5kdc: cannot initialize realm DOMAIN.COM - see log file for details Ok can you check the dirsrv logs and see if the KDC is actually trying (and perhaps getting auth refused) at all ? /var/log/dirsrv/slapd-DOMAIN-COM/access should show your KDC attempts to access the LDAP server and bind as the uid=kdc. user. Simo. Looks like an authentication failure: [25/Jan/2011:15:11:29 -0500] conn=391 op=0 BIND dn=uid=kdc,cn=sysaccounts,cn=etc,dc=domain,dc=com method=128 version=3 [25/Jan/2011:15:11:29 -0500] conn=391 op=0 RESULT err=49 tag=97 nentries=0 etime=0 [25/Jan/2011:15:11:29 -0500] conn=391 op=-1 fd=73 closed - B1 The ldappwd file on both systems look identical. I don't think that the SSL certificate comes into the equation, but I have no way of knowing whether it initiates TLS or not. No in ipa 1.2.x the kdc is configured to use ldap://127.0.0.1 with no auth. I wonder if your local DS is having problems. Can you change krb5.conf to point to the other server (maybe using ldaps:// so as to not expose the password in the clear) and see if the krb5kdc will start that way ? Don't use this in production, just as a test to identify where the problem lies. if it turns out it is the local DS that is having issues, then we can try to force sync it again. Ah btw, on what distribution version is this? what 389-ds base version are you using ? Simo. -- Simo Sorce * Red Hat, Inc * New York ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
