Gronde, Christopher (Contractor) wrote:
> This gave me a huge return!  Appears to be a long list of all the servers and 
> applications whose users authenticate to the IPA servers.
> 
> ldapsearch -x -h 172.16.100.161 -D "cn=directory manager" -W -b 
> "dc=itmodev,dc=gov" '(objectclass=krbprincipal)'
> 
> 
> # search result
> search: 2
> result: 0 Success
> 
> # numResponses: 142
> # numEntries: 141

Right, we need to see the sasl mapping:

$ ldapsearch -x -D 'cn=Directory Manager' -W -b cn=mapping,cn=sasl,cn=config

rob

> 
> -----Original Message-----
> From: freeipa-users-boun...@redhat.com 
> [mailto:freeipa-users-boun...@redhat.com] On Behalf Of Ludwig Krispenz
> Sent: Tuesday, November 10, 2015 11:37 AM
> To: freeipa-users@redhat.com
> Subject: Re: [Freeipa-users] krb5kdc will not start (kerberos authentication 
> error)
> 
> what do you get if you search for "objectclass=krbprincipal" ?
> 
> On 11/10/2015 05:27 PM, Rich Megginson wrote:
>> On 11/10/2015 09:16 AM, Gronde, Christopher (Contractor) wrote:
>>> Neither came back with anything
>>>
>>> # ldapsearch -x -h 172.16.100.161 -D "cn=directory manager" -W -b 
>>> "dc=itmodev,dc=gov" '(uid=ldap/comipa01.itmodev.gov)'
>>> Enter LDAP Password:
>>> # extended LDIF
>>> #
>>> # LDAPv3
>>> # base <dc=itmodev,dc=gov> with scope subtree # filter: 
>>> (uid=ldap/comipa01.itmodev.gov) # requesting: ALL #
>>>
>>> # search result
>>> search: 2
>>> result: 0 Success
>>>
>>> # numResponses: 1
>>> [root@comipa02 ~]# ldapsearch -x -h 172.16.100.161 -D "cn=directory 
>>> manager" -W -b "dc=itmodev,dc=gov" '(uid=ldap/*.gov)' uid Enter LDAP 
>>> Password:
>>> # extended LDIF
>>> #
>>> # LDAPv3
>>> # base <dc=itmodev,dc=gov> with scope subtree # filter: 
>>> (uid=ldap/*.gov) # requesting: uid #
>>>
>>> # search result
>>> search: 2
>>> result: 0 Success
>>>
>>> # numResponses: 1
>>
>> That means this server has no LDAP service principals?  I'm not sure 
>> how to recover IPA from this scenario.
>>
>>>
>>> -----Original Message-----
>>> From: freeipa-users-boun...@redhat.com 
>>> [mailto:freeipa-users-boun...@redhat.com] On Behalf Of Rich Megginson
>>> Sent: Tuesday, November 10, 2015 11:04 AM
>>> To: freeipa-users@redhat.com
>>> Subject: Re: [Freeipa-users] krb5kdc will not start (kerberos 
>>> authentication error)
>>>
>>> On 11/10/2015 08:18 AM, Gronde, Christopher (Contractor) wrote:
>>>> Thank you!  I should have caught that...
>>>>
>>>> I changed the log level and then restarted dirsrv and attempted to 
>>>> start krb5kdc and got the following...
>>> <snip>
>>>
>>> [10/Nov/2015:10:12:02 -0500] conn=5 fd=64 slot=64 connection from
>>> 172.16.100.208 to 172.16.100.161
>>> [10/Nov/2015:10:12:02 -0500] conn=5 op=0 BIND dn="" method=sasl
>>> version=3 mech=GSSAPI
>>> [10/Nov/2015:10:12:03 -0500] conn=5 op=0 RESULT err=14 tag=97
>>> nentries=0 etime=1, SASL bind in progress
>>> [10/Nov/2015:10:12:03 -0500] conn=5 op=1 BIND dn="" method=sasl
>>> version=3 mech=GSSAPI
>>> [10/Nov/2015:10:12:03 -0500] conn=5 op=1 RESULT err=14 tag=97
>>> nentries=0 etime=0, SASL bind in progress
>>> [10/Nov/2015:10:12:03 -0500] conn=5 op=2 BIND dn="" method=sasl
>>> version=3 mech=GSSAPI
>>> [10/Nov/2015:10:12:03 -0500] conn=Internal op=-1 SRCH 
>>> base="dc=itmodev,dc=gov" scope=2 
>>> filter="(uid=ldap/comipa01.itmodev.gov)" attrs=ALL
>>> [10/Nov/2015:10:12:03 -0500] conn=Internal op=-1 RESULT err=0 tag=48
>>> nentries=0 etime=0
>>> [10/Nov/2015:10:12:03 -0500] conn=5 op=2 RESULT err=49 tag=97 
>>> nentries=0
>>> etime=0
>>> [10/Nov/2015:10:12:03 -0500] conn=5 op=3 UNBIND
>>> [10/Nov/2015:10:12:03 -0500] conn=5 op=3 fd=64 closed - U1
>>>
>>> <snip>
>>>
>>> This is the SASL bind.  It thinks the principal in the Kerberos 
>>> credential is "ldap/comipa01.itmodev.gov", and the SASL map tells the 
>>> code to look for something with uid=ldap/comipa01.itmodev.gov under 
>>> dc=itmodev,dc=gov.  However, this entry is not found: RESULT err=0
>>> tag=48 nentries=0.  nentries=0 means no entries matched the search 
>>> criteria.
>>>
>>> You can do the search yourself with ldapsearch:
>>>
>>> ldapsearch -x -h 172.16.100.161 -D "cn=directory manager" -W -b 
>>> "dc=itmodev,dc=gov" '(uid=ldap/comipa01.itmodev.gov)'
>>>
>>> If you want to find out if there is some other ldap principal, do a 
>>> search like this:
>>>
>>> ldapsearch -x -h 172.16.100.161 -D "cn=directory manager" -W -b 
>>> "dc=itmodev,dc=gov" '(uid=ldap/*.gov)' uid
>>>
>>>>> Ran into an error trying to set that
>>>>>
>>>>> # ldapmodify -a -D "cn=directory manager" -W Enter LDAP Password:
>>>>> dn: cn=config
>>>>> changetype: modify
>>>>> replace: nsslapd-acesslog-level
>>>>> : 260
>>>>>
>>>>> modifying entry "cn=config"
>>>>> ldap_modify: Server is unwilling to perform (53)
>>>>>            additional info: Unknown attribute 
>>>>> nsslapd-acesslog-level will be ignored
>>>>>
>>>>> [root@comipa02 ~]# ldapmodify -a -D "cn=config" -W Enter LDAP
>>>>> Password:
>>>>> ldap_bind: Inappropriate authentication (48)
>>>>>
>>>>> -----Original Message-----
>>>>> From: Ludwig Krispenz [mailto:lkris...@redhat.com]
>>>>> Sent: Tuesday, November 10, 2015 9:48 AM
>>>>> To: Gronde, Christopher (Contractor) 
>>>>> <christopher.gro...@fincen.gov>
>>>>> Cc: freeipa-users@redhat.com
>>>>> Subject: Re: [Freeipa-users] krb5kdc will not start (kerberos 
>>>>> authentication error)
>>>>>
>>>>>
>>>>> On 11/10/2015 03:32 PM, Gronde, Christopher (Contractor) wrote:
>>>>>> How do I change that log setting? Is that done in LDAP?  Using 
>>>>>> ldapmodify?
>>>>> yes,
>>>>> ldapmodify ...
>>>>> dn: cn=config
>>>>> changetype: modify
>>>>> replace: nsslapd-acesslog-level
>>>>> nsslapd-acesslog-level: 260
>>>>>> -----Original Message-----
>>>>>> From: freeipa-users-boun...@redhat.com 
>>>>>> [mailto:freeipa-users-boun...@redhat.com] On Behalf Of Ludwig 
>>>>>> Krispenz
>>>>>> Sent: Tuesday, November 10, 2015 9:03 AM
>>>>>> To: freeipa-users@redhat.com
>>>>>> Subject: Re: [Freeipa-users] krb5kdc will not start (kerberos 
>>>>>> authentication error)
>>>>>>
>>>>>>
>>>>>> On 11/10/2015 02:40 PM, Alexander Bokovoy wrote:
>>>>>>> On Tue, 10 Nov 2015, Gronde, Christopher (Contractor) wrote:
>>>>>>>> Where can I verify or change the credentials it is trying to use?
>>>>>>>> Is it my LDAP password?
>>>>>>> No, according to your logs, it is your LDAP master trying to 
>>>>>>> replicate (push changes) to your LDAP replica:
>>>>>>>>> [09/Nov/2015:15:02:01 -0500] conn=2 fd=64 slot=64 connection 
>>>>>>>>> from <MASTER_IP> to <REPLICA_IP>
>>>>>>>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=0 BIND dn="" method=sasl
>>>>>>>>> version=3 mech=GSSAPI
>>>>>> err=49 could also be a result if the entry which is mapped from 
>>>>>> the principal is not found in the directory. A bit more info could 
>>>>>> be gained by enabling logging of internal searches.
>>>>>> Set nsslapd-acesslog-level: 260
>>>>>>
>>>>>> and then look what internal searches are done during the gssapi 
>>>>>> authentication
>>>>>>> If that is true, it would be ldap/<master> Kerberos principal 
>>>>>>> talking to ldap/<replica> Kerberos principal. If that fails, it 
>>>>>>> means master and replica KDCs have different understanding of 
>>>>>>> both ldap/<master> and ldap/<replica> keys which most likely 
>>>>>>> means keys were rotated on master and weren't propagated to replica.
>>>>>>>
>>>>>>> How to solve it? One possibility is to set master's hostname as 
>>>>>>> KDC address in krb5.conf on replica, forcing LDAP server on 
>>>>>>> replica to use master's KDC. I'm absolutely not sure this will 
>>>>>>> actually work but at least it allows to see if we are indeed 
>>>>>>> dealing with inconsistent state of service principals' keys.
>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Alexander Bokovoy [mailto:aboko...@redhat.com]
>>>>>>>> Sent: Tuesday, November 10, 2015 8:18 AM
>>>>>>>> To: Gronde, Christopher (Contractor) 
>>>>>>>> <christopher.gro...@fincen.gov>
>>>>>>>> Cc: Rob Crittenden <rcrit...@redhat.com>; 
>>>>>>>> freeipa-users@redhat.com
>>>>>>>> Subject: Re: [Freeipa-users] krb5kdc will not start (kerberos 
>>>>>>>> authentication error)
>>>>>>>>
>>>>>>>> On Tue, 10 Nov 2015, Gronde, Christopher (Contractor) wrote:
>>>>>>>>> When I tried to start the service again I got no response from 
>>>>>>>>> tail of the log, but this is a repeating entry I see in the 
>>>>>>>>> access log
>>>>>>>>>
>>>>>>>>> [09/Nov/2015:15:01:04 -0500] conn=1 fd=64 slot=64 connection 
>>>>>>>>> from
>>>>>>>>> 127.0.0.1 to 127.0.0.1
>>>>>>>>> [09/Nov/2015:15:01:04 -0500] conn=1 op=-1 fd=64 closed - B1
>>>>>>>>> [09/Nov/2015:15:02:01 -0500] conn=2 fd=64 slot=64 connection 
>>>>>>>>> from <MASTER_IP> to <REPLICA_IP>
>>>>>>>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=0 BIND dn="" method=sasl
>>>>>>>>> version=3 mech=GSSAPI
>>>>>>>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=0 RESULT err=14 tag=97
>>>>>>>>> nentries=0 etime=0, SASL bind in progress
>>>>>>>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=1 BIND dn="" method=sasl
>>>>>>>>> version=3 mech=GSSAPI
>>>>>>>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=1 RESULT err=14 tag=97
>>>>>>>>> nentries=0 etime=0, SASL bind in progress
>>>>>>>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=2 BIND dn="" method=sasl
>>>>>>>>> version=3 mech=GSSAPI
>>>>>>>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=2 RESULT err=49 tag=97
>>>>>>>>> nentries=0 etime=0
>>>>>>>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=3 UNBIND
>>>>>>>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=3 fd=64 closed - U1
>>>>>>>>>
>>>>>>>>> Does anyone know what err=14 or err=49 are?
>>>>>>>> err=14 means SASL bind in progress -- i.e. multi-round 
>>>>>>>> processing is ongoing. This is normal for SASL GSSAPI.
>>>>>>>>
>>>>>>>> err=49 is wrong password or username, i.e. credentials were 
>>>>>>>> incorrect.
>>>>>>>> It may also mean that LDAP server side was unable to process 
>>>>>>>> Kerberos negotiation due to not having a current Kerberos ticket 
>>>>>>>> for own service
>>>>>>>> (LDAP) and trying to request it from the Kerberos KDC but 
>>>>>>>> Kerberos KDC is down.
>>>>>>>>
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: Rob Crittenden [mailto:rcrit...@redhat.com]
>>>>>>>>> Sent: Monday, November 09, 2015 3:26 PM
>>>>>>>>> To: Gronde, Christopher (Contractor) 
>>>>>>>>> <christopher.gro...@fincen.gov>; Alexander Bokovoy 
>>>>>>>>> <aboko...@redhat.com>
>>>>>>>>> Cc: freeipa-users@redhat.com
>>>>>>>>> Subject: Re: [Freeipa-users] krb5kdc will not start (kerberos 
>>>>>>>>> authentication error)
>>>>>>>>>
>>>>>>>>> Gronde, Christopher (Contractor) wrote:
>>>>>>>>>> Nothing bad came back and there is definitely data in the tree.
>>>>>>>>> Ok, I guess I'd try to start the kdc again and then watch the 
>>>>>>>>> 389-ds access log (buffered) to:
>>>>>>>>>
>>>>>>>>> 1. See if it is binding at all
>>>>>>>>> 2. See what the search is and what, if any, results were 
>>>>>>>>> returned
>>>>>>>>>
>>>>>>>>> This would be in /var/log/dirsrv/slapd-YOUR_REALM/access
>>>>>>>>>
>>>>>>>>> rob
>>>>>>>>>
>>>>>>>>>> -----Original Message-----
>>>>>>>>>> From: Rob Crittenden [mailto:rcrit...@redhat.com]
>>>>>>>>>> Sent: Monday, November 09, 2015 11:46 AM
>>>>>>>>>> To: Gronde, Christopher (Contractor) 
>>>>>>>>>> <christopher.gro...@fincen.gov>; Alexander Bokovoy 
>>>>>>>>>> <aboko...@redhat.com>
>>>>>>>>>> Cc: freeipa-users@redhat.com
>>>>>>>>>> Subject: Re: [Freeipa-users] krb5kdc will not start (kerberos 
>>>>>>>>>> authentication error)
>>>>>>>>>>
>>>>>>>>>> Gronde, Christopher (Contractor) wrote:
>>>>>>>>>>> I restarted dirsrv and attempted to start krb5kdc and this is 
>>>>>>>>>>> what the error log shows
>>>>>>>>>>>
>>>>>>>>>>> # tail /var/log/dirsrv/slapd-ITMODEV-GOV/errors
>>>>>>>>>>> [09/Nov/2015:11:01:02 -0500] - WARNING: userRoot: entry cache 
>>>>>>>>>>> size 10485760B is less than db size 28016640B; We recommend 
>>>>>>>>>>> to increase the entry cache size nsslapd-cachememsize.
>>>>>>>>>>> [09/Nov/2015:11:01:02 -0500] - slapd started. Listening on 
>>>>>>>>>>> All Interfaces port 389 for LDAP requests
>>>>>>>>>>> [09/Nov/2015:11:06:04 -0500] - slapd shutting down - 
>>>>>>>>>>> signaling operation threads
>>>>>>>>>>> [09/Nov/2015:11:06:04 -0500] - slapd shutting down - closing 
>>>>>>>>>>> down internal subsystems and plugins
>>>>>>>>>>> [09/Nov/2015:11:06:04 -0500] - Waiting for 4 database threads 
>>>>>>>>>>> to stop
>>>>>>>>>>> [09/Nov/2015:11:06:04 -0500] - All database threads now 
>>>>>>>>>>> stopped
>>>>>>>>>>> [09/Nov/2015:11:06:04 -0500] - slapd stopped.
>>>>>>>>>>> [09/Nov/2015:11:14:20 -0500] - 389-Directory/1.2.11.15
>>>>>>>>>>> B2015.247.1737 starting up
>>>>>>>>>>> [09/Nov/2015:11:14:20 -0500] - WARNING: userRoot: entry cache 
>>>>>>>>>>> size 10485760B is less than db size 28016640B; We recommend 
>>>>>>>>>>> to increase the entry cache size nsslapd-cachememsize.
>>>>>>>>>>> [09/Nov/2015:11:14:20 -0500] - slapd started. Listening on 
>>>>>>>>>>> All Interfaces port 389 for LDAP requests
>>>>>>>>>> Ok, that's good.
>>>>>>>>>>
>>>>>>>>>> I'd do something like this to see what is in the db 
>>>>>>>>>> (substitute example.com with your domain):
>>>>>>>>>>
>>>>>>>>>> $ ldapsearch -x -D 'cn=Directory Manager' -W -s one -b 
>>>>>>>>>> cn=kerberos,dc=example,dc=com
>>>>>>>>>>
>>>>>>>>>> (don't post the output as it would include the kerberos master 
>>>>>>>>>> key).
>>>>>>>>>>
>>>>>>>>>> If that returns nothing that's bad.
>>>>>>>>>>
>>>>>>>>>> If it succeeds I'd broaden the search base a bit to see what 
>>>>>>>>>> data you do
>>>>>>>>>> have:
>>>>>>>>>>
>>>>>>>>>> $ ldapsearch -x -D 'cn=Directory Manager' -W -b 
>>>>>>>>>> cn=groups,cn=accounts,dc=example,dc=com
>>>>>>>>>>
>>>>>>>>>> I picked groups because usually groups << users in numbers. 
>>>>>>>>>> This is just to see if you have data in the tree.
>>>>>>>>>>
>>>>>>>>>> Let us know if either or both turns up nothing.
>>>>>>>>>>
>>>>>>>>>> rob
>>>>>>>>>>
>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>> From: Alexander Bokovoy [mailto:aboko...@redhat.com]
>>>>>>>>>>> Sent: Monday, November 09, 2015 10:51 AM
>>>>>>>>>>> To: Gronde, Christopher (Contractor) 
>>>>>>>>>>> <christopher.gro...@fincen.gov>
>>>>>>>>>>> Cc: freeipa-users@redhat.com
>>>>>>>>>>> Subject: Re: [Freeipa-users] krb5kdc will not start (kerberos 
>>>>>>>>>>> authentication error)
>>>>>>>>>>>
>>>>>>>>>>> On Mon, 09 Nov 2015, Gronde, Christopher (Contractor) wrote:
>>>>>>>>>>>> Hello all!
>>>>>>>>>>>>
>>>>>>>>>>>> On my replica IPA server after fixing a cert issue that had 
>>>>>>>>>>>> been going on for sometime, I have all my certs figured out 
>>>>>>>>>>>> but the krb5kdc service will not start.
>>>>>>>>>>>>
>>>>>>>>>>>> # service krb5kdc start
>>>>>>>>>>>> Starting Kerberos 5 KDC: krb5kdc: cannot initialize realm
>>>>>>>>>>>> ITMODEV.GOV - see log file for details                  
>>>>>>>>>>>> [FAILED]
>>>>>>>>>>>>
>>>>>>>>>>>> # cat /var/log/krb5kdc.log
>>>>>>>>>>>> krb5kdc: Server error - while fetching master key K/M for 
>>>>>>>>>>>> realm ITMODEV.GOV
>>>>>>>>>>>> krb5kdc: Server error - while fetching master key K/M for 
>>>>>>>>>>>> realm ITMODEV.GOV
>>>>>>>>>>>> krb5kdc: Server error - while fetching master key K/M for 
>>>>>>>>>>>> realm ITMODEV.GOV
>>>>>>>>>>>>
>>>>>>>>>>>> I found this article online:
>>>>>>>>>>>> http://research.imb.uq.edu.au/~l.rathbone/ldap/kerberos.shtm
>>>>>>>>>>>> l
>>>>>>>>>>>>
>>>>>>>>>>>> Which stated it might be because The slave KDC does not have 
>>>>>>>>>>>> a stash file (.k5.EXAMPLE.COM). You need to create one. 
>>>>>>>>>>>> Tried the command
>>>>>>>>>>>> listed:
>>>>>>>>>>>>
>>>>>>>>>>>> # kdb5_util stash
>>>>>>>>>>>> kdb5_util: Server error while retrieving master entry
>>>>>>>>>>>>
>>>>>>>>>>>> No further information found on the proceeding error above 
>>>>>>>>>>>> for the kdb5_util command.
>>>>>>>>>>>>
>>>>>>>>>>>> Any thoughts?
>>>>>>>>>>> First: don't use instructions which are not related to IPA, 
>>>>>>>>>>> please.
>>>>>>>>>>>
>>>>>>>>>>> FreeIPA has its own LDAP driver for KDC and instructions for 
>>>>>>>>>>> anything else do not apply here at all.
>>>>>>>>>>>
>>>>>>>>>>> If you see 'Server error - while fetching master key ..' it 
>>>>>>>>>>> means KDC LDAP driver was unable to contact LDAP server. Does 
>>>>>>>>>>> LDAP server work on the replica? What is in its error log 
>>>>>>>>>>> (/var/log/dirsrv/slapd-ITMODEV-GOV/errors)?
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> / Alexander Bokovoy
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>> --
>>>>>>>> / Alexander Bokovoy
>>>>>>>>
>>>>>> --
>>>>>> Manage your subscription for the Freeipa-users mailing list:
>>>>>> https://www.redhat.com/mailman/listinfo/freeipa-users
>>>>>> Go to http://freeipa.org for more info on the project
>>>>>>
>>>>
>>> --
>>> Manage your subscription for the Freeipa-users mailing list:
>>> https://www.redhat.com/mailman/listinfo/freeipa-users
>>> Go to http://freeipa.org for more info on the project
>>>
>>
> 
> --
> Manage your subscription for the Freeipa-users mailing list:
> https://www.redhat.com/mailman/listinfo/freeipa-users
> Go to http://freeipa.org for more info on the project
> 
> 

-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Reply via email to