On 22.6.2016 23:09, Sean Hogan wrote:
> SLAPD showing
>
> 22/Jun/2016:17:01:59 -0400] slapi_ldap_bind - Error: could not perform
> interactive bind for id [] mech [GSSAPI]: error 49 (Invalid credentials)
> [22/Jun/2016:17:06:59 -0400] slapd_ldap_sasl_interactive_bind - Error:
> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error 49
> (Invalid credentials) (SASL(-13): authentication failure: GSSAPI Failure:
> gss_accept_sec_context) errno 0 (Success)
>
>
> where would these creds be and what ID? I am using SASL so I assume it to
> be sasl_user DNS/FirstMaster.watson.local or something like that?
These are in /etc/dirsrv/ds.keytab.
I would start with
# klist -kt /etc/dirsrv/ds.keytab
and try to proceed with kinit etc. (very similarly to the bind-dyndb-ldap
how-to).
I hope it helps.
Petr^2 Spacek
> From: Sean Hogan/Durham/IBM@IBMUS
> To: Petr Spacek <pspa...@redhat.com>
> Cc: freeipa-users@redhat.com
> Date: 06/22/2016 08:36 AM
> Subject: Re: [Freeipa-users] IPA 3.0.47 to 3.0.50 Upgrade problem
> Sent by: freeipa-users-boun...@redhat.com
>
>
>
> Hi Peter...
>
> Yes..... this has me doing loops in my head to /dev/null
>
> You are correct I could not complete the BIND steps... I did them yesterday
> but did not post results as I wanted to stop bugging you all :)
> The initial credential section of that I could not complete nor can I get
> an keytab without it and I don't think I have an issue with cert versions
> (used the SASL section). The upgrade log from 3.47 to 3.50 on this one
> server did show an error with named though.
>
> I had the box powered down again last night after testing the BIND
> procedures... and its been up since then. Which makes we really not sure
> what is going on(DNS DOS from internal maybe? I get a lot of outside
> requests showing network unreachable and I don't forward to a outside DNS).
> If it was a password/cert/cipher/file perm issue then I don't see how it
> can work at all after a reboot.
>
> I am thinking it needs a rebuild.. I have not done this on a First Master
> IPA is there anything I need to be take into consider with it being first
> master? Right now I have 8 IPAs all DNS, NTP and CAs on differ vlans but
> the first master is the fail back IPA(on the only vlan that can talk to the
> others) in case there local vlan IPA dies. First Master is also the master
> CA in the realm where everything is enrolled to originally. We then mod
> everything to point to the vlan IPA with the Firstmaster as secondary with
> our vlan-specific scripts we run after ipa client install.
>
> With the box rebooted last night I am now getting normal functionality but
> it prob wont last long as indicated from the past...
>
> Working
> [bob@FirstMaster ~]# kinit admin
> Password for admin@DOMAIN.LOCAL:
> Warning: Your password will expire in 6 days on Tue Jun 28 14:55:52 2016
> [bob@FirstMaster ~]#
>
> I did post ldap logs in my first email though... will readd them to this
> and when it dies off again I will add more.
>
>
>> [20/Jun/2016:13:59:00 -0400] - Detected Disorderly Shutdown last time
>> Directory Server was running, recovering database.
>> [20/Jun/2016:13:59:01 -0400] schema-compat-plugin - warning: no entries
> set
>> up under cn=computers, cn=compat,dc=domain,dc=local
>> [20/Jun/2016:13:59:48 -0400] NSMMReplicationPlugin - ruv_compare_ruv: RUV
>> [database RUV] does not contain element [{replica 7} 55ca26a0000900070000
>> 5688d8e6001000070000] which is present in RUV [changelog max RUV]
>> [20/Jun/2016:13:59:48 -0400] NSMMReplicationPlugin -
>> replica_check_for_data_reload: Warning: for replica dc=domain,dc=local
>> there were some differences between the changelog max RUV and the
> database
>> RUV. If there are obsolete elements in the database RUV, you should
> remove
>> them using the CLEANALLRUV task. If they are not obsolete, you should
> check
>> their status to see why there are no changes from those servers in the
>> changelog.
>> [20/Jun/2016:13:59:48 -0400] set_krb5_creds - Could not get initial
>> credentials for principal [ldap/server1.domain.local@DOMAIN.LOCAL] in
>> keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC
>> for requested realm)
>> [20/Jun/2016:13:59:48 -0400] set_krb5_creds - Could not get initial
>> credentials for principal [ldap/server1.domain.local@DOMAIN.LOCAL] in
>> keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC
>> for requested realm)
>> [20/Jun/2016:13:59:48 -0400] set_krb5_creds - Could not get initial
>> credentials for principal [ldap/server1.domain.local@DOMAIN.LOCAL] in
>> keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC
>> for requested realm)
>> [20/Jun/2016:13:59:48 -0400] slapd_ldap_sasl_interactive_bind - Error:
>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
>> failure. Minor code may provide more information (Credentials cache file
>> '/tmp/krb5cc_495' not found)) errno 2 (No such file or directory)
>> [20/Jun/2016:13:59:48 -0400] slapi_ldap_bind - Error: could not perform
>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
>> [20/Jun/2016:13:59:48 -0400] slapd_ldap_sasl_interactive_bind - Error:
>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
>> failure. Minor code may provide more information (Credentials cache file
>> '/tmp/krb5cc_495' not found)) errno 2 (No such file or directory)
>> [20/Jun/2016:13:59:48 -0400] slapi_ldap_bind - Error: could not perform
>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
>> [20/Jun/2016:13:59:48 -0400] NSMMReplicationPlugin -
>> agmt="cn=meToserver9.domain.local" (server9:389): Replication bind with
>> GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic
> failure:
>> GSSAPI Error: Unspecified GSS failure. Minor code may provide more
>> information (Credentials cache file '/tmp/krb5cc_495' not found))
>> [20/Jun/2016:13:59:48 -0400] set_krb5_creds - Could not get initial
>> credentials for principal [ldap/server1.domain.local@DOMAIN.LOCAL] in
>> keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC
>> for requested realm)
>> [20/Jun/2016:13:59:48 -0400] NSMMReplicationPlugin -
>> agmt="cn=meToserver8.domain.local" (server8:389): Replication bind with
>> GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic
> failure:
>> GSSAPI Error: Unspecified GSS failure. Minor code may provide more
>> information (Credentials cache file '/tmp/krb5cc_495' not found))
>> [20/Jun/2016:13:59:48 -0400] slapd_ldap_sasl_interactive_bind - Error:
>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
>> failure. Minor code may provide more information (Credentials cache file
>> '/tmp/krb5cc_495' not found)) errno 2 (No such file or directory)
>> [20/Jun/2016:13:59:48 -0400] slapi_ldap_bind - Error: could not perform
>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
>> [20/Jun/2016:13:59:48 -0400] NSMMReplicationPlugin -
>> agmt="cn=meTobldvxl0011.domain.local" (1server:389): Replication bind
> with
>> GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic
> failure:
>> GSSAPI Error: Unspecified GSS failure. Minor code may provide more
>> information (Credentials cache file '/tmp/krb5cc_495' not found))
>> [20/Jun/2016:13:59:48 -0400] slapd_ldap_sasl_interactive_bind - Error:
>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
>> failure. Minor code may provide more information (Credentials cache file
>> '/tmp/krb5cc_495' not found)) errno 2 (No such file or directory)
>> [20/Jun/2016:13:59:48 -0400] slapi_ldap_bind - Error: could not perform
>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
>> [20/Jun/2016:13:59:48 -0400] NSMMReplicationPlugin -
>> agmt="cn=meToserver3.domain.local" (server3:389): Replication bind with
>> GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic
> failure:
>> GSSAPI Error: Unspecified GSS failure. Minor code may provide more
>> information (Credentials cache file '/tmp/krb5cc_495' not found))
>> [20/Jun/2016:13:59:48 -0400] set_krb5_creds - Could not get initial
>> credentials for principal [ldap/server1.domain.local@DOMAIN.LOCAL] in
>> keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC
>> for requested realm)
>> [20/Jun/2016:13:59:48 -0400] set_krb5_creds - Could not get initial
>> credentials for principal [ldap/server1.domain.local@DOMAIN.LOCAL] in
>> keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC
>> for requested realm)
>> [20/Jun/2016:13:59:48 -0400] slapd_ldap_sasl_interactive_bind - Error:
>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
>> failure. Minor code may provide more information (Credentials cache file
>> '/tmp/krb5cc_495' not found)) errno 2 (No such file or directory)
>> [20/Jun/2016:13:59:48 -0400] slapi_ldap_bind - Error: could not perform
>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
>> [20/Jun/2016:13:59:48 -0400] NSMMReplicationPlugin -
>> agmt="cn=meToserver7.domain.local" (server7:389): Replication bind with
>> GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic
> failure:
>> GSSAPI Error: Unspecified GSS failure. Minor code may provide more
>> information (Credentials cache file '/tmp/krb5cc_495' not found))
>> [20/Jun/2016:13:59:48 -0400] - slapd started. Listening on All Interfaces
>> port 389 for LDAP requests
>> [20/Jun/2016:13:59:48 -0400] - Listening on All Interfaces port 636 for
>> LDAPS requests
>> [20/Jun/2016:13:59:48 -0400] - Listening
>> on /var/run/slapd-DOMAIN-LOCAL.socket for LDAPI requests
>> [20/Jun/2016:13:59:48 -0400] slapd_ldap_sasl_interactive_bind - Error:
>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
>> failure. Minor code may provide more information (Credentials cache file
>> '/tmp/krb5cc_495' not found)) errno 2 (No such file or directory)
>> [20/Jun/2016:13:59:48 -0400] slapi_ldap_bind - Error: could not perform
>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
>> [20/Jun/2016:13:59:48 -0400] NSMMReplicationPlugin -
>> agmt="cn=meToserver4.domain.local" (server4:389): Replication bind with
>> GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic
> failure:
>> GSSAPI Error: Unspecified GSS failure. Minor code may provide more
>> information (Credentials cache file '/tmp/krb5cc_495' not found))
>> [20/Jun/2016:13:59:48 -0400] set_krb5_creds - Could not get initial
>> credentials for principal [ldap/server1.domain.local@DOMAIN.LOCAL] in
>> keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC
>> for requested realm)
>> [20/Jun/2016:13:59:48 -0400] slapd_ldap_sasl_interactive_bind - Error:
>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
>> failure. Minor code may provide more information (Credentials cache file
>> '/tmp/krb5cc_495' not found)) errno 0 (Success)
>> [20/Jun/2016:13:59:48 -0400] slapi_ldap_bind - Error: could not perform
>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
>> [20/Jun/2016:13:59:48 -0400] NSMMReplicationPlugin -
>> agmt="cn=meToserver2.domain.local" (server2:389): Replication bind with
>> GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic
> failure:
>> GSSAPI Error: Unspecified GSS failure. Minor code may provide more
>> information (Credentials cache file '/tmp/krb5cc_495' not found))
>> [20/Jun/2016:13:59:51 -0400] NSMMReplicationPlugin -
>> agmt="cn=meToserver8.domain.local" (server8:389): Replication bind with
>> GSSAPI auth resumed
>> [20/Jun/2016:13:59:51 -0400] slapd_ldap_sasl_interactive_bind - Error:
>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error 49
>> (Invalid credentials) (SASL(-13): authentication failure: GSSAPI Failure:
>> gss_accept_sec_context) errno 0 (Success)
>> [20/Jun/2016:13:59:51 -0400] slapi_ldap_bind - Error: could not perform
>> interactive bind for id [] mech [GSSAPI]: error 49 (Invalid credentials)
>> [20/Jun/2016:13:59:51 -0400] NSMMReplicationPlugin -
>> agmt="cn=meToserver3.domain.local" (server3:389): Replication bind with
>> GSSAPI auth failed: LDAP error 49 (Invalid credentials) (SASL(-13):
>> authentication failure: GSSAPI Failure: gss_accept_sec_context)
>> [20/Jun/2016:13:59:51 -0400] slapd_ldap_sasl_interactive_bind - Error:
>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
>> failure. Minor code may provide more information (No credentials cache
>> found)) errno 2 (No such file or directory)
>> [20/Jun/2016:13:59:51 -0400] slapi_ldap_bind - Error: could not perform
>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
>> [20/Jun/2016:13:59:51 -0400] slapd_ldap_sasl_interactive_bind - Error:
>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
>> failure. Minor code may provide more information (No credentials cache
>> found)) errno 2 (No such file or directory)
>> [20/Jun/2016:13:59:51 -0400] slapi_ldap_bind - Error: could not perform
>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
>> [20/Jun/2016:13:59:51 -0400] slapd_ldap_sasl_interactive_bind - Error:
>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
>> failure. Minor code may provide more information (No credentials cache
>> found)) errno 2 (No such file or directory)
>> [20/Jun/2016:13:59:51 -0400] slapi_ldap_bind - Error: could not perform
>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
>> [20/Jun/2016:13:59:51 -0400] slapd_ldap_sasl_interactive_bind - Error:
>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
>> failure. Minor code may provide more information (No credentials cache
>> found)) errno 2 (No such file or directory)
>> [20/Jun/2016:13:59:51 -0400] slapi_ldap_bind - Error: could not perform
>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
>> [20/Jun/2016:13:59:51 -0400] slapd_ldap_sasl_interactive_bind - Error:
>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
>> failure. Minor code may provide more information (No credentials cache
>> found)) errno 2 (No such file or directory)
>> [20/Jun/2016:13:59:51 -0400] slapi_ldap_bind - Error: could not perform
>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
>> [20/Jun/2016:13:59:57 -0400] slapd_ldap_sasl_interactive_bind - Error:
>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error 49
>> (Invalid credentials) (SASL(-13): authentication failure: GSSAPI Failure:
>> gss_accept_sec_context) errno 0 (Success)
>> [20/Jun/2016:13:59:57 -0400] slapi_ldap_bind - Error: could not perform
>> interactive bind for id [] mech [GSSAPI]: error 49 (Invalid credentials)
>> [20/Jun/2016:13:59:57 -0400] NSMMReplicationPlugin -
>> agmt="cn=meToserver2.domain.local" (server2:389): Replication bind with
>> GSSAPI auth resumed
>
>
>
> Sean Hogan
>
>
>
>
>
> Inactive hide details for Petr Spacek ---06/21/2016 10:20:43 PM---On
> 22.6.2016 02:56, Sean Hogan wrote: > More infoPetr Spacek ---06/21/2016
> 10:20:43 PM---On 22.6.2016 02:56, Sean Hogan wrote: > More info
>
> From: Petr Spacek <pspa...@redhat.com>
> To: freeipa-users@redhat.com
> Date: 06/21/2016 10:20 PM
> Subject: Re: [Freeipa-users] IPA 3.0.47 to 3.0.50 Upgrade problem
> Sent by: freeipa-users-boun...@redhat.com
>
>
>
> On 22.6.2016 02:56, Sean Hogan wrote:
>> More info
>>
>>
>> Krb5 log is showing:
>> Jun 21 20:42:47 Firstmaster.domain.local krb5kdc[2141](info): AS_REQ (4
>> etypes {18 17 16 23}) 10.x.x.x: LOOKING_UP_CLIENT: admin@domain.LOCAL for
>> krbtgt/DOMAIN.LOCAL@DOMAIN.LOCAL, Server error
>
>
> Hello,
>
> this is really fishy. I would bet that there is a problem with LDAP server
> and
> DNS errors are just consequence of it.
>
> I suspect that you will not be able to finish steps mentioned in
> https://fedorahosted.org/bind-dyndb-ldap/wiki/BIND9/NamedCannotStart#a3.FailedtoinitcredentialsorFailedtogetinitialcredentialsDecryptintegritycheckfailedorClientscredentialshavebeenrevoked
>
>
> If it is the case I would turn your attention to krb5kdc.log and LDAP
> server
> logs in /var/log/dirsrv/*
>
> There must be something wrong with the LDAP server.
>
> Petr^2 Spacek
>
>
>>
>> [bob@Firstmaster etc]# kinit -v admin
>> kinit: Credentials cache file '/tmp/krb5cc_0' not found while validating
>> credentials
>>
>>
>>
>>
>>
>>
>> Sean Hogan
>>
>>
>>
>>
>>
>>
>> From: Sean Hogan/Durham/IBM
>> To: freeipa-users <freeipa-users@redhat.com>
>> Date: 06/21/2016 12:02 PM
>> Subject: Re: [Freeipa-users] IPA 3.0.47 to 3.0.50 Upgrade problem
>>
>>
>> Has anyone seen these before?
>>
>>
>>
>> First Master IPA DNS logs show: Looks like the host names are getting
> the
>> domain twice domain.local.domain.local
>>
>>
>> client 10.x.x.x#58094: query failed (SERVFAIL) for
>> server1.domain.local.domain.local/IN/AAAA at query.c:6569
>> timeout in ldap_pool_getconnection(): try to raise 'connections'
> parameter;
>> potential deadlock?
>> client 10.x.x.x#44147: query failed (SERVFAIL) for
>> x.x.x.10.in-addr.arpa/IN/PTR at query.c:6569
>> timeout in ldap_pool_getconnection(): try to raise 'connections'
> parameter;
>> potential deadlock?
>> client 10.x.x.x#56466: query failed (SERVFAIL) for
>> x.x.x.10.in-addr.arpa/IN/PTR at query.c:6569
>> timeout in ldap_pool_getconnection(): try to raise 'connections'
> parameter;
>> potential deadlock?
>> client 10.x.x.x53367: query failed (SERVFAIL) for
>> server2.domain.local.domain.local/IN/A at query.c:6569
>> timeout in ldap_pool_getconnection(): try to raise 'connections'
> parameter;
>> potential deadlock?
>> client 10.x.x.x#53367: query failed (SERVFAIL) for
>> server2.domain.local.domain.local/IN/AAAA at query.c:6569
>>
>>
>>
>> So enrolls are failing at this point when tyring to enroll to a replica:
>>
>> [bob@server1 log]# ipa-client-install –enable-dns-updates
>> Discovery was successful!
>> Hostname: server1.watson.local
>> Realm: DOMAIN.LOCAL
>> DNS Domain: domain.local
>> IPA Server: ipareplica.domain.local
>> BaseDN: dc=domain,dc=local
>>
>> Continue to configure the system with these values? [no]: yes
>> User authorized to enroll computers: bob
>> Synchronizing time with KDC...
>> Password for bob@DOMAIN.LOCAL:
>> Successfully retrieved CA cert
>> Subject: CN=Certificate Authority,O=DOMAIN.LOCAL
>> Issuer: CN=Certificate Authority,O=DOMAIN.LOCAL
>> Valid From: Tue Jan 06 19:37:09 2015 UTC
>> Valid Until: Sat Jan 06 19:37:09 2035 UTC
>>
>> Enrolled in IPA realm DOMAIN.LOCAL
>> Attempting to get host TGT...
>> Created /etc/ipa/default.conf
>> New SSSD config will be created
>> Configured sudoers in /etc/nsswitch.conf
>> Configured /etc/sssd/sssd.conf
>> Configured /etc/krb5.conf for IPA realm DOMAIN.LOCAL
>> trying https://ipareplica.domain.local/ipa/xml
>> Cannot connect to the server due to Kerberos error: Kerberos error:
>> Kerberos error: ('Unspecified GSS failure. Minor code may provide more
>> information', 851968)/('KDC returned error string: PROCESS_TGS',
>> -1765328324)/. Trying with delegate=True
>> trying https://ipareplica.domain.local/ipa/xml
>> Second connect with delegate=True also failed: Kerberos error: Kerberos
>> error: ('Unspecified GSS failure. Minor code may provide more
>> information', 851968)/('KDC returned error string: PROCESS_TGS',
>> -1765328324)/
>> Cannot connect to the IPA server XML-RPC interface: Kerberos error:
>> Kerberos error: ('Unspecified GSS failure. Minor code may provide more
>> information', 851968)/('KDC returned error string: PROCESS_TGS',
>> -1765328324)/
>> Installation failed. Rolling back changes.
>> Unenrolling client from IPA server
>> Unenrolling host failed: Error obtaining initial credentials: Generic
> error
>> (see e-text).
>>
>> Removing Kerberos service principals from /etc/krb5.keytab
>> Disabling client Kerberos and LDAP configurations
>> Redundant SSSD configuration file /etc/sssd/sssd.conf was moved
>> to /etc/sssd/sssd.conf.deleted
>> Restoring client configuration files
>> nscd daemon is not installed, skip configuration
>> nslcd daemon is not installed, skip configuration
>> Client uninstall complete.
>>
>>
>> Sean Hogan
>>
>>
>>
>>
>>
>>
>>
>>
>> From: Sean Hogan/Durham/IBM
>> To: Sean Hogan/Durham/IBM@IBMUS
>> Cc: freeipa-users <freeipa-users@redhat.com>
>> Date: 06/20/2016 12:49 PM
>> Subject: Re: [Freeipa-users] IPA 3.0.47 to 3.0.50 Upgrade problem
>>
>>
>> Also seeing this in the upgrade log on the first master but not on the 7
>> ipas.
>>
>> ERROR Failed to restart named: Command '/sbin/service named restart '
>> returned non-zero exit status 7
>>
>>
>> which led me to
>>
>> https://bugzilla.redhat.com/show_bug.cgi?id=895298
>>
>>
>>
>>
>>
>> Sean Hogan
>>
>>
>>
>>
>>
>>
>>
>> From: Sean Hogan/Durham/IBM@IBMUS
>> To: freeipa-users <freeipa-users@redhat.com>
>> Date: 06/20/2016 11:46 AM
>> Subject: Re: [Freeipa-users] IPA 3.0.47 to 3.0.50 Upgrade problem
>> Sent by: freeipa-users-boun...@redhat.com
>>
>>
>>
>> Hi All..
>>
>> I thought we fixed this issue by rebooting the KVM host but it is showing
>> again. Our First Master IPA is being rebooted 2 -5 times a day now just
> to
>> keep it alive.
>>
>> What we are seeing:
>>
>> God@FirstMaster log]# kinit admin
>> kinit: Cannot contact any KDC for realm 'Domain.LOCAL' while getting
>> initial credentials
>>
>> DNS is not working as nslookup is failing to a replica.... think once we
>> lose DNS it all goes down hill which makes sense.
>>
>> [god@FirstMaster log]# ipactl stop -----> Just hangs forever.. no
> replies..
>> no error.. nothing
>>
>> I try service named stop and nothing happens
>>
>> I have the box hard shutdown from KVM console. Reboot it and it works for
> a
>> little while but eventually back to same behavior.
>>
>> At this point I can service named stop and it responds... ipactl status
> and
>> it responds.. but when if I try service named restart I get
>>
>> [god@FirstMaster log]# service named stop
>> Stopping named: ......
>>
>> [god@Firstmaster log]# service named start
>> Starting named: [FAILED]
>>
>> [god@FirstMaster log]# service named status
>> rndc: connect failed: 127.0.0.1#953: connection refused
>> named dead but pid file exists
>>
>> Rebooted box and it is hung on shutting down domain-local and never fully
>> shuts down.. have to get it hard shutdown again.
>> During an attempt to gracefully shut down we see this
>>
>> Shutting Down dirsrv:
>> PKI-IPA OK
>> DOMAIN-LOCAL FAILED
>> *** Error: 1 instance(s) unsuccessfully stopped FAILED
>>
>> Then it moves on to shut other things down and returns to dirsrv
>> Shutting Down dirsrv:
>> PKI-IPA....server already stopped FAILED {Makes sense.. it died earlier}
>> DOMAIN-LOCAL... {this sits here til we hard shutdown}
>>
>>
>>
>> bind-libs-9.8.2-0.47.rc1.el6.x86_64
>> bind-9.8.2-0.47.rc1.el6.x86_64
>> bind-utils-9.8.2-0.47.rc1.el6.x86_64
>>
>>
>> ipa-client-3.0.0-50.el6.1.x86_64
>> ipa-server-selinux-3.0.0-50.el6.1.x86_64
>> ipa-server-3.0.0-50.el6.1.x86_64
>> sssd-ipa-1.13.3-22.el6.x86_64
>>
>>
>> /var/log/dirsrv/slapd-DOMAIN-LOCAL
>> [20/Jun/2016:13:29:06 -0400] - 389-Directory/1.2.11.15 B2016.063.2110
>> starting up
>> [20/Jun/2016:13:29:06 -0400] schema-compat-plugin - warning: no entries
> set
>> up under cn=computers, cn=compat,dc=domain,dc=local
>> [20/Jun/2016:13:29:07 -0400] NSMMReplicationPlugin - ruv_compare_ruv: RUV
>> [database RUV] does not contain element [{replica 7} 55ca26a0000900070000
>> 5688d8e6001000070000] which is present in RUV [changelog max RUV]
>> [20/Jun/2016:13:29:07 -0400] NSMMReplicationPlugin -
>> replica_check_for_data_reload: Warning: for replica dc=domain,dc=local
>> there were some differences between the changelog max RUV and the
> database
>> RUV. If there are obsolete elements in the database RUV, you should
> remove
>> them using the CLEANALLRUV task. If they are not obsolete, you should
> check
>> their status to see why there are no changes from those servers in the
>> changelog.
>> [20/Jun/2016:13:29:07 -0400] set_krb5_creds - Could not get initial
>> credentials for principal [ldap/server1.domain.local@DOMAIN.LOCAL] in
>> keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC
>> for requested realm)
>> [20/Jun/2016:13:29:07 -0400] set_krb5_creds - Could not get initial
>> credentials for principal [ldap/server1.domain.local@DOMAIN.LOCAL] in
>> keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC
>> for requested realm)
>> [20/Jun/2016:13:29:07 -0400] set_krb5_creds - Could not get initial
>> credentials for principal [ldap/server1.domain.local@DOMAIN.LOCAL] in
>> keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC
>> for requested realm)
>> [20/Jun/2016:13:29:07 -0400] slapd_ldap_sasl_interactive_bind - Error:
>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
>> failure. Minor code may provide more information (Credentials cache file
>> '/tmp/krb5cc_495' not found)) errno 2 (No such file or directory)
>> [20/Jun/2016:13:29:07 -0400] slapi_ldap_bind - Error: could not perform
>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
>> [20/Jun/2016:13:29:07 -0400] NSMMReplicationPlugin -
>> agmt="cn=meToserver8.domain.local" (server8:389): Replication bind with
>> GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic
> failure:
>> GSSAPI Error: Unspecified GSS failure. Minor code may provide more
>> information (Credentials cache file '/tmp/krb5cc_495' not found))
>> [20/Jun/2016:13:29:07 -0400] set_krb5_creds - Could not get initial
>> credentials for principal [ldap/server1.domain.local@DOMAIN.LOCAL] in
>> keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC
>> for requested realm)
>> [20/Jun/2016:13:29:07 -0400] slapd_ldap_sasl_interactive_bind - Error:
>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
>> failure. Minor code may provide more information (Credentials cache file
>> '/tmp/krb5cc_495' not found)) errno 2 (No such file or directory)
>> [20/Jun/2016:13:29:07 -0400] slapi_ldap_bind - Error: could not perform
>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
>> [20/Jun/2016:13:29:07 -0400] NSMMReplicationPlugin -
>> agmt="cn=meToserver9.domain.local" (server9:389): Replication bind with
>> GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic
> failure:
>> GSSAPI Error: Unspecified GSS failure. Minor code may provide more
>> information (Credentials cache file '/tmp/krb5cc_495' not found))
>> [20/Jun/2016:13:29:07 -0400] set_krb5_creds - Could not get initial
>> credentials for principal [ldap/server1.domain.local@DOMAIN.LOCAL] in
>> keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC
>> for requested realm)
>> [20/Jun/2016:13:29:07 -0400] slapd_ldap_sasl_interactive_bind - Error:
>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
>> failure. Minor code may provide more information (Credentials cache file
>> '/tmp/krb5cc_495' not found)) errno 2 (No such file or directory)
>> [20/Jun/2016:13:29:07 -0400] slapi_ldap_bind - Error: could not perform
>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
>> [20/Jun/2016:13:29:07 -0400] NSMMReplicationPlugin -
>> agmt="cn=meToserver3.domain.local" (server3:389): Replication bind with
>> GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic
> failure:
>> GSSAPI Error: Unspecified GSS failure. Minor code may provide more
>> information (Credentials cache file '/tmp/krb5cc_495' not found))
>> [20/Jun/2016:13:29:07 -0400] set_krb5_creds - Could not get initial
>> credentials for principal [ldap/server1.domain.local@DOMAIN.LOCAL] in
>> keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC
>> for requested realm)
>> [20/Jun/2016:13:29:07 -0400] - slapd started. Listening on All Interfaces
>> port 389 for LDAP requests
>> [20/Jun/2016:13:29:07 -0400] - Listening on All Interfaces port 636 for
>> LDAPS requests
>> [20/Jun/2016:13:29:07 -0400] - Listening
>> on /var/run/slapd-DOMAIN-LOCAL.socket for LDAPI requests
>> [20/Jun/2016:13:29:07 -0400] set_krb5_creds - Could not get initial
>> credentials for principal [ldap/server1.domain.local@DOMAIN.LOCAL] in
>> keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC
>> for requested realm)
>> [20/Jun/2016:13:29:07 -0400] slapd_ldap_sasl_interactive_bind - Error:
>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
>> failure. Minor code may provide more information (Credentials cache file
>> '/tmp/krb5cc_495' not found)) errno 2 (No such file or directory)
>> [20/Jun/2016:13:29:07 -0400] slapi_ldap_bind - Error: could not perform
>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
>> [20/Jun/2016:13:29:07 -0400] NSMMReplicationPlugin -
>> agmt="cn=meToserver4.domain.local" (server4:389): Replication bind with
>> GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic
> failure:
>> GSSAPI Error: Unspecified GSS failure. Minor code may provide more
>> information (Credentials cache file '/tmp/krb5cc_495' not found))
>> [20/Jun/2016:13:29:07 -0400] slapd_ldap_sasl_interactive_bind - Error:
>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
>> failure. Minor code may provide more information (Credentials cache file
>> '/tmp/krb5cc_495' not found)) errno 2 (No such file or directory)
>> [20/Jun/2016:13:29:07 -0400] slapi_ldap_bind - Error: could not perform
>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
>> [20/Jun/2016:13:29:07 -0400] NSMMReplicationPlugin -
>> agmt="cn=meTo1server.domain.local" (1server:389): Replication bind with
>> GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic
> failure:
>> GSSAPI Error: Unspecified GSS failure. Minor code may provide more
>> information (Credentials cache file '/tmp/krb5cc_495' not found))
>> [20/Jun/2016:13:29:07 -0400] slapd_ldap_sasl_interactive_bind - Error:
>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
>> failure. Minor code may provide more information (Credentials cache file
>> '/tmp/krb5cc_495' not found)) errno 2 (No such file or directory)
>> [20/Jun/2016:13:29:07 -0400] slapi_ldap_bind - Error: could not perform
>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
>> [20/Jun/2016:13:29:07 -0400] NSMMReplicationPlugin -
>> agmt="cn=meToserver7.domain.local" (server7:389): Replication bind with
>> GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic
> failure:
>> GSSAPI Error: Unspecified GSS failure. Minor code may provide more
>> information (Credentials cache file '/tmp/krb5cc_495' not found))
>> [20/Jun/2016:13:29:07 -0400] slapd_ldap_sasl_interactive_bind - Error:
>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
>> failure. Minor code may provide more information (Credentials cache file
>> '/tmp/krb5cc_495' not found)) errno 0 (Success)
>> [20/Jun/2016:13:29:07 -0400] slapi_ldap_bind - Error: could not perform
>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
>> [20/Jun/2016:13:29:07 -0400] NSMMReplicationPlugin -
>> agmt="cn=meToserver2.domain.local" (server2:389): Replication bind with
>> GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic
> failure:
>> GSSAPI Error: Unspecified GSS failure. Minor code may provide more
>> information (Credentials cache file '/tmp/krb5cc_495' not found))
>> [20/Jun/2016:13:29:10 -0400] NSMMReplicationPlugin -
>
--
Petr Spacek @ Red Hat
--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project
