On 28.6.2016 20:21, Sean Hogan wrote: > Thanks Petr, > > Since the last recycle of the Host hosting the First Master it has been > stable for about a week now. Only thing I did was to spread out my > replication agreements. I had 8 replications hitting it but now have 4 > going to it and the other 4 to its backup replica with the first master and > the backup replica having an agreement. > > > Not sure that fixed it or not but it seems to be stable at this point and I > know the docs say no more than 4 replications agreements so maybe it was > the cause.
Generally more replication agreements mean more load on the server. Many replication agreements should not cause problems by itself if the server has sufficient performance. Petr^2 Spacek > Sean Hogan > > > > > > > > From: Petr Spacek <pspa...@redhat.com> > To: Sean Hogan/Durham/IBM@IBMUS > Cc: freeipa-users@redhat.com > Date: 06/28/2016 10:24 AM > Subject: Re: [Freeipa-users] IPA 3.0.47 to 3.0.50 Upgrade problem > > > > On 22.6.2016 23:09, Sean Hogan wrote: >> SLAPD showing >> >> 22/Jun/2016:17:01:59 -0400] slapi_ldap_bind - Error: could not perform >> interactive bind for id [] mech [GSSAPI]: error 49 (Invalid credentials) >> [22/Jun/2016:17:06:59 -0400] slapd_ldap_sasl_interactive_bind - Error: >> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error 49 >> (Invalid credentials) (SASL(-13): authentication failure: GSSAPI Failure: >> gss_accept_sec_context) errno 0 (Success) >> >> >> where would these creds be and what ID? I am using SASL so I assume it > to >> be sasl_user DNS/FirstMaster.watson.local or something like that? > > These are in /etc/dirsrv/ds.keytab. > > I would start with > # klist -kt /etc/dirsrv/ds.keytab > and try to proceed with kinit etc. (very similarly to the bind-dyndb-ldap > how-to). > > I hope it helps. > > Petr^2 Spacek > > >> From: Sean Hogan/Durham/IBM@IBMUS >> To: Petr Spacek <pspa...@redhat.com> >> Cc: freeipa-users@redhat.com >> Date: 06/22/2016 08:36 AM >> Subject: Re: [Freeipa-users] IPA 3.0.47 to 3.0.50 Upgrade >> problem >> Sent by: freeipa-users-boun...@redhat.com >> >> >> >> Hi Peter... >> >> Yes..... this has me doing loops in my head to /dev/null >> >> You are correct I could not complete the BIND steps... I did them > yesterday >> but did not post results as I wanted to stop bugging you all :) >> The initial credential section of that I could not complete nor can I get >> an keytab without it and I don't think I have an issue with cert versions >> (used the SASL section). The upgrade log from 3.47 to 3.50 on this one >> server did show an error with named though. >> >> I had the box powered down again last night after testing the BIND >> procedures... and its been up since then. Which makes we really not sure >> what is going on(DNS DOS from internal maybe? I get a lot of outside >> requests showing network unreachable and I don't forward to a outside > DNS). >> If it was a password/cert/cipher/file perm issue then I don't see how it >> can work at all after a reboot. >> >> I am thinking it needs a rebuild.. I have not done this on a First Master >> IPA is there anything I need to be take into consider with it being first >> master? Right now I have 8 IPAs all DNS, NTP and CAs on differ vlans but >> the first master is the fail back IPA(on the only vlan that can talk to > the >> others) in case there local vlan IPA dies. First Master is also the > master >> CA in the realm where everything is enrolled to originally. We then mod >> everything to point to the vlan IPA with the Firstmaster as secondary > with >> our vlan-specific scripts we run after ipa client install. >> >> With the box rebooted last night I am now getting normal functionality > but >> it prob wont last long as indicated from the past... >> >> Working >> [bob@FirstMaster ~]# kinit admin >> Password for admin@DOMAIN.LOCAL: >> Warning: Your password will expire in 6 days on Tue Jun 28 14:55:52 2016 >> [bob@FirstMaster ~]# >> >> I did post ldap logs in my first email though... will readd them to this >> and when it dies off again I will add more. >> >> >>> [20/Jun/2016:13:59:00 -0400] - Detected Disorderly Shutdown last time >>> Directory Server was running, recovering database. >>> [20/Jun/2016:13:59:01 -0400] schema-compat-plugin - warning: no entries >> set >>> up under cn=computers, cn=compat,dc=domain,dc=local >>> [20/Jun/2016:13:59:48 -0400] NSMMReplicationPlugin - ruv_compare_ruv: > RUV >>> [database RUV] does not contain element [{replica 7} > 55ca26a0000900070000 >>> 5688d8e6001000070000] which is present in RUV [changelog max RUV] >>> [20/Jun/2016:13:59:48 -0400] NSMMReplicationPlugin - >>> replica_check_for_data_reload: Warning: for replica dc=domain,dc=local >>> there were some differences between the changelog max RUV and the >> database >>> RUV. If there are obsolete elements in the database RUV, you should >> remove >>> them using the CLEANALLRUV task. If they are not obsolete, you should >> check >>> their status to see why there are no changes from those servers in the >>> changelog. >>> [20/Jun/2016:13:59:48 -0400] set_krb5_creds - Could not get initial >>> credentials for principal [ldap/server1.domain.local@DOMAIN.LOCAL] in >>> keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC >>> for requested realm) >>> [20/Jun/2016:13:59:48 -0400] set_krb5_creds - Could not get initial >>> credentials for principal [ldap/server1.domain.local@DOMAIN.LOCAL] in >>> keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC >>> for requested realm) >>> [20/Jun/2016:13:59:48 -0400] set_krb5_creds - Could not get initial >>> credentials for principal [ldap/server1.domain.local@DOMAIN.LOCAL] in >>> keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC >>> for requested realm) >>> [20/Jun/2016:13:59:48 -0400] slapd_ldap_sasl_interactive_bind - Error: >>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error > -2 >>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS >>> failure. Minor code may provide more information (Credentials cache file >>> '/tmp/krb5cc_495' not found)) errno 2 (No such file or directory) >>> [20/Jun/2016:13:59:48 -0400] slapi_ldap_bind - Error: could not perform >>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error) >>> [20/Jun/2016:13:59:48 -0400] slapd_ldap_sasl_interactive_bind - Error: >>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error > -2 >>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS >>> failure. Minor code may provide more information (Credentials cache file >>> '/tmp/krb5cc_495' not found)) errno 2 (No such file or directory) >>> [20/Jun/2016:13:59:48 -0400] slapi_ldap_bind - Error: could not perform >>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error) >>> [20/Jun/2016:13:59:48 -0400] NSMMReplicationPlugin - >>> agmt="cn=meToserver9.domain.local" (server9:389): Replication bind with >>> GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic >> failure: >>> GSSAPI Error: Unspecified GSS failure. Minor code may provide more >>> information (Credentials cache file '/tmp/krb5cc_495' not found)) >>> [20/Jun/2016:13:59:48 -0400] set_krb5_creds - Could not get initial >>> credentials for principal [ldap/server1.domain.local@DOMAIN.LOCAL] in >>> keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC >>> for requested realm) >>> [20/Jun/2016:13:59:48 -0400] NSMMReplicationPlugin - >>> agmt="cn=meToserver8.domain.local" (server8:389): Replication bind with >>> GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic >> failure: >>> GSSAPI Error: Unspecified GSS failure. Minor code may provide more >>> information (Credentials cache file '/tmp/krb5cc_495' not found)) >>> [20/Jun/2016:13:59:48 -0400] slapd_ldap_sasl_interactive_bind - Error: >>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error > -2 >>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS >>> failure. Minor code may provide more information (Credentials cache file >>> '/tmp/krb5cc_495' not found)) errno 2 (No such file or directory) >>> [20/Jun/2016:13:59:48 -0400] slapi_ldap_bind - Error: could not perform >>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error) >>> [20/Jun/2016:13:59:48 -0400] NSMMReplicationPlugin - >>> agmt="cn=meTobldvxl0011.domain.local" (1server:389): Replication bind >> with >>> GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic >> failure: >>> GSSAPI Error: Unspecified GSS failure. Minor code may provide more >>> information (Credentials cache file '/tmp/krb5cc_495' not found)) >>> [20/Jun/2016:13:59:48 -0400] slapd_ldap_sasl_interactive_bind - Error: >>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error > -2 >>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS >>> failure. Minor code may provide more information (Credentials cache file >>> '/tmp/krb5cc_495' not found)) errno 2 (No such file or directory) >>> [20/Jun/2016:13:59:48 -0400] slapi_ldap_bind - Error: could not perform >>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error) >>> [20/Jun/2016:13:59:48 -0400] NSMMReplicationPlugin - >>> agmt="cn=meToserver3.domain.local" (server3:389): Replication bind with >>> GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic >> failure: >>> GSSAPI Error: Unspecified GSS failure. Minor code may provide more >>> information (Credentials cache file '/tmp/krb5cc_495' not found)) >>> [20/Jun/2016:13:59:48 -0400] set_krb5_creds - Could not get initial >>> credentials for principal [ldap/server1.domain.local@DOMAIN.LOCAL] in >>> keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC >>> for requested realm) >>> [20/Jun/2016:13:59:48 -0400] set_krb5_creds - Could not get initial >>> credentials for principal [ldap/server1.domain.local@DOMAIN.LOCAL] in >>> keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC >>> for requested realm) >>> [20/Jun/2016:13:59:48 -0400] slapd_ldap_sasl_interactive_bind - Error: >>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error > -2 >>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS >>> failure. Minor code may provide more information (Credentials cache file >>> '/tmp/krb5cc_495' not found)) errno 2 (No such file or directory) >>> [20/Jun/2016:13:59:48 -0400] slapi_ldap_bind - Error: could not perform >>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error) >>> [20/Jun/2016:13:59:48 -0400] NSMMReplicationPlugin - >>> agmt="cn=meToserver7.domain.local" (server7:389): Replication bind with >>> GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic >> failure: >>> GSSAPI Error: Unspecified GSS failure. Minor code may provide more >>> information (Credentials cache file '/tmp/krb5cc_495' not found)) >>> [20/Jun/2016:13:59:48 -0400] - slapd started. Listening on All > Interfaces >>> port 389 for LDAP requests >>> [20/Jun/2016:13:59:48 -0400] - Listening on All Interfaces port 636 for >>> LDAPS requests >>> [20/Jun/2016:13:59:48 -0400] - Listening >>> on /var/run/slapd-DOMAIN-LOCAL.socket for LDAPI requests >>> [20/Jun/2016:13:59:48 -0400] slapd_ldap_sasl_interactive_bind - Error: >>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error > -2 >>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS >>> failure. Minor code may provide more information (Credentials cache file >>> '/tmp/krb5cc_495' not found)) errno 2 (No such file or directory) >>> [20/Jun/2016:13:59:48 -0400] slapi_ldap_bind - Error: could not perform >>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error) >>> [20/Jun/2016:13:59:48 -0400] NSMMReplicationPlugin - >>> agmt="cn=meToserver4.domain.local" (server4:389): Replication bind with >>> GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic >> failure: >>> GSSAPI Error: Unspecified GSS failure. Minor code may provide more >>> information (Credentials cache file '/tmp/krb5cc_495' not found)) >>> [20/Jun/2016:13:59:48 -0400] set_krb5_creds - Could not get initial >>> credentials for principal [ldap/server1.domain.local@DOMAIN.LOCAL] in >>> keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC >>> for requested realm) >>> [20/Jun/2016:13:59:48 -0400] slapd_ldap_sasl_interactive_bind - Error: >>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error > -2 >>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS >>> failure. Minor code may provide more information (Credentials cache file >>> '/tmp/krb5cc_495' not found)) errno 0 (Success) >>> [20/Jun/2016:13:59:48 -0400] slapi_ldap_bind - Error: could not perform >>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error) >>> [20/Jun/2016:13:59:48 -0400] NSMMReplicationPlugin - >>> agmt="cn=meToserver2.domain.local" (server2:389): Replication bind with >>> GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic >> failure: >>> GSSAPI Error: Unspecified GSS failure. Minor code may provide more >>> information (Credentials cache file '/tmp/krb5cc_495' not found)) >>> [20/Jun/2016:13:59:51 -0400] NSMMReplicationPlugin - >>> agmt="cn=meToserver8.domain.local" (server8:389): Replication bind with >>> GSSAPI auth resumed >>> [20/Jun/2016:13:59:51 -0400] slapd_ldap_sasl_interactive_bind - Error: >>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error > 49 >>> (Invalid credentials) (SASL(-13): authentication failure: GSSAPI > Failure: >>> gss_accept_sec_context) errno 0 (Success) >>> [20/Jun/2016:13:59:51 -0400] slapi_ldap_bind - Error: could not perform >>> interactive bind for id [] mech [GSSAPI]: error 49 (Invalid credentials) >>> [20/Jun/2016:13:59:51 -0400] NSMMReplicationPlugin - >>> agmt="cn=meToserver3.domain.local" (server3:389): Replication bind with >>> GSSAPI auth failed: LDAP error 49 (Invalid credentials) (SASL(-13): >>> authentication failure: GSSAPI Failure: gss_accept_sec_context) >>> [20/Jun/2016:13:59:51 -0400] slapd_ldap_sasl_interactive_bind - Error: >>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error > -2 >>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS >>> failure. Minor code may provide more information (No credentials cache >>> found)) errno 2 (No such file or directory) >>> [20/Jun/2016:13:59:51 -0400] slapi_ldap_bind - Error: could not perform >>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error) >>> [20/Jun/2016:13:59:51 -0400] slapd_ldap_sasl_interactive_bind - Error: >>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error > -2 >>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS >>> failure. Minor code may provide more information (No credentials cache >>> found)) errno 2 (No such file or directory) >>> [20/Jun/2016:13:59:51 -0400] slapi_ldap_bind - Error: could not perform >>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error) >>> [20/Jun/2016:13:59:51 -0400] slapd_ldap_sasl_interactive_bind - Error: >>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error > -2 >>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS >>> failure. Minor code may provide more information (No credentials cache >>> found)) errno 2 (No such file or directory) >>> [20/Jun/2016:13:59:51 -0400] slapi_ldap_bind - Error: could not perform >>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error) >>> [20/Jun/2016:13:59:51 -0400] slapd_ldap_sasl_interactive_bind - Error: >>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error > -2 >>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS >>> failure. Minor code may provide more information (No credentials cache >>> found)) errno 2 (No such file or directory) >>> [20/Jun/2016:13:59:51 -0400] slapi_ldap_bind - Error: could not perform >>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error) >>> [20/Jun/2016:13:59:51 -0400] slapd_ldap_sasl_interactive_bind - Error: >>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error > -2 >>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS >>> failure. Minor code may provide more information (No credentials cache >>> found)) errno 2 (No such file or directory) >>> [20/Jun/2016:13:59:51 -0400] slapi_ldap_bind - Error: could not perform >>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error) >>> [20/Jun/2016:13:59:57 -0400] slapd_ldap_sasl_interactive_bind - Error: >>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error > 49 >>> (Invalid credentials) (SASL(-13): authentication failure: GSSAPI > Failure: >>> gss_accept_sec_context) errno 0 (Success) >>> [20/Jun/2016:13:59:57 -0400] slapi_ldap_bind - Error: could not perform >>> interactive bind for id [] mech [GSSAPI]: error 49 (Invalid credentials) >>> [20/Jun/2016:13:59:57 -0400] NSMMReplicationPlugin - >>> agmt="cn=meToserver2.domain.local" (server2:389): Replication bind with >>> GSSAPI auth resumed >> >> >> >> Sean Hogan >> >> >> >> >> >> Inactive hide details for Petr Spacek ---06/21/2016 10:20:43 PM---On >> 22.6.2016 02:56, Sean Hogan wrote: > More infoPetr Spacek ---06/21/2016 >> 10:20:43 PM---On 22.6.2016 02:56, Sean Hogan wrote: > More info >> >> From: Petr Spacek <pspa...@redhat.com> >> To: freeipa-users@redhat.com >> Date: 06/21/2016 10:20 PM >> Subject: Re: [Freeipa-users] IPA 3.0.47 to 3.0.50 Upgrade problem >> Sent by: freeipa-users-boun...@redhat.com >> >> >> >> On 22.6.2016 02:56, Sean Hogan wrote: >>> More info >>> >>> >>> Krb5 log is showing: >>> Jun 21 20:42:47 Firstmaster.domain.local krb5kdc[2141](info): AS_REQ (4 >>> etypes {18 17 16 23}) 10.x.x.x: LOOKING_UP_CLIENT: admin@domain.LOCAL > for >>> krbtgt/DOMAIN.LOCAL@DOMAIN.LOCAL, Server error >> >> >> Hello, >> >> this is really fishy. I would bet that there is a problem with LDAP > server >> and >> DNS errors are just consequence of it. >> >> I suspect that you will not be able to finish steps mentioned in >> > https://fedorahosted.org/bind-dyndb-ldap/wiki/BIND9/NamedCannotStart#a3.FailedtoinitcredentialsorFailedtogetinitialcredentialsDecryptintegritycheckfailedorClientscredentialshavebeenrevoked > >> >> >> If it is the case I would turn your attention to krb5kdc.log and LDAP >> server >> logs in /var/log/dirsrv/* >> >> There must be something wrong with the LDAP server. >> >> Petr^2 Spacek >> >> >>> >>> [bob@Firstmaster etc]# kinit -v admin >>> kinit: Credentials cache file '/tmp/krb5cc_0' not found while validating >>> credentials >>> >>> >>> >>> >>> >>> >>> Sean Hogan >>> >>> >>> >>> >>> >>> >>> From: Sean Hogan/Durham/IBM >>> To: freeipa-users <freeipa-users@redhat.com> >>> Date: 06/21/2016 12:02 PM >>> Subject: Re: [Freeipa-users] IPA 3.0.47 to 3.0.50 Upgrade problem >>> >>> >>> Has anyone seen these before? >>> >>> >>> >>> First Master IPA DNS logs show: Looks like the host names are getting >> the >>> domain twice domain.local.domain.local >>> >>> >>> client 10.x.x.x#58094: query failed (SERVFAIL) for >>> server1.domain.local.domain.local/IN/AAAA at query.c:6569 >>> timeout in ldap_pool_getconnection(): try to raise 'connections' >> parameter; >>> potential deadlock? >>> client 10.x.x.x#44147: query failed (SERVFAIL) for >>> x.x.x.10.in-addr.arpa/IN/PTR at query.c:6569 >>> timeout in ldap_pool_getconnection(): try to raise 'connections' >> parameter; >>> potential deadlock? >>> client 10.x.x.x#56466: query failed (SERVFAIL) for >>> x.x.x.10.in-addr.arpa/IN/PTR at query.c:6569 >>> timeout in ldap_pool_getconnection(): try to raise 'connections' >> parameter; >>> potential deadlock? >>> client 10.x.x.x53367: query failed (SERVFAIL) for >>> server2.domain.local.domain.local/IN/A at query.c:6569 >>> timeout in ldap_pool_getconnection(): try to raise 'connections' >> parameter; >>> potential deadlock? >>> client 10.x.x.x#53367: query failed (SERVFAIL) for >>> server2.domain.local.domain.local/IN/AAAA at query.c:6569 >>> >>> >>> >>> So enrolls are failing at this point when tyring to enroll to a replica: >>> >>> [bob@server1 log]# ipa-client-install –enable-dns-updates >>> Discovery was successful! >>> Hostname: server1.watson.local >>> Realm: DOMAIN.LOCAL >>> DNS Domain: domain.local >>> IPA Server: ipareplica.domain.local >>> BaseDN: dc=domain,dc=local >>> >>> Continue to configure the system with these values? [no]: yes >>> User authorized to enroll computers: bob >>> Synchronizing time with KDC... >>> Password for bob@DOMAIN.LOCAL: >>> Successfully retrieved CA cert >>> Subject: CN=Certificate Authority,O=DOMAIN.LOCAL >>> Issuer: CN=Certificate Authority,O=DOMAIN.LOCAL >>> Valid From: Tue Jan 06 19:37:09 2015 UTC >>> Valid Until: Sat Jan 06 19:37:09 2035 UTC >>> >>> Enrolled in IPA realm DOMAIN.LOCAL >>> Attempting to get host TGT... >>> Created /etc/ipa/default.conf >>> New SSSD config will be created >>> Configured sudoers in /etc/nsswitch.conf >>> Configured /etc/sssd/sssd.conf >>> Configured /etc/krb5.conf for IPA realm DOMAIN.LOCAL >>> trying https://ipareplica.domain.local/ipa/xml >>> Cannot connect to the server due to Kerberos error: Kerberos error: >>> Kerberos error: ('Unspecified GSS failure. Minor code may provide more >>> information', 851968)/('KDC returned error string: PROCESS_TGS', >>> -1765328324)/. Trying with delegate=True >>> trying https://ipareplica.domain.local/ipa/xml >>> Second connect with delegate=True also failed: Kerberos error: Kerberos >>> error: ('Unspecified GSS failure. Minor code may provide more >>> information', 851968)/('KDC returned error string: PROCESS_TGS', >>> -1765328324)/ >>> Cannot connect to the IPA server XML-RPC interface: Kerberos error: >>> Kerberos error: ('Unspecified GSS failure. Minor code may provide more >>> information', 851968)/('KDC returned error string: PROCESS_TGS', >>> -1765328324)/ >>> Installation failed. Rolling back changes. >>> Unenrolling client from IPA server >>> Unenrolling host failed: Error obtaining initial credentials: Generic >> error >>> (see e-text). >>> >>> Removing Kerberos service principals from /etc/krb5.keytab >>> Disabling client Kerberos and LDAP configurations >>> Redundant SSSD configuration file /etc/sssd/sssd.conf was moved >>> to /etc/sssd/sssd.conf.deleted >>> Restoring client configuration files >>> nscd daemon is not installed, skip configuration >>> nslcd daemon is not installed, skip configuration >>> Client uninstall complete. >>> >>> >>> Sean Hogan >>> >>> >>> >>> >>> >>> >>> >>> >>> From: Sean Hogan/Durham/IBM >>> To: Sean Hogan/Durham/IBM@IBMUS >>> Cc: freeipa-users <freeipa-users@redhat.com> >>> Date: 06/20/2016 12:49 PM >>> Subject: Re: [Freeipa-users] IPA 3.0.47 to 3.0.50 Upgrade problem >>> >>> >>> Also seeing this in the upgrade log on the first master but not on the 7 >>> ipas. >>> >>> ERROR Failed to restart named: Command '/sbin/service named restart ' >>> returned non-zero exit status 7 >>> >>> >>> which led me to >>> >>> https://bugzilla.redhat.com/show_bug.cgi?id=895298 >>> >>> >>> >>> >>> >>> Sean Hogan >>> >>> >>> >>> >>> >>> >>> >>> From: Sean Hogan/Durham/IBM@IBMUS >>> To: freeipa-users <freeipa-users@redhat.com> >>> Date: 06/20/2016 11:46 AM >>> Subject: Re: [Freeipa-users] IPA 3.0.47 to 3.0.50 Upgrade problem >>> Sent by: freeipa-users-boun...@redhat.com >>> >>> >>> >>> Hi All.. >>> >>> I thought we fixed this issue by rebooting the KVM host but it is > showing >>> again. Our First Master IPA is being rebooted 2 -5 times a day now just >> to >>> keep it alive. >>> >>> What we are seeing: >>> >>> God@FirstMaster log]# kinit admin >>> kinit: Cannot contact any KDC for realm 'Domain.LOCAL' while getting >>> initial credentials >>> >>> DNS is not working as nslookup is failing to a replica.... think once we >>> lose DNS it all goes down hill which makes sense. >>> >>> [god@FirstMaster log]# ipactl stop -----> Just hangs forever.. no >> replies.. >>> no error.. nothing >>> >>> I try service named stop and nothing happens >>> >>> I have the box hard shutdown from KVM console. Reboot it and it works > for >> a >>> little while but eventually back to same behavior. >>> >>> At this point I can service named stop and it responds... ipactl status >> and >>> it responds.. but when if I try service named restart I get >>> >>> [god@FirstMaster log]# service named stop >>> Stopping named: ...... >>> >>> [god@Firstmaster log]# service named start >>> Starting named: [FAILED] >>> >>> [god@FirstMaster log]# service named status >>> rndc: connect failed: 127.0.0.1#953: connection refused >>> named dead but pid file exists >>> >>> Rebooted box and it is hung on shutting down domain-local and never > fully >>> shuts down.. have to get it hard shutdown again. >>> During an attempt to gracefully shut down we see this >>> >>> Shutting Down dirsrv: >>> PKI-IPA OK >>> DOMAIN-LOCAL FAILED >>> *** Error: 1 instance(s) unsuccessfully stopped FAILED >>> >>> Then it moves on to shut other things down and returns to dirsrv >>> Shutting Down dirsrv: >>> PKI-IPA....server already stopped FAILED {Makes sense.. it died earlier} >>> DOMAIN-LOCAL... {this sits here til we hard shutdown} >>> >>> >>> >>> bind-libs-9.8.2-0.47.rc1.el6.x86_64 >>> bind-9.8.2-0.47.rc1.el6.x86_64 >>> bind-utils-9.8.2-0.47.rc1.el6.x86_64 >>> >>> >>> ipa-client-3.0.0-50.el6.1.x86_64 >>> ipa-server-selinux-3.0.0-50.el6.1.x86_64 >>> ipa-server-3.0.0-50.el6.1.x86_64 >>> sssd-ipa-1.13.3-22.el6.x86_64 >>> >>> >>> /var/log/dirsrv/slapd-DOMAIN-LOCAL >>> [20/Jun/2016:13:29:06 -0400] - 389-Directory/1.2.11.15 B2016.063.2110 >>> starting up >>> [20/Jun/2016:13:29:06 -0400] schema-compat-plugin - warning: no entries >> set >>> up under cn=computers, cn=compat,dc=domain,dc=local >>> [20/Jun/2016:13:29:07 -0400] NSMMReplicationPlugin - ruv_compare_ruv: > RUV >>> [database RUV] does not contain element [{replica 7} > 55ca26a0000900070000 >>> 5688d8e6001000070000] which is present in RUV [changelog max RUV] >>> [20/Jun/2016:13:29:07 -0400] NSMMReplicationPlugin - >>> replica_check_for_data_reload: Warning: for replica dc=domain,dc=local >>> there were some differences between the changelog max RUV and the >> database >>> RUV. If there are obsolete elements in the database RUV, you should >> remove >>> them using the CLEANALLRUV task. If they are not obsolete, you should >> check >>> their status to see why there are no changes from those servers in the >>> changelog. >>> [20/Jun/2016:13:29:07 -0400] set_krb5_creds - Could not get initial >>> credentials for principal [ldap/server1.domain.local@DOMAIN.LOCAL] in >>> keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC >>> for requested realm) >>> [20/Jun/2016:13:29:07 -0400] set_krb5_creds - Could not get initial >>> credentials for principal [ldap/server1.domain.local@DOMAIN.LOCAL] in >>> keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC >>> for requested realm) >>> [20/Jun/2016:13:29:07 -0400] set_krb5_creds - Could not get initial >>> credentials for principal [ldap/server1.domain.local@DOMAIN.LOCAL] in >>> keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC >>> for requested realm) >>> [20/Jun/2016:13:29:07 -0400] slapd_ldap_sasl_interactive_bind - Error: >>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error > -2 >>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS >>> failure. Minor code may provide more information (Credentials cache file >>> '/tmp/krb5cc_495' not found)) errno 2 (No such file or directory) >>> [20/Jun/2016:13:29:07 -0400] slapi_ldap_bind - Error: could not perform >>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error) >>> [20/Jun/2016:13:29:07 -0400] NSMMReplicationPlugin - >>> agmt="cn=meToserver8.domain.local" (server8:389): Replication bind with >>> GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic >> failure: >>> GSSAPI Error: Unspecified GSS failure. Minor code may provide more >>> information (Credentials cache file '/tmp/krb5cc_495' not found)) >>> [20/Jun/2016:13:29:07 -0400] set_krb5_creds - Could not get initial >>> credentials for principal [ldap/server1.domain.local@DOMAIN.LOCAL] in >>> keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC >>> for requested realm) >>> [20/Jun/2016:13:29:07 -0400] slapd_ldap_sasl_interactive_bind - Error: >>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error > -2 >>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS >>> failure. Minor code may provide more information (Credentials cache file >>> '/tmp/krb5cc_495' not found)) errno 2 (No such file or directory) >>> [20/Jun/2016:13:29:07 -0400] slapi_ldap_bind - Error: could not perform >>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error) >>> [20/Jun/2016:13:29:07 -0400] NSMMReplicationPlugin - >>> agmt="cn=meToserver9.domain.local" (server9:389): Replication bind with >>> GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic >> failure: >>> GSSAPI Error: Unspecified GSS failure. Minor code may provide more >>> information (Credentials cache file '/tmp/krb5cc_495' not found)) >>> [20/Jun/2016:13:29:07 -0400] set_krb5_creds - Could not get initial >>> credentials for principal [ldap/server1.domain.local@DOMAIN.LOCAL] in >>> keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC >>> for requested realm) >>> [20/Jun/2016:13:29:07 -0400] slapd_ldap_sasl_interactive_bind - Error: >>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error > -2 >>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS >>> failure. Minor code may provide more information (Credentials cache file >>> '/tmp/krb5cc_495' not found)) errno 2 (No such file or directory) >>> [20/Jun/2016:13:29:07 -0400] slapi_ldap_bind - Error: could not perform >>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error) >>> [20/Jun/2016:13:29:07 -0400] NSMMReplicationPlugin - >>> agmt="cn=meToserver3.domain.local" (server3:389): Replication bind with >>> GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic >> failure: >>> GSSAPI Error: Unspecified GSS failure. Minor code may provide more >>> information (Credentials cache file '/tmp/krb5cc_495' not found)) >>> [20/Jun/2016:13:29:07 -0400] set_krb5_creds - Could not get initial >>> credentials for principal [ldap/server1.domain.local@DOMAIN.LOCAL] in >>> keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC >>> for requested realm) >>> [20/Jun/2016:13:29:07 -0400] - slapd started. Listening on All > Interfaces >>> port 389 for LDAP requests >>> [20/Jun/2016:13:29:07 -0400] - Listening on All Interfaces port 636 for >>> LDAPS requests >>> [20/Jun/2016:13:29:07 -0400] - Listening >>> on /var/run/slapd-DOMAIN-LOCAL.socket for LDAPI requests >>> [20/Jun/2016:13:29:07 -0400] set_krb5_creds - Could not get initial >>> credentials for principal [ldap/server1.domain.local@DOMAIN.LOCAL] in >>> keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC >>> for requested realm) >>> [20/Jun/2016:13:29:07 -0400] slapd_ldap_sasl_interactive_bind - Error: >>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error > -2 >>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS >>> failure. Minor code may provide more information (Credentials cache file >>> '/tmp/krb5cc_495' not found)) errno 2 (No such file or directory) >>> [20/Jun/2016:13:29:07 -0400] slapi_ldap_bind - Error: could not perform >>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error) >>> [20/Jun/2016:13:29:07 -0400] NSMMReplicationPlugin - >>> agmt="cn=meToserver4.domain.local" (server4:389): Replication bind with >>> GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic >> failure: >>> GSSAPI Error: Unspecified GSS failure. Minor code may provide more >>> information (Credentials cache file '/tmp/krb5cc_495' not found)) >>> [20/Jun/2016:13:29:07 -0400] slapd_ldap_sasl_interactive_bind - Error: >>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error > -2 >>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS >>> failure. Minor code may provide more information (Credentials cache file >>> '/tmp/krb5cc_495' not found)) errno 2 (No such file or directory) >>> [20/Jun/2016:13:29:07 -0400] slapi_ldap_bind - Error: could not perform >>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error) >>> [20/Jun/2016:13:29:07 -0400] NSMMReplicationPlugin - >>> agmt="cn=meTo1server.domain.local" (1server:389): Replication bind with >>> GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic >> failure: >>> GSSAPI Error: Unspecified GSS failure. Minor code may provide more>> >>> information (Credentials cache file '/tmp/krb5cc_495' not found)) >>> [20/Jun/2016:13:29:07 -0400] slapd_ldap_sasl_interactive_bind - Error: >>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error > -2 >>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS >>> failure. Minor code may provide more information (Credentials cache file >>> '/tmp/krb5cc_495' not found)) errno 2 (No such file or directory) >>> [20/Jun/2016:13:29:07 -0400] slapi_ldap_bind - Error: could not perform >>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error) >>> [20/Jun/2016:13:29:07 -0400] NSMMReplicationPlugin - >>> agmt="cn=meToserver7.domain.local" (server7:389): Replication bind with >>> GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic >> failure: >>> GSSAPI Error: Unspecified GSS failure. Minor code may provide more >>> information (Credentials cache file '/tmp/krb5cc_495' not found)) >>> [20/Jun/2016:13:29:07 -0400] slapd_ldap_sasl_interactive_bind - Error: >>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error > -2 >>> (Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS >>> failure. Minor code may provide more information (Credentials cache file >>> '/tmp/krb5cc_495' not found)) errno 0 (Success) >>> [20/Jun/2016:13:29:07 -0400] slapi_ldap_bind - Error: could not perform >>> interactive bind for id [] mech [GSSAPI]: error -2 (Local error) >>> [20/Jun/2016:13:29:07 -0400] NSMMReplicationPlugin - >>> agmt="cn=meToserver2.domain.local" (server2:389): Replication bind with >>> GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic >> failure: >>> GSSAPI Error: Unspecified GSS failure. Minor code may provide more >>> information (Credentials cache file '/tmp/krb5cc_495' not found)) >>> [20/Jun/2016:13:29:10 -0400] NSMMReplicationPlugin - -- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project