Thank you Rob! I now have two years till everything expires... On Tue, Jun 21, 2016 at 1:33 PM, Rob Crittenden <rcrit...@redhat.com> wrote:
> Marc Wiatrowski wrote: > >> Thanks for the reply Rob, >> >> So should fixing replication be more than running a re-initialize? >> I've tried this with no luck. Still the same errors in renewing the IPA >> certs. >> > > re-init drops one database and replaces it with another. If you really did > that then you have potentially lost a ton of records if indeed replication > was stalled. Knowing what commands you ran would help to know for sure. I'm thinking at some point in the past I may have done this backwards. So maybe not my original problem but making things worse. > > > status: CA_UNREACHABLE >> ca-error: Server at https://spider01a.iglass.net/ipa/xml failed request, >> will retry: 4301 (RPC failed at server. Certificate operation cannot be >> completed: EXCEPTION (Certificate serial number 0x3ffe000f not found)) >> >> Is there a procedure for getting these serial numbers back in to the >> system? or manually recreating somehow? >> > > When IPA gets a certificate request and the host/service it is requesting > it for already has a certificate, a revocation is done on the existing > certificate (which in this case is failing because the cert is unknown). If > you wipe out the usercertificate field from the entry ldap/ > spider01a.iglass.net then that should do it. This did the trick! I also had to delete userCertificate for dogtagldap/ spider01a.iglass.net and HTTP/spider01a.iglass.net for the other two certificates not renewing. > > > >> I was able to clear 4301 error. One ipaCert needed to be updated. >> > > Great! > > rob > > >> thanks >> >> On Thu, Jun 16, 2016 at 10:22 AM, Rob Crittenden <rcrit...@redhat.com >> <mailto:rcrit...@redhat.com>> wrote: >> >> Marc Wiatrowski wrote: >> >> Thanks Rob, >> >> Any suggestions on how make the CA aware of the current serial >> number? >> >> >> Serial numbers are dolled out like uid numbers, by the 389-ds DNA >> Plugin. So each CA that has ever issued a certificate has its own >> range, hence the quite different serial number values. >> >> Given that some issued certificates are unknown it stands to reason >> that replication is broken between one or more masters. Fixing that >> should resolve (most of) the other issues. >> >> Also started seeing the following error from two of the servers, >> spider01b and spider01o, but not spider01a when to navigate in >> the web >> gui. Though it doesn't appear to stop me from doing anything. >> >> IPA Error 4301 >> Certificate operation cannot be completed: EXCEPTION (Invalid >> Crential.) >> >> >> Dogtag does some of its access control by comparing the incoming >> client certificate with an expected value in its LDAP database, in >> this case uid=ipara,ou=People,o=ipaca. There you'll find a copy of >> the client certificate and a description field that contains the >> expected serial #, subject and issuer. >> >> These are out-of-whack if you're getting Invalid Credentials. It >> could be a number of things so I'd proceed cautiously. Given you >> have a working master I'd use that as a starting point. >> >> Look at the the RA cert is in /etc/httpd/alias: >> >> # certutil -L -d /etc/httpd/alias -n ipaCert | grep Serial >> >> See if it is the same on all masters, it should be. >> >> If it is, look at the uid=ipara entry on all the masters. Again, >> should be the same. >> >> Note that fixing this won't address any replication issues. >> >> rob >> >> >> Marc >> >> On Tue, Jun 14, 2016 at 2:07 PM, Marc Wiatrowski <w...@iglass.net >> <mailto:w...@iglass.net> >> <mailto:w...@iglass.net <mailto:w...@iglass.net>>> wrote: >> >> >> >> On Tue, Jun 14, 2016 at 11:22 AM, Rob Crittenden >> <rcrit...@redhat.com <mailto:rcrit...@redhat.com> >> <mailto:rcrit...@redhat.com <mailto:rcrit...@redhat.com>>> wrote: >> >> Marc Wiatrowski wrote: >> >> Hello, I'm having issues with the 3 ipa >> certificates of type >> CA: IPA >> renewing on 2 of 3 replicas. Particularly on the 2 >> that are >> not the CA >> master. The other 5 certificates from getcert list >> do renew >> and all >> certificates on the CA master do look to renew. >> >> Both servers running >> ipa-server-3.0.0-50.el6.centos.1.x86_64 I've done >> full updates and rebooted. >> >> >> Can you check on the replication status for each CA? >> >> $ ipa-csreplica-manage list -v ipa.example.com >> <http://ipa.example.com> >> <http://ipa.example.com> >> >> The hostname is important because including that will >> show the >> agreements that host has. Do this for each master with >> a CA. >> >> The CA being asked to do the renewal is unaware of the >> current >> serial number so it is refusing to proceed. >> >> rob >> >> >> >> [root@spider01o]$ ipa-csreplica-manage list -v >> spider01a.iglass.net <http://spider01a.iglass.net> >> <http://spider01a.iglass.net> >> Directory Manager password: >> >> spider01b.iglass.net <http://spider01b.iglass.net> >> <http://spider01b.iglass.net> >> last init status: None >> last init ended: None >> last update status: 0 Replica acquired successfully: >> Incremental >> update succeeded >> last update ended: 2016-06-14 17:49:16+00:00 >> spider01o.iglass.net <http://spider01o.iglass.net> >> <http://spider01o.iglass.net> >> last init status: None >> last init ended: None >> last update status: 0 Replica acquired successfully: >> Incremental >> update started >> last update ended: 2016-06-14 17:55:20+00:00 >> >> [root@spider01o]$ ipa-csreplica-manage list -v >> spider01o.iglass.net <http://spider01o.iglass.net> >> <http://spider01o.iglass.net> >> Directory Manager password: >> >> spider01a.iglass.net <http://spider01a.iglass.net> >> <http://spider01a.iglass.net> >> last init status: None >> last init ended: None >> last update status: 0 Replica acquired successfully: >> Incremental >> update started >> last update ended: 2016-06-14 17:57:44+00:00 >> spider01b.iglass.net <http://spider01b.iglass.net> >> <http://spider01b.iglass.net> >> last init status: None >> last init ended: None >> last update status: 0 Replica acquired successfully: >> Incremental >> update started >> last update ended: 2016-06-14 17:57:41+00:00 >> >> [root@spider01o]$ ipa-csreplica-manage list -v >> spider01b.iglass.net <http://spider01b.iglass.net> >> <http://spider01b.iglass.net> >> Directory Manager password: >> >> spider01a.iglass.net <http://spider01a.iglass.net> >> <http://spider01a.iglass.net> >> last init status: 0 Total update succeeded >> last init ended: 2016-06-03 19:43:12+00:00 >> last update status: 0 Replica acquired successfully: >> Incremental >> update succeeded >> last update ended: 2016-06-14 17:44:17+00:00 >> spider01o.iglass.net <http://spider01o.iglass.net> >> <http://spider01o.iglass.net> >> last init status: 0 Total update succeeded >> last init ended: 2016-06-03 19:44:38+00:00 >> last update status: 0 Replica acquired successfully: >> Incremental >> update started >> last update ended: 2016-06-14 17:57:53+00:00 >> spider01a.iglass.net <http://spider01a.iglass.net> >> <http://spider01a.iglass.net> >> last init status: None >> last init ended: None >> last update status: 0 Replica acquired successfully: >> Incremental >> update succeeded >> last update ended: 2016-06-14 17:44:13+00:00 >> spider01o.iglass.net <http://spider01o.iglass.net> >> <http://spider01o.iglass.net> >> last init status: None >> last init ended: None >> last update status: 0 Replica acquired successfully: >> Incremental >> update started >> last update ended: 2016-06-14 17:57:54+00:00 >> >> >> Not sure what this is telling... This an issue with the >> last being >> doubled? Thanks >> >> >> >> The failed renews look like: >> >> [root@spider01a]$ getcert list -i 20141202144354 >> Number of certificates and requests being tracked: 8. >> Request ID '20141202144354': >> status: CA_UNREACHABLE >> ca-error: Server at https://spider01a.iglass.net/ipa/xml >> failed request, >> will retry: 4301 (RPC failed at server. Certificate >> operation cannot be >> completed: EXCEPTION (Certificate serial number 0x3ffe0010 >> not found)). >> stuck: no >> key pair storage: >> >> >> type=NSSDB,location='/etc/dirsrv/slapd-PKI-IPA',nickname='Server-Cert',token='NSS >> Certificate >> DB',pinfile='/etc/dirsrv/slapd-PKI-IPA/pwdfile.txt' >> certificate: >> >> >> type=NSSDB,location='/etc/dirsrv/slapd-PKI-IPA',nickname='Server-Cert',token='NSS >> Certificate DB' >> CA: IPA >> issuer: CN=Certificate Authority,O=IGLASS.NET >> <http://IGLASS.NET> >> <http://iglass.net/> <http://IGLASS.NET <http://iglass.net/ >> >> >> subject: CN=spider01a.iglass.net >> <http://spider01a.iglass.net> <http://spider01a.iglass.net/> >> <http://spider01a.iglass.net >> <http://spider01a.iglass.net/>>,O=IGLASS.NET >> <http://IGLASS.NET> >> >> <http://iglass.net/> <http://IGLASS.NET <http://iglass.net/ >> >> >> expires: 2016-12-02 14:38:45 UTC >> key usage: >> >> digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment >> eku: id-kp-serverAuth,id-kp-clientAuth >> pre-save command: >> post-save command: /usr/lib64/ipa/certmonger/restart_dirsrv >> PKI-IPA >> track: yes >> auto-renew: yes >> >> [root@spider01a]$ getcert list -i 20141202144616 >> Number of certificates and requests being tracked: 8. >> Request ID '20141202144616': >> status: CA_UNREACHABLE >> ca-error: Server at https://spider01a.iglass.net/ipa/xml >> failed request, >> will retry: 4301 (RPC failed at server. Certificate >> operation cannot be >> completed: EXCEPTION (Certificate serial number 0x3ffe000f >> not found)). >> stuck: no >> key pair storage: >> >> >> type=NSSDB,location='/etc/dirsrv/slapd-IGLASS-NET',nickname='Server-Cert',token='NSS >> Certificate >> DB',pinfile='/etc/dirsrv/slapd-IGLASS-NET/pwdfile.txt' >> certificate: >> >> >> type=NSSDB,location='/etc/dirsrv/slapd-IGLASS-NET',nickname='Server-Cert',token='NSS >> Certificate DB' >> CA: IPA >> issuer: CN=Certificate Authority,O=IGLASS.NET >> <http://IGLASS.NET> >> <http://iglass.net/> <http://IGLASS.NET <http://iglass.net/ >> >> >> subject: CN=spider01a.iglass.net >> <http://spider01a.iglass.net> <http://spider01a.iglass.net/> >> <http://spider01a.iglass.net >> <http://spider01a.iglass.net/>>,O=IGLASS.NET >> <http://IGLASS.NET> >> >> <http://iglass.net/> <http://IGLASS.NET <http://iglass.net/ >> >> >> expires: 2016-12-02 14:38:43 UTC >> key usage: >> >> digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment >> eku: id-kp-serverAuth,id-kp-clientAuth >> pre-save command: >> post-save command: /usr/lib64/ipa/certmonger/restart_dirsrv >> IGLASS-NET >> track: yes >> auto-renew: yes >> >> [root@spider01a]$ getcert list -i 20141202144733 >> Number of certificates and requests being tracked: 8. >> Request ID '20141202144733': >> status: CA_UNREACHABLE >> ca-error: Server at https://spider01a.iglass.net/ipa/xml >> failed request, >> will retry: 4301 (RPC failed at server. Certificate >> operation cannot be >> completed: EXCEPTION (Certificate serial number 0x3ffe0011 >> not found)). >> stuck: no >> key pair storage: >> >> >> type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS >> Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt' >> certificate: >> >> >> type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS >> Certificate DB' >> CA: IPA >> issuer: CN=Certificate Authority,O=IGLASS.NET >> <http://IGLASS.NET> >> <http://iglass.net/> <http://IGLASS.NET <http://iglass.net/ >> >> >> subject: CN=spider01a.iglass.net >> <http://spider01a.iglass.net> <http://spider01a.iglass.net/> >> <http://spider01a.iglass.net >> <http://spider01a.iglass.net/>>,O=IGLASS.NET >> <http://IGLASS.NET> >> >> <http://iglass.net/> <http://IGLASS.NET <http://iglass.net/ >> >> >> expires: 2016-12-02 14:38:46 UTC >> key usage: >> >> digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment >> eku: id-kp-serverAuth,id-kp-clientAuth >> pre-save command: >> post-save command: /usr/lib64/ipa/certmonger/restart_httpd >> track: yes >> auto-renew: yes >> >> >> From >> [root@spider01a]$ getcert resubmit -i 20141202144354 >> >> On the replica issuing the resubmit >> >> ==> /var/log/httpd/access_log <== >> 192.168.176.2 - - [13/Jun/2016:15:49:32 -0400] "POST >> /ipa/xml HTTP/1.1" >> 401 1370 >> >> ==> /var/log/httpd/error_log <== >> [Mon Jun 13 15:49:33 2016] [error] ipa: ERROR: >> ipaserver.plugins.dogtag.ra.get_certificate(): EXCEPTION >> (Certificate >> serial number 0x3ffe0010 not found) >> [Mon Jun 13 15:49:33 2016] [error] ipa: INFO: >> host/spider01a.iglass....@iglass.net >> <mailto:spider01a.iglass....@iglass.net> >> <mailto:spider01a.iglass....@iglass.net >> <mailto:spider01a.iglass....@iglass.net>> >> <mailto:spider01a.iglass....@iglass.net >> <mailto:spider01a.iglass....@iglass.net> >> <mailto:spider01a.iglass....@iglass.net >> <mailto:spider01a.iglass....@iglass.net>>>: >> >> >> cert_request(u'MIIDsTCCApkCAQAwNDETMBEGA1UEChMKSUdMQVNTLk5FVDEdMBsGA1UEAxMUc3BpZGVyMDFhLml...UVrN8lbKn17V5COjnj6k0mdbz3KptL0UI/l0BPlFBWGN5MFYaDx2F+y6LWv/aXeu2V4E6LA==', >> principal=u'dogtagldap/spider01a.iglass....@iglass.net >> <mailto:spider01a.iglass....@iglass.net> >> <mailto:spider01a.iglass....@iglass.net >> <mailto:spider01a.iglass....@iglass.net>> >> <mailto:spider01a.iglass....@iglass.net >> <mailto:spider01a.iglass....@iglass.net> >> <mailto:spider01a.iglass....@iglass.net >> <mailto:spider01a.iglass....@iglass.net>>>', add=True): >> CertificateOperationError >> >> ==> /var/log/httpd/access_log <== >> 192.168.176.2 - - [13/Jun/2016:15:49:33 -0400] "POST >> /ca/agent/ca/displayBySerial HTTP/1.1" 200 262 >> 192.168.176.2 - host/spider01a.iglass....@iglass.net >> <mailto:spider01a.iglass....@iglass.net> >> <mailto:spider01a.iglass....@iglass.net >> <mailto:spider01a.iglass....@iglass.net>> >> <mailto:spider01a.iglass....@iglass.net >> <mailto:spider01a.iglass....@iglass.net> >> <mailto:spider01a.iglass....@iglass.net >> <mailto:spider01a.iglass....@iglass.net>>> [13/Jun/2016:15:49:32 >> -0400] >> "POST /ipa/xml HTTP/1.1" 200 376 >> >> ==> /var/log/pki-ca/system <== >> 2508.TP-Processor6 - [13/Jun/2016:15:49:33 EDT] [3] [3] >> Servlet >> caDisplayBySerial: Error encountered in DisplayBySerial. >> Error Record >> not found. >> >> >> On the CA master spider01o: >> >> ==> /var/log/httpd/access_log <== >> 192.168.176.2 - - [13/Jun/2016:15:49:33 -0400] "POST >> /ipa/xml HTTP/1.1" >> 401 1370 >> >> ==> krb5kdc.log <== >> Jun 13 15:49:34 spider01o.iglass.net >> <http://spider01o.iglass.net> >> <http://spider01o.iglass.net/> <http://spider01o.iglass.net >> <http://spider01o.iglass.net/>> >> krb5kdc[1963](info): TGS_REQ (4 etypes {18 17 16 23}) >> 192.168.177.2 >> <http://192.168.177.2 <http://192.168.177.2/>>: ISSUE: >> authtime >> 1465847372, etypes {rep=18 >> tkt=18 ses=18}, host/spider01a.iglass....@iglass.net >> <mailto:spider01a.iglass....@iglass.net> >> <mailto:spider01a.iglass....@iglass.net >> <mailto:spider01a.iglass....@iglass.net>> >> <mailto:spider01a.iglass....@iglass.net >> <mailto:spider01a.iglass....@iglass.net> >> <mailto:spider01a.iglass....@iglass.net >> <mailto:spider01a.iglass....@iglass.net>>> for >> ldap/spider01o.iglass....@iglass.net >> <mailto:spider01o.iglass....@iglass.net> >> <mailto:spider01o.iglass....@iglass.net >> <mailto:spider01o.iglass....@iglass.net>> >> <mailto:spider01o.iglass....@iglass.net >> <mailto:spider01o.iglass....@iglass.net> >> <mailto:spider01o.iglass....@iglass.net >> <mailto:spider01o.iglass....@iglass.net>>> >> >> ==> /var/log/httpd/error_log <== >> [Mon Jun 13 15:49:34 2016] [error] ipa: ERROR: >> ipaserver.plugins.dogtag.ra.get_certificate(): EXCEPTION >> (Invalid >> Credential.) >> [Mon Jun 13 15:49:34 2016] [error] ipa: INFO: >> host/spider01a.iglass....@iglass.net >> <mailto:spider01a.iglass....@iglass.net> >> <mailto:spider01a.iglass....@iglass.net >> <mailto:spider01a.iglass....@iglass.net>> >> <mailto:spider01a.iglass....@iglass.net >> <mailto:spider01a.iglass....@iglass.net> >> <mailto:spider01a.iglass....@iglass.net >> <mailto:spider01a.iglass....@iglass.net>>>: >> >> >> cert_request(u'MIIDsTCCApkCAQAwNDETMBEGA1UEChMKSUdMQVNTLk5FVDEdMBsGA1UEAxMUc3BpZGVyMDFhLml...UVrN8lbKn17V5COjnj6k0mdbz3KptL0UI/l0BPlFBWGN5MFYaDx2F+y6LWv/aXeu2V4E6LA==', >> principal=u'dogtagldap/spider01a.iglass....@iglass.net >> <mailto:spider01a.iglass....@iglass.net> >> <mailto:spider01a.iglass....@iglass.net >> <mailto:spider01a.iglass....@iglass.net>> >> <mailto:spider01a.iglass....@iglass.net >> <mailto:spider01a.iglass....@iglass.net> >> <mailto:spider01a.iglass....@iglass.net >> <mailto:spider01a.iglass....@iglass.net>>>', add=True): >> CertificateOperationError >> >> ==> /var/log/httpd/access_log <== >> 192.168.177.2 - - [13/Jun/2016:15:49:34 -0400] "POST >> /ca/agent/ca/displayBySerial HTTP/1.1" 200 235 >> 192.168.176.2 - host/spider01a.iglass....@iglass.net >> <mailto:spider01a.iglass....@iglass.net> >> <mailto:spider01a.iglass....@iglass.net >> <mailto:spider01a.iglass....@iglass.net>> >> <mailto:spider01a.iglass....@iglass.net >> <mailto:spider01a.iglass....@iglass.net> >> <mailto:spider01a.iglass....@iglass.net >> <mailto:spider01a.iglass....@iglass.net>>> [13/Jun/2016:15:49:33 >> -0400] >> "POST /ipa/xml HTTP/1.1" 200 349 >> >> ==> /var/log/pki-ca/system <== >> 2231.TP-Processor3 - [13/Jun/2016:15:49:34 EDT] [6] [3] >> Cannot >> authenticate agent with certificate Serial 0x5ffc0008 >> Subject DN CN=IPA >> RA,O=IGLASS.NET <http://IGLASS.NET> <http://iglass.net/> >> <http://IGLASS.NET >> <http://iglass.net/>>. Error: User not found >> >> >> I realize they expire at the end of the year, but I've had my >> certificates expire before and would rather not go through >> that again. >> Any idea on what's wrong or suggestions on where to look >> would be >> appreciated. >> >> Thanks, >> Marc >> >> >> >> >> >> >> >
-- Manage your subscription for the Freeipa-users mailing list: https://www.redhat.com/mailman/listinfo/freeipa-users Go to http://freeipa.org for more info on the project