Hi Fraser, I am making some progress. Let's please continue. [1] I was able to follow your info and find common date in past for all certs to be valid. Note, in case this is important, I have four IPA servers and I do this on CA renewal master.
[2] Then system clock was set to past time (about 2 weeks before expire time) , stop ntp, restart krb5kdc, dirsrv, httpd, CA. The I verify that CA is running, with command : SSL_DIR=/etc/httpd/alias/ curl -v -o /dev/null --cacert /etc/ipa/ca.crt https://`hostname`:8443/ca/agent/ca/profileReview * Initializing NSS with certpath: sql:/etc/httpd/alias/ * CAfile: /etc/ipa/ca.crt CApath: none * NSS: client certificate not found (nickname not specified) * SSL connection using TLS_RSA_WITH_AES_256_CBC_SHA * Server certificate: * subject: CN=ca-ldap04,O=REALM.COM * start date: Aug 01 17:18:06 2018 GMT * expire date: Jul 21 17:18:06 2020 GMT * common name: ca-ldap04 * issuer: CN=Certificate Authority,O=US.ORACLE.COM > GET /ca/agent/ca/profileReview HTTP/1.1 > User-Agent: curl/7.29.0 > Host: ca-ldap04:8443 > Accept: */* > * NSS: client certificate not found (nickname not specified) < HTTP/1.1 200 OK < Server: Apache-Coyote/1.1 < Content-Type: text/html;charset=UTF-8 < Transfer-Encoding: chunked < Date: Wed, 01 Aug 2018 18:28:04 GMT < { [data not shown] 100 17641 0 17641 0 0 230k 0 --:--:-- --:--:-- --:--:-- 232k * Connection #0 to host ca-ldap04.realm.com left intact [3] Restart certmonger, and ONLY ONE cert is renewed, it's "Server-Cert cert-pki-ca". status: CA_UNREACHABLE certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB' expires: 2018-08-14 20:49:38 UTC status: CA_UNREACHABLE certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB' expires: 2018-08-14 20:49:35 UTC status: CA_UNREACHABLE certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB' expires: 2018-08-14 20:49:36 UTC status: MONITORING certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB' expires: 2036-08-24 20:49:35 UTC status: MONITORING certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB' expires: 2020-07-21 17:18:06 UTC status: CA_UNREACHABLE certificate: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB' expires: 2018-08-14 20:50:00 UTC [4] From "journalctl -fu certmonger --full " basically there is " Insufficient access: Invalid credentials" Aug 01 11:04:45 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:45 [7447] Will revisit CA6('dogtag-ipa-ca-renew-agent').default_profile now. Aug 01 11:04:45 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:45 [7447] CA6('dogtag-ipa-ca-renew-agent').default_profile moved to state 'DISABLED' Aug 01 11:04:45 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:45 [7447] Waiting for instructions for CA6('dogtag-ipa-ca-renew-agent').default_profile. Aug 01 11:04:45 ca-ldap04.realm.com dogtag-ipa-ca-renew-agent-submit[7526]: Traceback (most recent call last): File "/usr/libexec/certmonger/dogtag-ipa-ca-renew-agent-submit", line 511, in <module> sys.exit(main()) File "/usr/libexec/certmonger/dogtag-ipa-ca-renew-agent-submit", line 497, in main if ca.is_renewal_master(): File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 1188, in is_renewal_master self.ldap_connect() File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 177, in ldap_connect conn.do_bind(self.dm_password, autobind=self.autobind) File "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 1690, in do_bind self.do_sasl_gssapi_bind(timeout=timeout) File "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 1668, in do_sasl_gssapi_bind self.__bind_with_wait(self.gssapi_bind, timeout) File "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 1650, in __bind_with_wait bind_func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 1108, in gssapi_bind '', auth_tokens, server_controls, client_controls) File "/usr/lib64/python2.7/contextlib.py", line 35, in __exit__ self.gen.throw(type, value, traceback) File "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 973, in error_handler raise errors.ACIError(info="%s %s" % (info, desc)) ACIError: Insufficient access: Invalid credentials Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Certificate submission still ongoing. Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Will revisit Request1('20161112162146') on traffic from 92. Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] CA6('dogtag-ipa-ca-renew-agent').profiles retrieval unsupported Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] CA6('dogtag-ipa-ca-renew-agent').profiles moved to state 'DISABLED' Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Will revisit CA6('dogtag-ipa-ca-renew-agent').profiles now. Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] CA6('dogtag-ipa-ca-renew-agent').profiles moved to state 'DISABLED' Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Waiting for instructions for CA6('dogtag-ipa-ca-renew-agent').profiles. Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] CA6('dogtag-ipa-ca-renew-agent').capabilities retrieval unsupported Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] CA6('dogtag-ipa-ca-renew-agent').capabilities moved to state 'DISABLED' Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Will revisit CA6('dogtag-ipa-ca-renew-agent').capabilities now. Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] CA6('dogtag-ipa-ca-renew-agent').capabilities moved to state 'DISABLED' Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Waiting for instructions for CA6('dogtag-ipa-ca-renew-agent').capabilities. Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] CA6('dogtag-ipa-ca-renew-agent').renewal_reqs retrieval unsupported Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] CA6('dogtag-ipa-ca-renew-agent').renewal_reqs moved to state 'DISABLED' Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Will revisit CA6('dogtag-ipa-ca-renew-agent').renewal_reqs now. Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] CA6('dogtag-ipa-ca-renew-agent').renewal_reqs moved to state 'DISABLED' Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Waiting for instructions for CA6('dogtag-ipa-ca-renew-agent').renewal_reqs. Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] CA6('dogtag-ipa-ca-renew-agent').enrollment_reqs retrieval unsupported Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] CA6('dogtag-ipa-ca-renew-agent').enrollment_reqs moved to state 'DISABLED' Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Will revisit CA6('dogtag-ipa-ca-renew-agent').enrollment_reqs now. Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] CA6('dogtag-ipa-ca-renew-agent').enrollment_reqs moved to state 'DISABLED' Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Waiting for instructions for CA6('dogtag-ipa-ca-renew-agent').enrollment_reqs. Aug 01 11:04:46 ca-ldap04.realm.com python2[7533]: GSSAPI client step 1 Aug 01 11:04:46 ca-ldap04.realm.com python2[7533]: GSSAPI client step 1 Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] CA6('dogtag-ipa-ca-renew-agent').encryption_certs retrieval unsupported Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] CA6('dogtag-ipa-ca-renew-agent').encryption_certs moved to state 'DISABLED' Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Will revisit CA6('dogtag-ipa-ca-renew-agent').encryption_certs now. Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] CA6('dogtag-ipa-ca-renew-agent').encryption_certs moved to state 'DISABLED' Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Waiting for instructions for CA6('dogtag-ipa-ca-renew-agent').encryption_certs. Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Certificate submission attempt complete. Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Child status = 3. Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Child output: Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: "Internal error Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: " Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Internal error Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Certificate not (yet?) issued. Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Request1('20161112162146') moved to state 'CA_UNREACHABLE' Aug 01 11:04:46 ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Will revisit Request1('20161112162146') in 566546 seconds. _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org