Hi,
On Wed, Apr 16, 2025 at 1:25 AM Mark Selby <[email protected]> wrote:
Thank you very much for your response. I have done some further
debugging and have more info in out situation and what the genesis
of the issue is. I am hoping you have some suggestions on a resolution
I am in a bit of a pickle
(1) We have nightly full backups of IPA but they have been running
on a NON CA server. So we do not have the ipaca tree in our backups.
(2) We had a broken CA replication agreement between out two CA
servers and mostly all requests went to only one of the servers.
(3) We regenerated our CAs using the server that did not get the
replicated requests. So we lost all certs and requests from "time
A" onward.
The situation on our two CA servers now is that we have the below
certs in /etc/pki/pki-tomcat/alias and
/var/lib/certmonger/requests, but not in ldap
"CN=CA Audit,O=SNAFU.NET <http://SNAFU.NET>" Wed Jun 17 17:45:30 2026
"CN=CA Subsystem,O=SNAFU.NET <http://SNAFU.NET>" Wed Jul 01
17:52:00 2026
"CN=OCSP Subsystem,O=SNAFU.NET <http://SNAFU.NET>" Thu Apr 03
13:00:01 2025
(4) This is what I have on the CA server(s)
certutil -L -d . -n 'ocspSigningCert cert-pki-ca'
Serial Number: 580 (0x244)
Signature Algorithm: PKCS #1 SHA-256 With RSA Encryption
Issuer: "CN=Certificate Authority,O=SNAFU.NET <http://SNAFU.NET>"
Validity:
Not Before: Fri Apr 14 13:00:01 2023
Not After : Thu Apr 03 13:00:01 2025
Subject: "CN=OCSP Subsystem,O=SNAFU.NET <http://SNAFU.NET>"
If you have the cert in the NSSDB, you can export it to a pem format with
certutil -L -d . -n 'ocspSigningCert cert-pki-ca' -a
As you need to have the content without the header/footer and in a
single line, you can use:
certutil -L -d /etc/pki/pki-tomcat/alias -n 'ocspSigningCert
cert-pki-ca' -a | tail -n +2 | head -n -1 | tr -d '\r\n' > /tmp/cert.out
Then you can re-create the LDAP entry with this content in the
"userCertificate;binary" attribute. You can copy-paste from an
existing entry in order to add the proper attributes and objectclass.
flo
/var/lib/certmonger/requests/20241109195943 - which is the req for
the above
(5) The OCSP is expired and ipa-cert-fix fails because we do not
have either of the below in ldap
cn=580,ou=certificateRepository,ou=ca,o=ipaca (with metaInfo:
requestId:####)
cn=####,ou=ca,ou=requests,o=ipaca
(6) What I do have is an older version of the cert and request in
ldap, but I do not have that cert locally in the nss db or in
certmonger
dn: cn=558,ou=certificateRepository,ou=ca,o=ipaca
issuerName: CN=Certificate Authority,O=VOLEON.NET <http://VOLEON.NET>
subjectName: CN=OCSP Subsystem,O=VOLEON.NET <http://VOLEON.NET>
duration: 1162208000000
notAfter: 20240707001455Z
notBefore: 20220718001455Z
metaInfo: requestId:685
metaInfo: profileId:caOCSPCert
serialno: 03558
objectClass: top
objectClass: certificateRecord
dn: cn=685,ou=ca,ou=requests,o=ipaca
cn: 685
dateOfCreate: 20220718001455Z
requestId: 03685
extdata-authenticatedname: CN=OCSP Subsystem,O=SNAFU.NET
<http://SNAFU.NET>
I was thinking that I could use ipa-cert-fix against the older
cert pair but I am unsure of how do accomplish this. Is it as
simple as replacing
the cert in the nss db? Do I need to hack a
/var/lib/certmonger/requests file.
I will take any suggestion even if it is a painful one,.
On 4/14/25 1:19 AM, Florence Blanc-Renaud wrote:
Hi,
On Sun, Apr 13, 2025 at 7:25 PM Mark Selby via FreeIPA-users
<[email protected]> wrote:
Our old FreeIPA cluster ipa-server-4.6.8-5.el7 (which we will
upgrade if/when this issue resolved) has a non functional CA
due to the ocspSigningCert being expired.
I have tried all of the suggested fixes that others with this
issue have suggested. ipa-cert-fix and running pki-server
cert-fix directly all fail.
ipa-cert-fix
pki-server cert-fix --ldapi-socket
/var/run/slapd-SNAFU-NET.socket --agent-uid ipara --cert
ca_ocsp_signing
I have also tried setting back the clock on one of the CA
servers and running both ipa-cert-fix and getcert resubmit.
This sugesstion https://access.redhat.com/solutions/3939431
was also tried.
What is the output of ipa-cert-fix --verbose or pki-server
cert-fix --verbose?
When I turn back the clock and restart the pki server at
least I can look at the certificates as the CA. When the
clock is normal the pki server will not start due to the
invalid cert
[13/Apr/2025:09:10:03][localhost-startStop-1]: CertUtils:
verifySystemCertsByTag() failed: java.lang.Exception:
Certutils.verifySystemCertValidityByNickname: faliled:
nickname:ocspSigningCert cert-pki-cacause:
java.lang.Exception:
Certutils.verifySystemCertValidityByNickname: failed:
nickname: ocspSigningCert cert-pki-ca
I turned on verbose=4 logging in CS.cfg and received the
below debug output. The serial number of the ocsp cert is 580
as fas the certmonger is concerned. The pki server is
seemingly connecting to LDAP to lookup that cert and ends up
with the Record not found error. I can not seem to find which
ldap tree the server is looking in for this record or which
record it is searching for.
The cert-fix tool is looking for the CSR in CS.cfg and for the
cert in the LDAP tree. The CSR are also stored below
ou=ca,ou=requests,o=ipaca and the certs below
ou=certificateRepository,ou=ca,o=ipaca.
Do you have your ocsp cert in the subtree
ou=certificateRepository,ou=ca,o=ipaca? You can check with
ldapsearch -D cn=directory\ manager -W -b
"ou=certificateRepository,ou=ca,o=ipaca" "(subjectname=cn=ocsp*)"
We had a few bugs with cert-fix, happening when the CSR was
missing from /etc/pki/pki-tomcat/ca/CS.cfg (1780782
<https://bugzilla.redhat.com/show_bug.cgi?id=1780782>) or when
some parameters were missing from CS.cfg (1930586
<https://bugzilla.redhat.com/show_bug.cgi?id=1930586>). With your
verbose output from cert-fix we may be able to find if you're
hitting one of those issues.
flo
I believe that I have all the correct certs in ldap under
cn=ca_renewal, but of course the ocspSigningCert in ldap is
expired.
Does anyone know what is happening here and what a possible
fix might be. All and any help is greatly appreciated
cd /etc/pki/pki-tomcat/alias
certutil -L -d . -n 'ocspSigningCert cert-pki-ca'
Validity:
Not Before: Fri Apr 14 13:00:01 2023
Not After : Thu Apr 03 13:00:01 2025
Subject: "CN=OCSP Subsystem,O=SNAFU.NET
<http://SNAFU.NET>"
Version: 3 (0x2)
Serial Number: 580 (0x244)
dn: cn=ca_renewal,cn=ipa,cn=etc,dc=snafu,dc=net
dn: cn=ipaCert,cn=ca_renewal,cn=ipa,cn=etc,dc=snafu,dc=net
dn: cn=subsystemCert
cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=snafu,dc=net
dn: cn=auditSigningCert
cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=snafu,dc=net
dn: cn=ocspSigningCert
cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=snafu,dc=net
[12/Apr/2025:17:55:31][http-bio-8443-exec-1]:
CAProcessor.java:252:printParameterValues() CAProcessor:
Input Parameters:
[12/Apr/2025:17:55:31][http-bio-8443-exec-1]:
CAProcessor.java:264:printParameterValues() CAProcessor: -
isRenewal: true
[12/Apr/2025:17:55:31][http-bio-8443-exec-1]:
CAProcessor.java:264:printParameterValues() CAProcessor: -
remoteHost: 10.17.1.18
[12/Apr/2025:17:55:31][http-bio-8443-exec-1]:
CAProcessor.java:264:printParameterValues() CAProcessor: -
profileId: caManualRenewal
[12/Apr/2025:17:55:31][http-bio-8443-exec-1]:
CAProcessor.java:264:printParameterValues() CAProcessor: -
serial_num: 580
[12/Apr/2025:17:55:31][http-bio-8443-exec-1]:
CAProcessor.java:264:printParameterValues() CAProcessor: -
remoteAddr: 10.17.1.18
[12/Apr/2025:17:55:31][http-bio-8443-exec-1]:
RenewalProcessor.java:90:processRenewal() RenewalProcessor:
processRenewal()
[12/Apr/2025:17:55:31][http-bio-8443-exec-1]:
RenewalProcessor.java:98:processRenewal() RenewalProcessor:
profile: caManualRenewal
[12/Apr/2025:17:55:31][http-bio-8443-exec-1]:
RenewalProcessor.java:149:processRenewal() RenewalProcessor:
found SerialNumRenewInput
[12/Apr/2025:17:55:31][http-bio-8443-exec-1]:
RenewalProcessor.java:157:processRenewal() RenewalProcessor:
profile input serial_num value: 580
[12/Apr/2025:17:55:31][http-bio-8443-exec-1]:
RenewalProcessor.java:181:processRenewal() processRenewal:
serial number of cert to renew:580
[12/Apr/2025:17:55:31][http-bio-8443-exec-1]:
LdapBoundConnFactory.java:324:getConn() In
LdapBoundConnFactory::getConn()
[12/Apr/2025:17:55:31][http-bio-8443-exec-1]:
LdapBoundConnFactory.java:326:getConn() masterConn is
connected: true
[12/Apr/2025:17:55:31][http-bio-8443-exec-1]:
LdapBoundConnFactory.java:368:getConn() getConn: conn is
connected true
[12/Apr/2025:17:55:31][http-bio-8443-exec-1]:
LdapBoundConnFactory.java:398:getConn() getConn: mNumConns now 2
[12/Apr/2025:17:55:31][http-bio-8443-exec-1]:
LdapBoundConnFactory.java:444:returnConn() returnConn:
mNumConns now 3
Record not found
at
com.netscape.cmscore.dbs.DBSSession.read(DBSSession.java:182)
at
com.netscape.cmscore.dbs.DBSSession.read(DBSSession.java:137)
at
com.netscape.cmscore.dbs.CertificateRepository.readCertificateRecord(CertificateRepository.java:1023)
at
com.netscape.cms.servlet.cert.RenewalProcessor.processRenewal(RenewalProcessor.java:182)
at
com.netscape.cms.servlet.cert.CertRequestDAO.submitRequest(CertRequestDAO.java:194)
at
org.dogtagpki.server.ca.rest.CertRequestService.enrollCert(CertRequestService.java:155)
at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:137)
at
org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:280)
at
org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:234)
at
org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:221)
at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:356)
at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:179)
at
org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:220)
at
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
at
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
at
javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
at
org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
at java.security.AccessController.doPrivileged(Native
Method)
at
javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
at
org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
at
org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:175)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:297)
at
org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:55)
at
org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:191)
at
org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:187)
at java.security.AccessController.doPrivileged(Native
Method)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:186)
at
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
at
org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
at java.security.AccessController.doPrivileged(Native
Method)
at
javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
at
org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
at
org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:260)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:237)
at
org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:55)
at
org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:191)
at
org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:187)
at java.security.AccessController.doPrivileged(Native
Method)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:186)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:218)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:498)
--
_______________________________________________
FreeIPA-users mailing list --
[email protected]
To unsubscribe send an email to
[email protected]
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedorahosted.org/archives/list/[email protected]
Do not reply to spam, report it:
https://pagure.io/fedora-infrastructure/new_issue
