On 24/01/2018 15:22, Rob Crittenden via FreeIPA-users wrote:
Roderick Johnstone via FreeIPA-users wrote:
On 23/01/2018 14:34, Rob Crittenden via FreeIPA-users wrote:
Roderick Johnstone via FreeIPA-users wrote:
On 15/01/2018 20:07, Rob Crittenden via FreeIPA-users wrote:
Roderick Johnstone via FreeIPA-users wrote:
On 15/01/2018 16:06, Rob Crittenden via FreeIPA-users wrote:
Roderick Johnstone via FreeIPA-users wrote:
Hi
Our freeipa certificates need to be renewed due to passing their
expiry
dates.
While some certificates have renewed ok, the ipaCert and
auditSigningCert are renewing but the new certificates have the
wrong
Subject.
Environment is:
serverA (CRL, first, master) RHEL 7.3, ipa 4.4
serverB (replica) RHEL 7.3, ipa 4.4
serverC (replica) RHEL 7.4, ipa 4.5
Once there are renewed certificates with the wrong Subject present,
there are various problems with renewing the remaining certificates,
which I think might be related to the bad Subject:
1) When just ipaCert has the wrong subject no further renewals
happen
2) When auditSigningCert has the wrong subject the ipa pki-tomcatd
service will not start and no further renewals happen.
I've been round the following loop many times on ServerA, our first
master:
1) Restore good certificates from backup
2) Put the clock back to a time when certificates are all valid
3) Resubmit certificates for renewal
Each time the ipaCert renews it has the same wrong Subject. The
wrong
Subject includes the host name of one of our ipa client systems.
Each time the auditSigningCert renews it has the same wrong Subject
but
a different subject to the ipaCert. The wrong Subject in this case
includes the host name of a system which has never been an ipa
client,
but might have been added and removed with ipa host-add and ipa
host-del
for testing something, a while ago.
As far as I can see, the "cert_subject" is set correctly in the file
/var/lib/certmonger/<request id> until the point at which the
certificate is actually renewed.
I'd be very grateful for some pointers as to which configuration
options
and logs to check through to resolve this problem on our production
system.
If its of any relevance we did change which server is the first
master
some time ago.
I'd pull the CSR out of dogtag (CS.cfg) and/or certmonger to see what
the subject is.
I'm not seeing any obvious CSR fields in the
/etc/pki/pki-tomcat/ca/CS.cfg file.
foo.bar.certreq=
The CSR in the certmonger requests file for the auditSigningCert seems
to be showing with the correct Subject. This is different from the bad
subject showing in the requests file field:
cert_subject=
The value of cert_subject comes from the issued certificate.
and the Subject which is showing in the 'getcert list' output
(which is
the same as that in the cert_subject= field.>
I'm not quite sure what this all means.
It is displayed from the data within the tracked certmonger request.
certmonger logs to syslog so you can check there or you can stop the
process and run it manually with: certmonger -n -d 9 2>&1 | tee
certmonger.log
That will provide a lot of debugging output that may show what is
going on.
I've restored certificate databases from backup and put the clock back
to a time when certificates are valid and renewed the ocspSigining
certificate with:
getcert resubmit -N "CN=OCSP Subsystem,O=<REALM>" -i 20161124081302
(I've previously tried without the -N with similar results)
What I am seeing in the certmonger logs is:
2017-10-23 00:05:28 [438] Located the key 'ocspSigningCert cert-pki-ca'.
2017-10-23 00:05:28 [438] Converted private key 'ocspSigningCert
cert-pki-ca' to public key.
2017-10-23 00:05:28 [439] Located the certificate "ocspSigningCert
cert-pki-ca".
2017-10-23 00:05:28 [440] 0x1d Certificate named "ocspSigningCert
cert-pki-ca" in token "NSS Certificate DB" in database
"/etc/pki/pki-tomcat/alias" will not be valid after 20171025122401.
2017-10-23 00:05:28 [442] Located the key 'ocspSigningCert cert-pki-ca'.
2017-10-23 00:05:28 [442] Converted private key 'ocspSigningCert
cert-pki-ca' to public key.
2017-10-23 00:05:28 [443] Located the certificate "ocspSigningCert
cert-pki-ca".
2017-10-23 00:05:28 [444] Located the key 'ocspSigningCert cert-pki-ca'.
2017-10-23 00:05:28 [444] Converted private key 'ocspSigningCert
cert-pki-ca' to public key.
2017-10-23 00:05:39 [581] Found a certificate with the same nickname but
different subject, removing certificate "ocspSigningCert cert-pki-ca"
with subject "CN=OCSP Subsystem,O=<REALM>".
2017-10-23 00:05:39 [581] Imported certificate "ocspSigningCert
cert-pki-ca", got nickname "ocspSigningCert cert-pki-ca".
2017-10-23 00:05:39 [583] Located the certificate "ocspSigningCert
cert-pki-ca".
2017-10-23 00:05:39 [48576] Adding hook
"/usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert
cert-pki-ca"" (0).
2017-10-23 00:10:43 [942] 0x1d Certificate named "ocspSigningCert
cert-pki-ca" in token "NSS Certificate DB" in database
"/etc/pki/pki-tomcat/alias" issued by CA and saved.
I now have a date valid ocspSigningCertificate, but with the wrong
subject, and a broken certificate system which will no longer start.
ipactl status
...
pki-tomcatd Service: STOPPED
I can't renew other expired certificates.
I also note that there is now no key for ocspSigningCertificate as shown
by:
certutil -K -d /etc/pki/pki-tomcat/alias
I wonder if this is because the certificate subject changed? There was a
key before the certificate renewed.
The ca debug logs are showing:
[23/Oct/2017:00:55:47][localhost-startStop-1]: Found cert by nickname:
'ocspSigningCert cert-pki-ca' with serial number: 268370108
[23/Oct/2017:00:55:47][localhost-startStop-1]: converted to x509CertImpl
[23/Oct/2017:00:55:47][localhost-startStop-1]: SigningUnit: Certificate
object not found
Certificate object not found
at com.netscape.ca.SigningUnit.init(SigningUnit.java:184)
Any help in repairing my broken ipa system will be much appreciated.
Sorry for the delay. I think my previous reading of the certmonger
csrgen code was wrong.
IIRC in your certmonger entry the cert_subject has the hostname value
right? Do you also have cert_subject_der?
You can decode that by:
1. Running a hex-to-bin, something hacky like this in python:
import binascii
hex_string = "<hex value>"
binary_string = binascii.unhexlify(hex_string)
fd = open("out", "wb")
fd.write(binary_string)
fd.close()
2. Run: openssl asn1parse -in out -inform der
I'm going to assume this also has the hostname encoded in it.
Hi Again
Yes, correct. The cert_subject_der does have the bad CN with the
hostname encoded in it.
Can you try this:
1. Make a backup of the request file (just in case) > 2. Remove
cert_subject_der
3. Modify cert_subject to be CN=OCSP Subsystem,O=<YOUR_REALM>
3. Try the renewal again
I ran though all of this, checking that the request file was still what
we wanted after restarting certmonger with verbose logging, restoring
all the databases, fixing permissions, checking keys etc., and ran the
getcert resubmit.
It still generates a certificate with the bad CN including a hostname.
Then the pki-tomcat server fails to start again.
Other things I have checked are:
1) The csr= field in the (modified) request file has the correct subject.
2) The dogtag ca debug log file is showing:
processRenewal: origNotAfter =Wed Oct 25 13:24:01 BST 2017
processRenewal: orig subj dn =CN=OCSP Subsystem,O=AST.CAM.AC.UK
...
RenewalSubmitter: renewal original profileId=caIPAserviceCert
RenewalSubmitter: for renewal, original authenticator raCertAuth found
CertProcessor: No authenticator credentials required
processRenewal: set original Inputs into profile Context
RenewalSubmitter: setInputsIntoContext() getting input name=
cert_request_type
RenewalSubmitter: setInputsIntoContext() setting value in ctx:pkcs10
RenewalSubmitter: setInputsIntoContext() getting input name= cert_request
RenewalSubmitter: setInputsIntoContext() setting value in ctx:<hex
encoded csr>
...
Finish parsePKCS10 - CN=<HOSTNAME>,O=<REALM>
The hex encoded csr in the last line decodes to have the bad subject
where the hostname is one of our other ipa servers.
The certificate always getthe same hostname in the bad subject cn.
Do you have any other ideas of things to check to find out what's
generating the bad subject?
The order certmonger uses for the subject is:
1. cert_subject_der
2. If there is no DER, try to encode cert_subject
3. If that fails try again a different way
4. If it fails yet again use CN=localhost
It is baffling that it would pick a hostname much less one that
certmonger shouldn't even know about.
Indeed, and if I try to renew other certificates for some of them it
chooses other host names that should not be known about. Each
certificate seems to get the same bad hostname each time I try to renew.
I assume the CSR found in the dogtag logs matches the csr value in the
certmonger request?
No, its different. The issued certificate matches the csr seen in dogtag
which makes sense. But the csr in the dogtag logs has the bad subject.
The csr in the request directory file has the good subject.
Can you share the certmonger request file privately?
Yes, certainly.
Thanks for your continued help on this.
Roderick
_______________________________________________
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org
