Hi Florence,
thank you very much for your help.
On Fri, 21 Dec 2018, Florence Blanc-Renaud via FreeIPA-users wrote:
On 12/20/18 6:52 PM, dbischof--- via FreeIPA-users wrote:
On Thu, 20 Dec 2018, Florence Blanc-Renaud via FreeIPA-users wrote:
On 12/20/18 4:22 PM, dbischof--- via FreeIPA-users wrote:
my IPA system consists of 2 masters with their own self-signed CAs,
one of
them being the certificate renewal master (ipa1). The system has been
running for years and has been migrated from an IPA 3 system.
Since a while, the Web UI logins on ipa1 don't work anymore ("Login
failed
due to an unknown reason.").
Web UI logins on the other server (ipa2) work and everything else is
working fine, too, ipactl status reports all services running.
On login attempt:
--- httpd log
[...]
[:error] [pid 15551] [remote 141.51.X.X:0] mod_wsgi (pid=15551):
Exception
occurred processing WSGI script '/usr/share/ipa/wsgi.py'.
[...]
[:error] [pid 15551] [remote 141.51.X.X:0] CalledProcessError: Command
'/usr/bin/kinit -n -c /var/run/ipa/ccaches/armor_15551 -X
X509_anchors=FILE:/var/kerberos/krb5kdc/kdc.crt -X
X509_anchors=FILE:/var/lib/ipa-client/pki/kdc-ca-bundle.pem' returned
non-zero exit status 1
---
--- krb5kdc.log
[...]
Dec 20 16:06:54 ipa1.example.com krb5kdc[15517](info): AS_REQ (8 etypes
{18 17 20 19 16 23 25 26}) 141.51.X.Y: NEEDED_PREAUTH:
WELLKNOWN/[email protected] for krbtgt/[email protected],
Additional pre-authentication required
Dec 20 16:06:54 ipa1.example.com krb5kdc[15517](info): closing down fd
11
Dec 20 16:06:54 ipa1.example.com krb5kdc[15518](info): AS_REQ (8 etypes
{18 17 20 19 16 23 25 26}) 141.51.X.Y: KDC_RETURN_PADATA:
WELLKNOWN/[email protected] for krbtgt/[email protected],
Failed
to verify own certificate (depth 0): certificate has expired
Dec 20 16:06:54 ipa1.example.com krb5kdc[15518](info): closing down fd
11
---
--- ipa-checkcerts.py
IPA version 4.5.4-10.el7.centos.3
Check CA status
Check tracking
Check NSS trust
Check dates
Checking certificates in CS.cfg
Comparing certificates to requests in LDAP
Checking RA certificate
Checking authorities
Checking host keytab
Validating certificates
Checking renewal master
End-to-end cert API test
Checking permissions and ownership
Failures:
Unable to find request for serial 268304391
Unable to find request for serial 268304394
Unable to find request for serial 268304393
Unable to find request for serial 268304392
Subject O=EXAMPLE.COM,CN=ipa2.example.com and template subject
CN=ipa1.example.com,O=EXAMPLE.COM do not match for serial 57
---
--- ipa pkinit-status --all
-----------------
2 servers matched
-----------------
Server name: ipa2.example.com
PKINIT status: enabled
Server name: ipa1.example.com
PKINIT status: enabled
----------------------------
Number of entries returned 2
----------------------------
To my understanding, proper certificate exchange between my two servers
ceased working at some point. How do i track this down and fix it?
your issue looks similar to ticket #6792 [1]. Can you check the result of
upgrade in /var/log/ipaupgrade.log?
Also check the output of
$ ipa-pkinit-manage status
and if the files /var/lib/ipa-client/pki/kdc-ca-bundle.pem and
/var/lib/ipa-client/pki/ca-bundle.pem exist, with -rw-r--r-- permissions.
Regarding the certificates, does getcert list show expired certs?
flo
[1] https://pagure.io/freeipa/issue/6792
---
$ ipa-pkinit-manage status
PKINIT is enabled
---
There are no expired certificates, kdc-ca-bundle.pem and ca-bundle.pem
exist with proper permissions, but I found something in ipaupgrade.log:
---
2018-09-12T13:37:18Z DEBUG args=/usr/bin/certutil -d /etc/httpd/alias -L
-f /etc/httpd/alias/pwdfile.txt
2018-09-12T13:37:19Z DEBUG Process finished, return code=0
2018-09-12T13:37:19Z DEBUG stdout=
Certificate Nickname Trust
Attributes
SSL,S/MIME,JAR/XPI
Server-Cert u,u,u
EXAMPLE.COM IPA CA CT,C,C
2018-09-12T13:37:19Z DEBUG stderr=
2018-09-12T13:37:19Z DEBUG Starting external process
2018-09-12T13:37:19Z DEBUG args=/usr/bin/certutil -d /etc/httpd/alias -L
-n EXAMPLE.COM IPA CA -a -f /etc/httpd/alias/pwdfile.txt
2018-09-12T13:37:19Z DEBUG Process finished, return code=0
2018-09-12T13:37:19Z DEBUG stdout=
-----BEGIN CERTIFICATE-----
[...]
-----END CERTIFICATE-----
2018-09-12T13:37:19Z DEBUG stderr=
2018-09-12T13:37:19Z DEBUG Executing upgrade plugin: update_ra_cert_store
2018-09-12T13:37:19Z DEBUG raw: update_ra_cert_store
2018-09-12T13:37:19Z DEBUG raw: ca_is_enabled(version=u'2.228')
2018-09-12T13:37:19Z DEBUG ca_is_enabled(version=u'2.228')
2018-09-12T13:37:19Z DEBUG Starting external process
2018-09-12T13:37:19Z DEBUG args=/usr/bin/certutil -d /etc/httpd/alias -L
-n ipaCert -a -f /etc/httpd/alias/pwdfile.txt
2018-09-12T13:37:19Z DEBUG Process finished, return code=255
2018-09-12T13:37:19Z DEBUG stdout=
2018-09-12T13:37:19Z DEBUG stderr=certutil: Could not find cert: ipaCert
: PR_FILE_NOT_FOUND_ERROR: File not found
[...]
---
this error can be ignored in most of the cases. The upgrade is trying to
move ipaCert (cert+key) from the NSS database /etc/httpd/alias to the
files /var/lib/ipa/ra-agent.pem and /var/lib/ipa/ra-agent.key. So if the
upgrade is run a second time, he won't find ipaCert in the NSS database.
To be sure, you can check if /var/lib/ipa/ra-agent.{pem|key} are present
and contain a certificate with Subject CN=IPA RA,O=DOMAIN.COM. The files
must be readable by root and ipaapi group, and must contain the same
cert as the other masters.
checked this, is ok.
What is the content of /var/lib/ipa-client/pki/kdc-ca-bundle.pem and
ca-bundle.pem? both must contain IPA CA certificate.
True on both servers.
What are the permissions of /var/kerberos/krb5kdc/kdc.crt? It needs to
be readable by everyone. And what is the content of this cert? It should
be issued by IPA CA.
Permissions are ok, contents:
--- ipa1
$ openssl x509 -in /var/kerberos/krb5kdc/kdc.crt -text -noout -fingerprint
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 1 (0x1)
Signature Algorithm: sha256WithRSAEncryption
Issuer: O=EXAMPLE.COM, CN=ipa1.example.com
Validity
Not Before: Nov 28 12:43:05 2017 GMT
Not After : Nov 28 12:43:05 2018 GMT
Subject: O=EXAMPLE.COM, CN=ipa1.example.com
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
[...]
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Subject Alternative Name:
othername:<unsupported>, othername:<unsupported>
X509v3 Basic Constraints: critical
CA:FALSE
X509v3 Subject Key Identifier:
86:52:EC:A1:C3:FB:EC:CC:6D:F2:09:E7:64:88:D1:80:F4:71:81:AE
1.3.6.1.4.1.311.20.2:
.".K.D.C.s._.P.K.I.N.I.T._.C.e.r.t.s
[...]
---
--- ipa2
$ openssl x509 -in /var/kerberos/krb5kdc/kdc.crt -text -noout -fingerprint
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 805240833 (0x2fff0001)
Signature Algorithm: sha256WithRSAEncryption
Issuer: O=EXAMPLE.COM, CN=Certificate Authority
Validity
Not Before: Jan 18 13:04:17 2018 GMT
Not After : Jan 19 13:04:17 2020 GMT
Subject: O=EXAMPLE.COM, CN=ipa2.example.com
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
[...]
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Authority Key Identifier:
keyid:4B:BA:AA:46:F1:29:E4:43:8B:DC:30:B4:90:3E:66:72:DD:F6:C7:FB
Authority Information Access:
OCSP - URI:http://ipa-ca.example.com/ca/ocsp
X509v3 Key Usage: critical
Digital Signature, Non Repudiation, Key Encipherment, Data
Encipherment
X509v3 Extended Key Usage:
TLS Web Server Authentication, 1.3.6.1.5.2.3.5
X509v3 CRL Distribution Points:
Full Name:
URI:http://ipa-ca.example.com/ipa/crl/MasterCRL.bin
CRL Issuer:
DirName: O = ipaca, CN = Certificate Authority
X509v3 Subject Key Identifier:
96:C3:94:70:7E:46:77:DB:91:F8:DF:D6:27:FE:73:0A:45:F3:78:F3
X509v3 Subject Alternative Name:
othername:<unsupported>, othername:<unsupported>
[...]
---
Additional info: I have DNS separate from IPA, but i (hopefully) made
proper records as IPA would have done it. In particular, i made an A
record "ipa-ca" that has IPs of both ipa1 and ipa2 - hope, this is not the
root cause of my problems, since DNS is not under my control.
Since /var/kerberos/krb5kdc/kdc.crt on ipa1 appears to be not issued by
IPA CA, might this be the actual problem?
About the errors spotted by ipa-checkcerts.py, what are the certificates
with errors reported? You can find them with ipa cert-show <serial>.
---
$ipa cert-show 57
Issuing CA: ipa
Certificate: [...]
Subject: CN=ipa1.example.com,O=EXAMPLE.COM
Issuer: CN=Certificate Authority,O=EXAMPLE.COM
Not Before: Tue Nov 28 12:42:07 2017 UTC
Not After: Mon Nov 18 12:42:07 2019 UTC
Serial number: 57
Serial number (hex): 0x39
Revoked: False
$ ipa cert-show 268304391
Issuing CA: ipa
Certificate: [...]
Subject: CN=IPA RA,O=EXAMPLE.COM
Issuer: CN=Certificate Authority,O=EXAMPLE.COM
Not Before: Wed Jul 11 14:26:35 2018 UTC
Not After: Tue Jun 30 14:26:35 2020 UTC
Serial number: 268304391
Serial number (hex): 0xFFE0007
Revoked: False
$ ipa cert-show 268304392
Issuing CA: ipa
Certificate: [...]
Subject: CN=CA Subsystem,O=EXAMPLE.COM
Issuer: CN=Certificate Authority,O=EXAMPLE.COM
Not Before: Wed Jul 11 14:26:45 2018 UTC
Not After: Tue Jun 30 14:26:45 2020 UTC
Serial number: 268304392
Serial number (hex): 0xFFE0008
Revoked: False
$ ipa cert-show 268304393
Issuing CA: ipa
Certificate: [...]
Subject: CN=OCSP Subsystem,O=EXAMPLE.COM
Issuer: CN=Certificate Authority,O=EXAMPLE.COM
Not Before: Wed Jul 11 14:27:05 2018 UTC
Not After: Tue Jun 30 14:27:05 2020 UTC
Serial number: 268304393
Serial number (hex): 0xFFE0009
Revoked: False
$ ipa cert-show 268304394
Issuing CA: ipa
Certificate: [...]
Subject: CN=CA Audit,O=EXAMPLE.COM
Issuer: CN=Certificate Authority,O=EXAMPLE.COM
Not Before: Wed Jul 11 14:27:25 2018 UTC
Not After: Tue Jun 30 14:27:25 2020 UTC
Serial number: 268304394
Serial number (hex): 0xFFE000A
Revoked: False
---
ipa cert-show on server ipa2 fails with "ipa: ERROR: Certificate operation
cannot be completed: EXCEPTION (Invalid Credential.)" btw.
Mit freundlichen Gruessen/With best regards,
--Daniel.
_______________________________________________
FreeIPA-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedorahosted.org/archives/list/[email protected]
