Responses in-line.
Duarte Petiz wrote:
> Hello, thank you Rob
> [root@prod-us-freeipa /]# ipa-healthcheck
> [
> {
> "source": "ipahealthcheck.ipa.certs",
> "check": "IPACertMatchCheck",
> "result": "ERROR",
> "uuid": "4bb93268-3bee-4c0c-8dc9-c1bd03293d34",
> "when": "20240905082820Z",
> "duration": "0.011836",
> "kw": {
> "key": "cn=JSCRAMBLER.COM <http://JSCRAMBLER.COM> IPA
> CA,cn=certificates,cn=ipa,cn=etc,dc=jscrambler,dc=com",
> "dn": "cn=JSCRAMBLER.COM <http://JSCRAMBLER.COM> IPA
> CA,cn=certificates,cn=ipa,cn=etc,dc=jscrambler,dc=com",
> "serial_number": 1,
> "msg": "CA Certificate serial number {serial} is in LDAP '{dn}'
> but is not in /etc/ipa/ca.crt"
> }
> },
Run this and see what certificate(s) are currently in /etc/ipa/ca.crt:
openssl crl2pkcs7 -nocrl -certfile /etc/ipa/ca.crt | openssl pkcs7
-print_certs -text -noout
It is unexpected that the IPA CA cert is not present in /etc/ipa/ca.crt.
> {
> "source": "ipahealthcheck.ipa.certs",
> "check": "IPADogtagCertsMatchCheck",
> "result": "ERROR",
> "uuid": "f2f7f0e4-5d46-4e45-9ca6-56ae855e56e9",
> "when": "20240905082820Z",
> "duration": "0.070035",
> "kw": {
> "key": "caSigningCert cert-pki-ca",
> "nickname": "caSigningCert cert-pki-ca",
> "dbdir": "/etc/pki/pki-tomcat/alias",
> "msg": "{nickname} certificate in NSS DB {dbdir} does not match
> entry in LDAP"
> }
> },
> {
> "source": "ipahealthcheck.ipa.certs",
> "check": "IPADogtagCertsMatchCheck",
> "result": "ERROR",
> "uuid": "8c591194-50d3-489e-a078-7f9fadb6c2fd",
> "when": "20240905082820Z",
> "duration": "0.399892",
> "kw": {
> "key": "ocspSigningCert cert-pki-ca",
> "nickname": "ocspSigningCert cert-pki-ca",
> "dbdir": "/etc/pki/pki-tomcat/alias",
> "msg": "{nickname} certificate in NSS DB {dbdir} does not match
> entry in LDAP"
> }
> },
> {
> "source": "ipahealthcheck.ipa.certs",
> "check": "IPADogtagCertsMatchCheck",
> "result": "ERROR",
> "uuid": "10a1cb1c-c08c-4f05-87ea-ee1aac253692",
> "when": "20240905082820Z",
> "duration": "0.448749",
> "kw": {
> "key": "subsystemCert cert-pki-ca",
> "nickname": "subsystemCert cert-pki-ca",
> "dbdir": "/etc/pki/pki-tomcat/alias",
> "msg": "{nickname} certificate in NSS DB {dbdir} does not match
> entry in LDAP"
> }
> },
> {
> "source": "ipahealthcheck.ipa.certs",
> "check": "IPADogtagCertsMatchCheck",
> "result": "ERROR",
> "uuid": "ac4edfc6-c254-45c5-8ec6-63cbd653f010",
> "when": "20240905082820Z",
> "duration": "0.496190",
> "kw": {
> "key": "auditSigningCert cert-pki-ca",
> "nickname": "auditSigningCert cert-pki-ca",
> "dbdir": "/etc/pki/pki-tomcat/alias",
> "msg": "{nickname} certificate in NSS DB {dbdir} does not match
> entry in LDAP"
> }
> },
> {
> "source": "ipahealthcheck.ipa.certs",
> "check": "IPADogtagCertsMatchCheck",
> "result": "ERROR",
> "uuid": "edf86d59-2a47-4af8-a97c-bcaf8613897d",
> "when": "20240905082820Z",
> "duration": "0.568801",
> "kw": {
> "key": "Server-Cert cert-pki-ca",
> "nickname": "Server-Cert cert-pki-ca",
> "dbdir": "/etc/pki/pki-tomcat/alias",
> "msg": "{nickname} certificate in NSS DB {dbdir} does not match
> entry in LDAP"
> }
> },
> {
> "source": "ipahealthcheck.ipa.certs",
> "check": "IPARAAgent",
> "result": "ERROR",
> "uuid": "bfb90711-9ab8-4b86-be1c-7fce8a0c673c",
> "when": "20240905082820Z",
> "duration": "0.006922",
> "kw": {
> "key": "ldap_mismatch",
> "certfile": "/var/lib/ipa/ra-agent.pem",
> "dn": "uid=ipara,ou=people,o=ipaca",
> "msg": "RA agent certificate in {certfile} not found in LDAP
> userCertificate attribute for the entry {dn}"
> }
> },
So it's strange that the certs don't match. I imagine they all have the
same issue so let's pick just one to examine further.
ldapsearch -LLL -x -D 'cn=directory manager' -W -b
ou=certificateRepository,ou=ca,o=ipaca '(subjectName=CN=CA
Audit,O=EXAMPLE.TEST)' cn
Then certutil -L -d /etc/pki/pki-tomcat/alias -n 'auditSigningCert
cert-pki-ca' | grep 'Serial Number'
The cn should match the serial number. If not what is the LDAP serial
number? It's possible it got written to LDAP and not to the NSS db.
The remaining issues shouldn't cause the CA to not function so let's
focus on this.
rob
> {
> "source": "ipahealthcheck.ipa.dna",
> "check": "IPADNARangeCheck",
> "result": "CRITICAL",
> "uuid": "f9e6c147-ef0d-4a80-b19d-cbb3cbe4ce1d",
> "when": "20240905082822Z",
> "duration": "0.074268",
> "kw": {
> "exception": "Insufficient access: SASL(-1): generic failure:
> GSSAPI Error: No credentials were supplied, or the credentials were
> unavailable or inaccessible (Preauthentication failed)",
> "traceback": "Traceback (most recent call last):\n File
> \"/usr/lib/python3.9/site-packages/ipapython/ipaldap.py\", line 1087, in
> error_handler\n yield\n File
> \"/usr/lib/python3.9/site-packages/ipapython/ipaldap.py\", line 1258, in
> gssapi_bind\n self.conn.sasl_interactive_bind_s(\n File
> \"/usr/lib64/python3.9/site-packages/ldap/ldapobject.py\", line 270, in
> sasl_interactive_bind_s\n return
> self._ldap_call(self._l.sasl_interactive_bind_s,who,auth,RequestControlTuples(serverctrls),RequestControlTuples(clientctrls),sasl_flags)\n
> File \"/usr/lib64/python3.9/site-packages/ldap/ldapobject.py\", line
> 128, in _ldap_call\n result = func(*args,**kwargs)\nldap.LOCAL_ERROR:
> {'result': -2, 'desc': 'Local error', 'ctrls': [], 'info': 'SASL(-1):
> generic failure: GSSAPI Error: No credentials were supplied, or the
> credentials were unavailable or inaccessible (Preauthentication
> failed)'}\n\nDuring handling of the above exception, another exception
> occurred:\n\nTraceback (most recent call last):\n File
> \"/usr/lib/python3.9/site-packages/ipahealthcheck/core/core.py\", line
> 56, in run_plugin\n for result in plugin.check():\n File
> \"/usr/lib/python3.9/site-packages/ipahealthcheck/core/plugin.py\", line
> 18, in wrapper\n for result in f(*args, **kwds):\n File
> \"/usr/lib/python3.9/site-packages/ipahealthcheck/ipa/dna.py\", line 32,
> in check\n agmt = replication.ReplicationManager(api.env.realm,
> api.env.host)\n File
> \"/usr/lib/python3.9/site-packages/ipaserver/install/replication.py\",
> line 268, in __init__\n self.conn.gssapi_bind()\n File
> \"/usr/lib/python3.9/site-packages/ipapython/ipaldap.py\", line 1258, in
> gssapi_bind\n self.conn.sasl_interactive_bind_s(\n File
> \"/usr/lib64/python3.9/contextlib.py\", line 137, in __exit__\n
> self.gen.throw(typ, value, traceback)\n File
> \"/usr/lib/python3.9/site-packages/ipapython/ipaldap.py\", line 1143, in
> error_handler\n raise
> errors.ACIError(info=info)\nipalib.errors.ACIError: Insufficient access:
> SASL(-1): generic failure: GSSAPI Error: No credentials were supplied,
> or the credentials were unavailable or inaccessible (Preauthentication
> failed)\n"
> }
> },
> {
> "source": "ipahealthcheck.ipa.idns",
> "check": "IPADNSSystemRecordsCheck",
> "result": "WARNING",
> "uuid": "5c105f14-6f10-4931-8986-5effd0be9b27",
> "when": "20240905082822Z",
> "duration": "0.070505",
> "kw": {
> "msg": "Expected SRV record missing",
> "key": "_ldap._tcp.jscrambler.com
> <http://tcp.jscrambler.com>.:prod-us-freeipa.jscrambler.com
> <http://prod-us-freeipa.jscrambler.com>."
> }
> },
> {
> "source": "ipahealthcheck.ipa.idns",
> "check": "IPADNSSystemRecordsCheck",
> "result": "WARNING",
> "uuid": "50e2ccb5-7d62-4edb-a9e4-691e49c6966d",
> "when": "20240905082822Z",
> "duration": "0.077283",
> "kw": {
> "msg": "Expected SRV record missing",
> "key": "_kerberos._tcp.jscrambler.com
> <http://tcp.jscrambler.com>.:prod-us-freeipa.jscrambler.com
> <http://prod-us-freeipa.jscrambler.com>."
> }
> },
> {
> "source": "ipahealthcheck.ipa.idns",
> "check": "IPADNSSystemRecordsCheck",
> "result": "WARNING",
> "uuid": "ddbd6ec8-de76-4b1d-a3df-bbf45e1669e4",
> "when": "20240905082822Z",
> "duration": "0.098272",
> "kw": {
> "msg": "Expected SRV record missing",
> "key": "_kerberos._udp.jscrambler.com
> <http://udp.jscrambler.com>.:prod-us-freeipa.jscrambler.com
> <http://prod-us-freeipa.jscrambler.com>."
> }
> },
> {
> "source": "ipahealthcheck.ipa.idns",
> "check": "IPADNSSystemRecordsCheck",
> "result": "WARNING",
> "uuid": "be3790d6-158e-40ae-b1d2-c0b5b532be96",
> "when": "20240905082822Z",
> "duration": "0.105748",
> "kw": {
> "msg": "Expected SRV record missing",
> "key": "_kerberos-master._tcp.jscrambler.com
> <http://tcp.jscrambler.com>.:prod-us-freeipa.jscrambler.com
> <http://prod-us-freeipa.jscrambler.com>."
> }
> },
> {
> "source": "ipahealthcheck.ipa.idns",
> "check": "IPADNSSystemRecordsCheck",
> "result": "WARNING",
> "uuid": "b3fbd679-4906-4ad8-889b-923426ec9bc2",
> "when": "20240905082822Z",
> "duration": "0.121375",
> "kw": {
> "msg": "Expected SRV record missing",
> "key": "_kerberos-master._udp.jscrambler.com
> <http://udp.jscrambler.com>.:prod-us-freeipa.jscrambler.com
> <http://prod-us-freeipa.jscrambler.com>."
> }
> },
> {
> "source": "ipahealthcheck.ipa.idns",
> "check": "IPADNSSystemRecordsCheck",
> "result": "WARNING",
> "uuid": "550ed58a-309d-4756-8e20-61d8e2a7ea60",
> "when": "20240905082822Z",
> "duration": "0.129817",
> "kw": {
> "msg": "Expected SRV record missing",
> "key": "_kpasswd._tcp.jscrambler.com
> <http://tcp.jscrambler.com>.:prod-us-freeipa.jscrambler.com
> <http://prod-us-freeipa.jscrambler.com>."
> }
> },
> {
> "source": "ipahealthcheck.ipa.idns",
> "check": "IPADNSSystemRecordsCheck",
> "result": "WARNING",
> "uuid": "e22c24d3-e1d1-4db8-9640-332fb3c3d901",
> "when": "20240905082822Z",
> "duration": "0.139263",
> "kw": {
> "msg": "Expected SRV record missing",
> "key": "_kpasswd._udp.jscrambler.com
> <http://udp.jscrambler.com>.:prod-us-freeipa.jscrambler.com
> <http://prod-us-freeipa.jscrambler.com>."
> }
> },
> {
> "source": "ipahealthcheck.ipa.idns",
> "check": "IPADNSSystemRecordsCheck",
> "result": "WARNING",
> "uuid": "265a4ab3-fb2e-4f2b-b83d-0264e11a13d7",
> "when": "20240905082822Z",
> "duration": "0.146888",
> "kw": {
> "msg": "Expected URI record missing",
> "key":
> "_kerberos.jscrambler.com.:krb5srv:m:tcp:prod-us-freeipa.jscrambler.com
> <http://prod-us-freeipa.jscrambler.com>."
> }
> },
> {
> "source": "ipahealthcheck.ipa.idns",
> "check": "IPADNSSystemRecordsCheck",
> "result": "WARNING",
> "uuid": "df8e8a86-69c7-4d6d-a850-597c017ad9da",
> "when": "20240905082822Z",
> "duration": "0.146902",
> "kw": {
> "msg": "Expected URI record missing",
> "key":
> "_kerberos.jscrambler.com.:krb5srv:m:udp:prod-us-freeipa.jscrambler.com
> <http://prod-us-freeipa.jscrambler.com>."
> }
> },
> {
> "source": "ipahealthcheck.ipa.idns",
> "check": "IPADNSSystemRecordsCheck",
> "result": "WARNING",
> "uuid": "591655a6-fddc-42e9-9cdd-b310cddd912e",
> "when": "20240905082822Z",
> "duration": "0.152359",
> "kw": {
> "msg": "Expected URI record missing",
> "key":
> "_kpasswd.jscrambler.com.:krb5srv:m:tcp:prod-us-freeipa.jscrambler.com
> <http://prod-us-freeipa.jscrambler.com>."
> }
> },
> {
> "source": "ipahealthcheck.ipa.idns",
> "check": "IPADNSSystemRecordsCheck",
> "result": "WARNING",
> "uuid": "82727f16-ba22-4696-b542-1479ac8ffa3f",
> "when": "20240905082822Z",
> "duration": "0.152373",
> "kw": {
> "msg": "Expected URI record missing",
> "key":
> "_kpasswd.jscrambler.com.:krb5srv:m:udp:prod-us-freeipa.jscrambler.com
> <http://prod-us-freeipa.jscrambler.com>."
> }
> },
> {
> "source": "ipahealthcheck.ipa.idns",
> "check": "IPADNSSystemRecordsCheck",
> "result": "WARNING",
> "uuid": "1ab64a7e-0381-4fe3-a38b-301826551f6e",
> "when": "20240905082822Z",
> "duration": "0.174484",
> "kw": {
> "key": "ipa_ca_missing_prod-us-freeipa.jscrambler.com
> <http://ipa_ca_missing_prod-us-freeipa.jscrambler.com>",
> "server": "prod-us-freeipa.jscrambler.com
> <http://prod-us-freeipa.jscrambler.com>",
> "ipaddr": "172.18.0.2",
> "msg": "expected ipa-ca to contain {ipaddr} for {server}"
> }
> },
> {
> "source": "ipahealthcheck.ipa.host",
> "check": "IPAHostKeytab",
> "result": "ERROR",
> "uuid": "34eea407-73ae-46be-849c-71dddd4c01a4",
> "when": "20240905082822Z",
> "duration": "0.014935",
> "kw": {
> "msg": "Failed to obtain host TGT: Major (458752): No credentials
> were supplied, or the credentials were unavailable or inaccessible,
> Minor (2529638936): Preauthentication failed"
> }
> }
> ]
>
> On Wed, Sep 4, 2024 at 6:41 PM Rob Crittenden <[email protected]
> <mailto:[email protected]>> wrote:
>
> Duarte Petiz via FreeIPA-users wrote:
> > I will copy/paste it so:
> >
> > Hello
> >
> > [root@prod-us-freeipa backup]# getcert list | grep expires
> > expires: 2026-08-25 10:47:09 WEST
> > expires: 2026-08-25 10:46:16 WEST
> > expires: 2026-08-25 10:46:05 WEST
> > expires: 2026-08-25 10:46:12 WEST
> > expires: 2044-09-04 10:46:01 WEST
> > expires: 2026-08-25 10:46:09 WEST
> > expires: 2026-09-05 10:47:42 WEST
> >
> > Request ID '20240904094741':
> > status: MONITORING
> > stuck: no
> > key pair storage: type=FILE,location='/var/kerberos/krb5kdc/kdc.key'
> > certificate: type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
> > CA: IPA
> > issuer: CN=Certificate Authority,O=EXAMPLE.COM
> <http://EXAMPLE.COM> <http://example.com/>
> > subject: CN=prod-us-freeipa.example.com
> <http://prod-us-freeipa.example.com>
> > <http://prod-us-freeipa.example.com/>,O=EXAMPLE.COM
> <http://EXAMPLE.COM> <http://example.com/>
> > issued: 2024-09-04 10:47:42 WEST
> > expires: 2026-09-05 10:47:42 WEST
> > dns: prod-us-freeipa.example.com
> <http://prod-us-freeipa.example.com>
> <http://prod-us-freeipa.example.com/>
> > principal name: krbtgt/[email protected]
> <mailto:[email protected]>
> > <mailto:[email protected] <mailto:[email protected]>>
> > key usage:
> digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> > eku: id-kp-serverAuth,id-pkinit-KPKdc
> > profile: KDCs_PKINIT_Certs
> > pre-save command:
> > post-save command: /usr/libexec/ipa/certmonger/renew_kdc_cert
> > track: yes
> > auto-renew: yes
> >
> >
> > I think the problem is related with some certificate that is
> included in
> > the backup with "--data".
> > If I do a full restore it brokes de PKI and seems to be impossible to
> > restore.
> > The "--data" flag maybe have some certificate injected different else
> > than dn: uid=pkidbuser,ou=people,o=ipaca.
> > On the dashboard the error is: Your session has expired. Please log in
> > again.
> >
> > on the logs
> > [Wed Sep 04 10:09:13.645078 2024] [wsgi:error] [pid 254:tid 438]
> > [remote 172.21.0.1:48490 <http://172.21.0.1:48490>
> <http://172.21.0.1:48490/>] ipa: INFO: 401
> > Unauthorized: Insufficient access: Invalid credentials
> > [Wed Sep 04 10:09:13.819864 2024] [:warn] [pid 257:tid 325]
> > [client 172.21.0.1:48500 <http://172.21.0.1:48500>
> <http://172.21.0.1:48500/>] failed to set perms
> > (3140) on file (/run/ipa/ccaches/[email protected])!,
> > referer: https://prod-us-freeipa.example.com/ipa/ui
> > [Wed Sep 04 10:09:14.045869 2024] [wsgi:error] [pid 253:tid 432]
> > [remote 172.21.0.1:48504 <http://172.21.0.1:48504>
> <http://172.21.0.1:48504/>] ipa: INFO: 401
> > Unauthorized: Insufficient access: Invalid credentials
> >
> >
> > But i cannot find what is really causing the issue...
> >
> >
> > If I do a full-restore with the following procedure I can login into
> > the web portal, but the PKI goes broken.
> >
> > 1º install new freeipa 2º restore
> /var/lib/ipa/private/httpd.key 3º
> > restore /var/lib/ipa/gssproxy/http.keytab 5º docker exec -ti
> > ipa-freeipa-1 bash 6º ipa-restore /var/lib/ipa/backup/backup
> >
> > There is some way to do a new "pki" install in an existing
> > installed/restored freeipa?
> > I cannot let the PKI down, it will break the upgrades for future
> > versions. That's why I tried to restore data-only.
>
> There is no supported way to replace the CA.
>
> Can you install the {free}ipa-healthcheck package and run
> ipa-healthcheck? It is likely to return a slew of errors because the CA
> will be unresponsive but it will also check a number of things that
> could lead to a root cause.
>
> rob
>
> >
> >
> > On Wed, Sep 4, 2024 at 1:35 PM flo--- via FreeIPA-users
> > <[email protected]
> <mailto:[email protected]>
> > <mailto:[email protected]
> <mailto:[email protected]>>> wrote:
> >
> > The content of this message was lost. It was probably
> cross-posted to
> > multiple lists and previously handled on another list.
> > --
> > _______________________________________________
> > FreeIPA-users mailing list --
> [email protected]
> <mailto:[email protected]>
> > <mailto:[email protected]
> <mailto:[email protected]>>
> > To unsubscribe send an email to
> > [email protected]
> <mailto:[email protected]>
> > <mailto:[email protected]
> <mailto:[email protected]>>
> > Fedora Code of Conduct:
> > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines:
> https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
> >
>
> https://lists.fedorahosted.org/archives/list/[email protected]
> > Do not reply to spam, report it:
> > https://pagure.io/fedora-infrastructure/new_issue
> >
> >
> >
> > --
> > /Kind Regards/
> >
> > *Duarte Petiz*
> > *DevOps Team Lead *| jscrambler.com <http://jscrambler.com>
> <http://jscrambler.com/>
> > *
> > *
> > *
> > *
> >
> >
>
>
>
> --
> /Kind Regards/
>
> *Duarte Petiz*
> *DevOps Team Lead *| jscrambler.com <http://jscrambler.com/>
> *
> *
> *
> *
>
--
_______________________________________________
FreeIPA-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedorahosted.org/archives/list/[email protected]
Do not reply to spam, report it:
https://pagure.io/fedora-infrastructure/new_issue
