On 08/15/2018 01:20 PM, Jokinen Eemeli via FreeIPA-users wrote:
Hi!
Anybody can help me with this one?
Summary:
2 node freeipa server cluster, node 2 was initially down for other reasons and
node 1 (renewal master) had forgot to update own certificates which resulted
faulty cluster. With help from mailing list we got the node 1 back online and
it's working great! Now I'm trying to get node2 back to working order in
cluster but it won't update the certificates even when trying the timejump.
Seems like it tries to renew certificates locally although somehow I tought
that it should renew the certificates from node 1...?
Hi,
you probably have a combination of multiple issues on your second node.
The ipa-server-upgrade failure may leave your instance in a wrong state,
where dse.ldif has disabled the ports for 389-ds (see BZ
https://bugzilla.redhat.com/show_bug.cgi?id=1569011 or pagure ticket
https://pagure.io/freeipa/issue/7534).
During the upgrade, dse.ldif is edited in order to temporarily disable
the LDAP ports (to prevent ldap modifications during the upgrade).
Sometimes, if the upgrade fails, dse.ldif is not restored and the ports
remain disabled. You will have to stop the ldap server, manually edit
dse.ldif (in /etc/dirsrv/slapd-DOMxxx) and set:
nsslapd-port: 389
nsslapd-security: on
then restart the LDAP server.
For the cert renewal, your procedure is the valid one. The kerberos
error is probably linked to 389-ds not being accessible.
HTH,
flo
Eemeli
-----Original Message-----
From: Jokinen Eemeli
Sent: keskiviikko 4. heinäkuuta 2018 16.08
To: 'Rob Crittenden' <[email protected]>; FreeIPA users list
<[email protected]>; Florence Blanc-Renaud <[email protected]>
Subject: RE: [Freeipa-users] Re: Problems after IPA upgrade: ipa-server-upgrade
doesn't complete, pki-tomcatd won't start
Hi!
I reply to this since there's some data in this message queue already related
to my problem:
I had 2 ipa node cluster, where the second node had been offline for some time
and at some point we had an error while trying to reboot node1 which was a
Renewal Master. The issue was that some certs had expired and after a bit of
special work we got the node1 back on track. I can spot three problems and I
can't (again) figure out which one is the cause and which one I should repair
first.
Now I got assigned the case to get the node2 back on track also. It had some certificates expired
(obviously) so I did a small time jump and some of the certs were renewed. However not all of them
were upgraded. "getcert list" reports 3 certs "CA Unreachable", other 3 certs
seem fine.
--
getcert list |grep -A 10 "CA_UNREACH"
status: CA_UNREACHABLE
ca-error: Internal error
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=<<REALM>>
subject: CN=OCSP Subsystem,O=<<REALM>>
expires: 2018-03-21 09:42:04 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
eku: id-kp-OCSPSigning
--
status: CA_UNREACHABLE
ca-error: Internal error
stuck: no
key pair storage:
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate:
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS
Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=<<REALM>>
subject: CN=IPA RA,O=<<REALM>>
expires: 2018-03-21 09:42:29 UTC
key usage:
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
--
status: CA_UNREACHABLE
ca-error: Error 7 connecting to
http://<<ipa2.fqdn>>:8080/ca/ee/ca/profileSubmit: Couldn't connect to server.
stuck: no
key pair storage:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
cert-pki-ca',token='NSS Certificate DB',pin set
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-renew-agent
issuer: CN=Certificate Authority,O=<<REALM>>
subject: CN=<<ipa2.fqdn>>,O=<<REALM>>
expires: 2018-06-27 07:01:38 UTC
key usage:
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
--
Seems like "Server-Cert cert-pki-ca" is trying to renew on itself (node2) but
shouldn't node1 be the renewal master? Restarting httpd, certmonger and pki-tomcat don't
seem to help, time traveling helped on other certs but not on these.
Directory service seems to work if I start it manually but ipa-server-upgrade fails on
directory server not starting with "No ports specified" so something wrong with
it or is it the certificates?
--
ipa-server-upgrade
Upgrading IPA:. Estimated time: 1 minute 30 seconds
[1/10]: stopping directory server
[2/10]: saving configuration
[3/10]: disabling listeners
[4/10]: enabling DS global lock
[5/10]: starting directory server
--
<<ipa2.fqdn>> ns-slapd[24503]: [04/Jul/2018:13:43:48.829927675 +0300] - EMERG -
main - Fatal Error---No ports specified. Exiting now.
--
Also certmonger has issues:
--
dogtag-ipa-ca-renew-agent-submit[1892]: Traceback (most recent call last):
File
"/usr/libexec/certmonger/dogtag-ipa-ca-renew-agent-submit", line 541, in
<module>
sys.exit(main())
File "/usr/libexec/certmonger/dogtag-ipa-ca-renew-agent-submit", line 515, in
main
kinit_keytab(principal, paths.KRB5_KEYTAB, ccache_filename)
File "/usr/lib/python2.7/site-packages/ipalib/install/kinit.py", line 43, in
kinit_keytab
cred = gssapi.Credentials(name=name, store=store, usage='initiate')
File "/usr/lib64/python2.7/site-packages/gssapi/creds.py", line 64, in __new__
store=store)
File "/usr/lib64/python2.7/site-packages/gssapi/creds.py", line 148, in acquire
usage)
File "ext_cred_store.pyx", line 182, in
gssapi.raw.ext_cred_store.acquire_cred_from (gssapi/raw/ext_cred_store.c:1732)
GSSError: Major (851968): Unspecified GSS failure. Minor code may provide more
information, Minor (2529639068): Cannot contact any KDC for realm '<<REALM>>'
--
but KDCs should be able to be resolved even from ipa node2
--
nslookup -type=srv _kerberos._tcp.<<REALM>>
Server: <<ipa1.ip>>
Address: <<ipa1.ip>>#53
_kerberos._tcp.<<REALM>> service = 0 100 88 <<ipa1.fqdn>>.
_kerberos._tcp.<<REALM>> service = 0 100 88 <<ipa2.fqdn>>.
--
For testing purposes I turned off firewall on ipa node1
Eemeli
_______________________________________________
FreeIPA-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/[email protected]/message/XOUL2VQ26BKQHNY2XB3CDSJRKYQCHJ3X/
_______________________________________________
FreeIPA-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/[email protected]/message/7RIEZQSILKIAJUSZXTGU4EN52LH345LB/
