All,

I did a routine server updates last night on my IPA server. After the reboot I 
first noticed the DNS was not resolving and the ipa.service failed. The 
ipa.service failed to start so I ran the following:


# ipactl start
IPA version error: data needs to be upgraded (expected version 
'4.6.6-11.el7.centos', current version '4.6.5-11.el7.centos.4')
Automatically running upgrade, for details see /var/log/ipaupgrade.log
Be patient, this may take a few minutes.
Automatic upgrade failed: Update complete
Upgrading the configuration of the IPA services
[Verifying that root certificate is published]
[Migrate CRL publish directory]
CRL tree already moved
[Verifying that CA proxy configuration is correct]
[Verifying that KDC configuration is using ipa-kdb backend]
[Fix DS schema file syntax]
Syntax already fixed
[Removing RA cert from DS NSS database]
RA cert already removed
[Enable sidgen and extdom plugins by default]
[Updating HTTPD service IPA configuration]
[Updating HTTPD service IPA WSGI configuration]
Nothing to do for configure_httpd_wsgi_conf
[Updating mod_nss protocol versions]
Protocol versions already updated
[Updating mod_nss cipher suite]
[Updating mod_nss enabling OCSP]
[Fixing trust flags in /etc/httpd/alias]
Trust flags already processed
[Moving HTTPD service keytab to gssproxy]
[Removing self-signed CA]
[Removing Dogtag 9 CA]
[Checking for deprecated KDC configuration files]
[Checking for deprecated backups of Samba configuration files]
[Remove FILE: prefix from 'dedicated keytab file' in Samba configuration]
[Update 'max smbd processes' in Samba configuration to prevent unlimited 
SMBLoris attack amplification]
[Add missing CA DNS records]
IPA CA DNS records already processed
[Removing deprecated DNS configuration options]
[Ensuring minimal number of connections]
[Updating GSSAPI configuration in DNS]
[Updating pid-file configuration in DNS]
[Checking global forwarding policy in named.conf to avoid conflicts with 
automatic empty zones]
Changes to named.conf have been made, restart named
[Upgrading CA schema]
CA schema update complete (no changes)
[Verifying that CA audit signing cert has 2 year validity]
[Update certmonger certificate renewal configuration]
Certmonger certificate renewal configuration already up-to-date
[Enable PKIX certificate path discovery and validation]
PKIX already enabled
[Authorizing RA Agent to modify profiles]
[Authorizing RA Agent to manage lightweight CAs]
[Ensuring Lightweight CAs container exists in Dogtag database]
[Adding default OCSP URI configuration]
[Ensuring CA is using LDAPProfileSubsystem]
[Migrating certificate profiles to LDAP]
IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and run command 
ipa-server-upgrade manually.
Unexpected error - see /var/log/ipaupgrade.log for details:
NetworkError: cannot connect to 
'https://FAKE-IPA-HOST.FAKE-IPA-DOMAIN.LAN:8443/ca/rest/account/login': [SSL: 
CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:618)
The ipa-server-upgrade command failed. See /var/log/ipaupgrade.log for more 
information

See the upgrade log for more details and/or run /usr/sbin/ipa-server-upgrade 
again
Aborting ipactl


The end of the /var/log/ipaupgrade.log file:

2020-06-29T22:43:38Z DEBUG stderr=
2020-06-29T22:43:38Z DEBUG Loading Index file from 
'/var/lib/ipa/sysrestore/sysrestore.index'
2020-06-29T22:43:38Z DEBUG Starting external process
2020-06-29T22:43:38Z DEBUG args=/usr/bin/certutil -d 
dbm:/etc/pki/pki-tomcat/alias -L -f /etc/pki/pki-tomcat/alias/pwdfile.txt
2020-06-29T22:43:38Z DEBUG Process finished, return code=0
2020-06-29T22:43:38Z DEBUG stdout=
Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI

caSigningCert cert-pki-ca                                    CTu,Cu,Cu
subsystemCert cert-pki-ca                                    u,u,u
Server-Cert cert-pki-ca                                      u,u,u
ocspSigningCert cert-pki-ca                                  u,u,u
auditSigningCert cert-pki-ca                                 u,u,Pu

2020-06-29T22:43:38Z DEBUG stderr=
2020-06-29T22:43:38Z INFO Certmonger certificate renewal configuration already 
up-to-date
2020-06-29T22:43:38Z INFO [Enable PKIX certificate path discovery and 
validation]
2020-06-29T22:43:38Z DEBUG Loading StateFile from 
'/var/lib/ipa/sysupgrade/sysupgrade.state'
2020-06-29T22:43:38Z INFO PKIX already enabled
2020-06-29T22:43:38Z INFO [Authorizing RA Agent to modify profiles]
2020-06-29T22:43:38Z INFO [Authorizing RA Agent to manage lightweight CAs]
2020-06-29T22:43:38Z INFO [Ensuring Lightweight CAs container exists in Dogtag 
database]
2020-06-29T22:43:38Z DEBUG Created connection context.ldap2_140346851657552
2020-06-29T22:43:38Z DEBUG flushing 
ldapi://%2fvar%2frun%2fslapd-SEQUOIARC-LAN.socket from SchemaCache
2020-06-29T22:43:38Z DEBUG retrieving schema for SchemaCache 
url=ldapi://%2fvar%2frun%2fslapd-SEQUOIARC-LAN.socket 
conn=<ldap.ldapobject.SimpleLDAPObject instance at 0x7fa50c3e8e60>
2020-06-29T22:43:39Z DEBUG Destroyed connection context.ldap2_140346851657552
2020-06-29T22:43:39Z INFO [Adding default OCSP URI configuration]
2020-06-29T22:43:39Z INFO [Ensuring CA is using LDAPProfileSubsystem]
2020-06-29T22:43:39Z INFO [Migrating certificate profiles to LDAP]
2020-06-29T22:43:39Z DEBUG Created connection context.ldap2_140346825804304
2020-06-29T22:43:39Z DEBUG flushing 
ldapi://%2fvar%2frun%2fslapd-SEQUOIARC-LAN.socket from SchemaCache
2020-06-29T22:43:39Z DEBUG retrieving schema for SchemaCache 
url=ldapi://%2fvar%2frun%2fslapd-SEQUOIARC-LAN.socket 
conn=<ldap.ldapobject.SimpleLDAPObject instance at 0x7fa50ac19b90>
2020-06-29T22:43:39Z DEBUG Destroyed connection context.ldap2_140346825804304
2020-06-29T22:43:39Z DEBUG request GET 
https://FAKE-IPA-HOST.FAKE-IPA-DOMAIN.LAN:8443/ca/rest/account/login
2020-06-29T22:43:39Z DEBUG request body ''
2020-06-29T22:43:39Z DEBUG httplib request failed:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ipapython/dogtag.py", line 220, in 
_httplib_request
    conn.request(method, path, body=request_body, headers=headers)
  File "/usr/lib64/python2.7/httplib.py", line 1056, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib64/python2.7/httplib.py", line 1090, in _send_request
    self.endheaders(body)
  File "/usr/lib64/python2.7/httplib.py", line 1052, in endheaders
    self._send_output(message_body)
  File "/usr/lib64/python2.7/httplib.py", line 890, in _send_output
    self.send(msg)
  File "/usr/lib64/python2.7/httplib.py", line 852, in send
    self.connect()
  File "/usr/lib64/python2.7/httplib.py", line 1275, in connect
    server_hostname=sni_hostname)
  File "/usr/lib64/python2.7/ssl.py", line 348, in wrap_socket
    _context=self)
  File "/usr/lib64/python2.7/ssl.py", line 609, in __init__
    self.do_handshake()
  File "/usr/lib64/python2.7/ssl.py", line 831, in do_handshake
    self._sslobj.do_handshake()
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed 
(_ssl.c:618)
2020-06-29T22:43:39Z ERROR IPA server upgrade failed: Inspect 
/var/log/ipaupgrade.log and run command ipa-server-upgrade manually.
2020-06-29T22:43:39Z DEBUG   File 
"/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 178, in execute
    return_value = self.run()
  File 
"/usr/lib/python2.7/site-packages/ipaserver/install/ipa_server_upgrade.py", 
line 54, in run
    server.upgrade()
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/upgrade.py", 
line 2166, in upgrade
    upgrade_configuration()
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/upgrade.py", 
line 2038, in upgrade_configuration
    ca_enable_ldap_profile_subsystem(ca)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/upgrade.py", 
line 425, in ca_enable_ldap_profile_subsystem
    cainstance.migrate_profiles_to_ldap()
  File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 
2027, in migrate_profiles_to_ldap
    _create_dogtag_profile(profile_id, profile_data, overwrite=False)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 
2033, in _create_dogtag_profile
    with api.Backend.ra_certprofile as profile_api:
  File "/usr/lib/python2.7/site-packages/ipaserver/plugins/dogtag.py", line 
1311, in __enter__
    method='GET'
  File "/usr/lib/python2.7/site-packages/ipapython/dogtag.py", line 167, in 
https_request
    method=method, headers=headers)
  File "/usr/lib/python2.7/site-packages/ipapython/dogtag.py", line 229, in 
_httplib_request
    raise NetworkError(uri=uri, error=str(e))

2020-06-29T22:43:39Z DEBUG The ipa-server-upgrade command failed, exception: 
NetworkError: cannot connect to 
'https://FAKE-IPA-HOST.FAKE-IPA-DOMAIN.LAN:8443/ca/rest/account/login': [SSL: 
CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:618)
2020-06-29T22:43:39Z ERROR Unexpected error - see /var/log/ipaupgrade.log for 
details:
NetworkError: cannot connect to 
'https://FAKE-IPA-HOST.FAKE-IPA-DOMAIN.LAN:8443/ca/rest/account/login': [SSL: 
CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:618)
2020-06-29T22:43:39Z ERROR The ipa-server-upgrade command failed. See 
/var/log/ipaupgrade.log for more information


What should be my next debug steps?

Thanks in advance,
-ms

_______________________________________________
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org

Reply via email to