URL: https://github.com/freeipa/freeipa/pull/5667 Author: rcritten Title: #5667: [Backport][ipa-4-9] Don't start the CA during healthcheck grace period check so uninstall succeeds Action: opened
PR body: """ This PR was opened manually because PR #5573 was pushed to master and backport to ipa-4-9 is required. A manual backport was needed because there was a merge conflict in updating the test_integration/test_ipahealthcheck.py::TestIpaHealthCheck timeouts Adding needs_review to double-check I did the merge correctly. """ To pull the PR as Git branch: git remote add ghfreeipa https://github.com/freeipa/freeipa git fetch ghfreeipa pull/5667/head:pr5667 git checkout pr5667
From d5e05754d2fbaa53539524e5494e8b9374dd71a7 Mon Sep 17 00:00:00 2001 From: Rob Crittenden <[email protected]> Date: Thu, 18 Mar 2021 11:04:27 -0400 Subject: [PATCH 1/2] Uninstall without starting the CA in cert expiration test Some certificates may have started renewal so returning to present time can bind the server up with trying to renew. certmonger fires off helpers when it's time to renew certificates. This scenario puts the time within the renewal window. If certmonger notices while the test is running it will kick off renewal for all 12 certificates. A lock is used to serialize things. The CA was shut down prior to changing time so there is no chance of issuing new certs. A fixture was used to ensure that things restarted when the test was over. This was for chronyd and the CA. By restarting the CA we allow the chance that it will be able to do some work, versus returning a connection error and letting certmonger just error out (CA_UNREACHABLE). During uninstallation we call certmonger remove_request over DBus (the equivalent to stop-tracking). As part of this certmonger waits for any child (helper) processes to go away. This used to do it via SIGKILL but that caused other problems so it was changed to waitpid(). We know that it isn't going to return for a while because the CA isn't up. DBus has a hardcoded 25 second timeout. So we're guaranteed to get a DBus timeout. We *could* try to play with it and change the timeout, or retry a bunch of times, but it isn't worth the hassle. This is a contrived scenario that uninstalls immediately after tweaking time forward. So rather than trying to make this succesful, uninstall at the future time with the CA stopped so that helpers won't be hanging around and certmonger can remove the certs. This is the last test so also the last time we need the replica so to avoid replication bogging things down remove that prior to executing the test. It's one less moving part during the uninstall phase. https://pagure.io/freeipa/issue/8506 Signed-off-by: Rob Crittenden <[email protected]> Reviewed-By: Florence Blanc-Renaud <[email protected]> --- .../test_integration/test_ipahealthcheck.py | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/ipatests/test_integration/test_ipahealthcheck.py b/ipatests/test_integration/test_ipahealthcheck.py index ec066a769af..90bde5b585b 100644 --- a/ipatests/test_integration/test_ipahealthcheck.py +++ b/ipatests/test_integration/test_ipahealthcheck.py @@ -1336,6 +1336,10 @@ def execute_expiring_check(check): else: assert check["kw"]["days"] == 10 + # Remove the replica now since it will be out of sync with the + # updated certificates and replication will break. + tasks.uninstall_replica(self.master, self.replicas[0]) + # Store the current date to restore at the end of the test now = datetime.utcnow() now_str = datetime.strftime(now, "%Y-%m-%d %H:%M:%S Z") @@ -1345,8 +1349,14 @@ def execute_expiring_check(check): cert = x509.load_certificate_list(certfile) cert_expiry = cert[0].not_valid_after - for service in ('chronyd', 'pki_tomcatd',): - restart_service(self.master, service) + # Stop chronyd so it doesn't freak out with time so off + restart_service(self.master, 'chronyd') + + # Stop pki_tomcatd so certs are not renewable. Don't restart + # it because by the time the test is done the server is gone. + self.master.run_command( + ["systemctl", "stop", "pki-tomcatd@pki-tomcat"] + ) try: # move date to the grace period @@ -1361,6 +1371,10 @@ def execute_expiring_check(check): execute_nsscheck_cert_expiring(check) finally: + # Uninstall the master here so that the certs don't try + # to renew after the CA is running again. + tasks.uninstall_master(self.master) + # After restarting chronyd, the date may need some time to get # synced. Help chrony by resetting the date self.master.run_command(['date', '-s', now_str]) From 52091e2858ac3542ac6ab814e02e2e0268d7414c Mon Sep 17 00:00:00 2001 From: Rob Crittenden <[email protected]> Date: Wed, 24 Mar 2021 10:24:49 -0400 Subject: [PATCH 2/2] Increase timeout for TestIpaHealthCheck to 5400s During development of a fix to workaround certmonger effectivly hanging server uninstallation the test was re-worked to force uninstall during the test execution itself. https://pagure.io/freeipa/issue/8506 Signed-off-by: Rob Crittenden <[email protected]> Reviewed-By: Florence Blanc-Renaud <[email protected]> --- ipatests/prci_definitions/nightly_ipa-4-9_latest.yaml | 2 +- ipatests/prci_definitions/nightly_ipa-4-9_latest_selinux.yaml | 2 +- ipatests/prci_definitions/nightly_ipa-4-9_previous.yaml | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/ipatests/prci_definitions/nightly_ipa-4-9_latest.yaml b/ipatests/prci_definitions/nightly_ipa-4-9_latest.yaml index 6bfa81acc9a..f273df16b82 100644 --- a/ipatests/prci_definitions/nightly_ipa-4-9_latest.yaml +++ b/ipatests/prci_definitions/nightly_ipa-4-9_latest.yaml @@ -1363,7 +1363,7 @@ jobs: build_url: '{fedora-latest-ipa-4-9/build_url}' test_suite: test_integration/test_ipahealthcheck.py::TestIpaHealthCheck template: *ci-ipa-4-9-latest - timeout: 3600 + timeout: 5400 topology: *master_1repl fedora-latest-ipa-4-9/test_ipahealthcheck_nodns_extca_file: diff --git a/ipatests/prci_definitions/nightly_ipa-4-9_latest_selinux.yaml b/ipatests/prci_definitions/nightly_ipa-4-9_latest_selinux.yaml index facc7453e17..8ceb5eab792 100644 --- a/ipatests/prci_definitions/nightly_ipa-4-9_latest_selinux.yaml +++ b/ipatests/prci_definitions/nightly_ipa-4-9_latest_selinux.yaml @@ -1470,7 +1470,7 @@ jobs: selinux_enforcing: True test_suite: test_integration/test_ipahealthcheck.py::TestIpaHealthCheck template: *ci-ipa-4-9-latest - timeout: 3600 + timeout: 5400 topology: *master_1repl fedora-latest-ipa-4-9/test_ipahealthcheck_nodns_extca_file: diff --git a/ipatests/prci_definitions/nightly_ipa-4-9_previous.yaml b/ipatests/prci_definitions/nightly_ipa-4-9_previous.yaml index 93e7d61ae46..5d3097d414c 100644 --- a/ipatests/prci_definitions/nightly_ipa-4-9_previous.yaml +++ b/ipatests/prci_definitions/nightly_ipa-4-9_previous.yaml @@ -1363,7 +1363,7 @@ jobs: build_url: '{fedora-previous-ipa-4-9/build_url}' test_suite: test_integration/test_ipahealthcheck.py::TestIpaHealthCheck template: *ci-ipa-4-9-previous - timeout: 3600 + timeout: 5400 topology: *master_1repl fedora-previous-ipa-4-9/test_ipahealthcheck_nodns_extca_file:
_______________________________________________ FreeIPA-devel mailing list -- [email protected] To unsubscribe send an email to [email protected] Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/[email protected] Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
