URL: https://github.com/freeipa/freeipa/pull/5045 Author: rcritten Title: #5045: Improve performance of ipa-server-guard Action: opened
PR body: """ * Drop support for python 2 * Only import traceback and syslog when needed * Only import ipaserver.install.certs when the lock is needed * Only import ipautil when run is needed For the unsupported operations case this improves performance by 95% For the supported operations that don't require a lock the improvement is about 50%. For the supported operations that require a lock the improvement is about 20% When configuring a CA certmonger calls its helper with the following operations: IDENTIFY FETCH-ROOTS GET-SUPPORTED-TEMPLATES GET-DEFAULT-TEMPLATE GET-NEW-REQUEST-REQUIREMENTS GET-RENEW-REQUEST-REQUIREMENTS FETCH-SCEP-CA-CAPS FETCH-SCEP-CA-CERTS Only IDENTIFY, FETCH-ROOTS and GET-NEW-REQUEST-REQUIREMENTS are supported by ipa-submit, along with the request options SUBMIT and POLL. Which means every time the IPA CA in certmonger is updated eight calls to ipa-server-guard are made so the savings are cumulative. The savings when executing these eight operations is a 73% decrease (.7 sec vs 2.5 sec). https://pagure.io/freeipa/issue/8425 """ To pull the PR as Git branch: git remote add ghfreeipa https://github.com/freeipa/freeipa git fetch ghfreeipa pull/5045/head:pr5045 git checkout pr5045
From 9ecb723bd7a7f1bc21a3e54fec707107202cadb8 Mon Sep 17 00:00:00 2001 From: Rob Crittenden <rcrit...@redhat.com> Date: Tue, 18 Aug 2020 13:34:06 -0400 Subject: [PATCH] Improve performance of ipa-server-guard * Drop support for python 2 * Only import traceback and syslog when needed * Only import ipaserver.install.certs when the lock is needed * Only import ipautil when run is needed For the unsupported operations case this improves performance by 95% For the supported operations that don't require a lock the improvement is about 50%. For the supported operations that require a lock the improvement is about 20% When configuring a CA certmonger calls its helper with the following operations: IDENTIFY FETCH-ROOTS GET-SUPPORTED-TEMPLATES GET-DEFAULT-TEMPLATE GET-NEW-REQUEST-REQUIREMENTS GET-RENEW-REQUEST-REQUIREMENTS FETCH-SCEP-CA-CAPS FETCH-SCEP-CA-CERTS Only IDENTIFY, FETCH-ROOTS and GET-NEW-REQUEST-REQUIREMENTS are supported by ipa-submit, along with the request options SUBMIT and POLL. Which means every time the IPA CA in certmonger is updated eight calls to ipa-server-guard are made so the savings are cumulative. The savings when executing these eight operations is a 73% decrease (.7 sec vs 2.5 sec). https://pagure.io/freeipa/issue/8425 --- install/certmonger/ipa-server-guard.in | 43 +++++++++++++------------- 1 file changed, 21 insertions(+), 22 deletions(-) diff --git a/install/certmonger/ipa-server-guard.in b/install/certmonger/ipa-server-guard.in index 265f78adfb..499088297e 100644 --- a/install/certmonger/ipa-server-guard.in +++ b/install/certmonger/ipa-server-guard.in @@ -19,27 +19,31 @@ # You should have received a copy of the GNU General Public License # along with this program. If not, see <http://www.gnu.org/licenses/>. -from __future__ import print_function - import os # Prevent garbage from readline on standard output # (see https://fedorahosted.org/freeipa/ticket/4064) if not os.isatty(1): os.environ['TERM'] = 'dumb' import sys -import syslog -import traceback - -import six - -from ipapython import ipautil -from ipaserver.install import certs # Return codes. Names of the constants are taken from # https://git.fedorahosted.org/cgit/certmonger.git/tree/src/submit-e.h OPERATION_NOT_SUPPORTED_BY_HELPER = 6 +def run_operation(cmd): + from ipapython import ipautil + + result = ipautil.run(cmd, raiseonerr=False, env=os.environ) + # Write bytes directly + sys.stdout.buffer.write(result.raw_output) #pylint: disable=no-member + sys.stderr.buffer.write(result.raw_error_output) #pylint: disable=no-member + sys.stdout.flush() + sys.stderr.flush() + + return result.returncode + + def main(): if len(sys.argv) < 2: raise RuntimeError("Not enough arguments") @@ -53,24 +57,19 @@ def main(): 'POLL'): return OPERATION_NOT_SUPPORTED_BY_HELPER - with certs.renewal_lock: - result = ipautil.run(sys.argv[1:], raiseonerr=False, env=os.environ) - if six.PY2: - sys.stdout.write(result.raw_output) - sys.stderr.write(result.raw_error_output) - else: - # Write bytes directly - sys.stdout.buffer.write(result.raw_output) #pylint: disable=no-member - sys.stderr.buffer.write(result.raw_error_output) #pylint: disable=no-member - sys.stdout.flush() - sys.stderr.flush() - - return result.returncode + if operation in ('SUBMIT', 'POLL', 'FETCH-ROOTS'): + from ipaserver.install import certs + with certs.renewal_lock: + return run_operation(sys.argv[1:]) + else: + return run_operation(sys.argv[1:]) try: sys.exit(main()) except Exception as e: + import traceback + import syslog syslog.syslog(syslog.LOG_ERR, traceback.format_exc()) print("Internal error") sys.exit(3)
_______________________________________________ FreeIPA-devel mailing list -- freeipa-devel@lists.fedorahosted.org To unsubscribe send an email to freeipa-devel-le...@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-devel@lists.fedorahosted.org
