(kudu) branch branch-1.18.x updated: [security] make sure OpenSSL initialized as expected

[alexey]

This is an automated email from the ASF dual-hosted git repository.

alexey pushed a commit to branch branch-1.18.x
in repository https://gitbox.apache.org/repos/asf/kudu.git


The following commit(s) were added to refs/heads/branch-1.18.x by this push:
     new 72fdc7f8c [security] make sure OpenSSL initialized as expected
72fdc7f8c is described below

commit 72fdc7f8c89ae03e195d9f5d11186ce9604f39af
Author: Alexey Serbin <ale...@apache.org>
AuthorDate: Thu Aug 7 12:40:21 2025 -0700

    [security] make sure OpenSSL initialized as expected
    
    Per OpenSSL's documentation [1][2], numerous internal OpenSSL functions
    call OPENSSL_init_{crypto,ssl}().  Therefore, in order to perform
    nondefault initialisation which Kudu does since addressing KUDU-3635,
    OPENSSL_init_{crypto,ssl}() MUST be called by application code prior
    to any other OpenSSL function calls.  And this is also applicable to
    ERR_peek_error() which calls OPENSSL_init_crypt with the
    OPENSSL_INIT_BASE_ONLY option, at least in OpenSSL 1.1.1.  Meanwhile,
    ERR_peek_error() is called in the constructor and the destructor
    of the underlying object in the SCOPED_OPENSSL_NO_PENDING_ERRORS macro.
    
    Even if Kudu adds a non-default OPENSSL_INIT_NO_ATEXIT flag only when
    initializing the OpenSSL library early in the process start-up phase,
    it makes sense to address this to avoid unexpected surprises if any
    extra flag is added.  Also, placing SCOPED_OPENSSL_NO_PENDING_ERRORS
    before InitializeOpenSSL() was an apparent typo, so it needs to be fixed
    anyway.
    
    This is a follow-up 5f1ca4f3948a61b22946255e4ada895c77bc6adf.
    
    [1] https://docs.openssl.org/1.1.1/man3/OPENSSL_init_crypto/
    [2] https://docs.openssl.org/1.1.1/man3/OPENSSL_init_ssl/
    
    Change-Id: If56b5d23e1f974aa2ab5677458b125cb011a36dc
    Reviewed-on: http://gerrit.cloudera.org:8080/23268
    Tested-by: Alexey Serbin <ale...@apache.org>
    Reviewed-by: Abhishek Chennaka <achenn...@cloudera.com>
    (cherry picked from commit b92f16d1c86a753c597b46c7575bfa6a1479726a)
    Reviewed-on: http://gerrit.cloudera.org:8080/23272
    Reviewed-by: Alexey Serbin <ale...@apache.org>
---
 src/kudu/security/ca/cert_management.cc | 4 ++--
 src/kudu/security/crypto.cc             | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/src/kudu/security/ca/cert_management.cc 
b/src/kudu/security/ca/cert_management.cc
index 175fcee42..2a9e8eff8 100644
--- a/src/kudu/security/ca/cert_management.cc
+++ b/src/kudu/security/ca/cert_management.cc
@@ -287,9 +287,9 @@ CertSigner::CertSigner(const Cert* ca_cert,
 }
 
 Status CertSigner::Sign(const CertSignRequest& req, Cert* ret) const {
-  SCOPED_OPENSSL_NO_PENDING_ERRORS;
-  InitializeOpenSSL();
   CHECK(ret);
+  InitializeOpenSSL();
+  SCOPED_OPENSSL_NO_PENDING_ERRORS;
 
   // If we are not self-signing, then make sure that the provided CA
   // cert and key match each other. Technically this would be programmer
diff --git a/src/kudu/security/crypto.cc b/src/kudu/security/crypto.cc
index ea9fc22f3..820ba0b96 100644
--- a/src/kudu/security/crypto.cc
+++ b/src/kudu/security/crypto.cc
@@ -301,9 +301,9 @@ Status PrivateKey::MakeSignature(DigestType digest,
 }
 
 Status GeneratePrivateKey(int num_bits, PrivateKey* ret) {
-  SCOPED_OPENSSL_NO_PENDING_ERRORS;
   CHECK(ret);
   InitializeOpenSSL();
+  SCOPED_OPENSSL_NO_PENDING_ERRORS;
   auto key = ssl_make_unique(EVP_PKEY_new());
   {
     auto bn = ssl_make_unique(BN_new());