This is an automated email from the ASF dual-hosted git repository.

zwoop pushed a commit to branch 9.2.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git


The following commit(s) were added to refs/heads/9.2.x by this push:
     new 79d05777b3 9.2.x: Fix an error on SSL config reload (plus some 
cleanup). (#9334) (#9647)
79d05777b3 is described below

commit 79d05777b3111abd0b911866e532cefd12bb262c
Author: Walt Karas <[email protected]>
AuthorDate: Tue May 30 11:19:23 2023 -0500

    9.2.x: Fix an error on SSL config reload (plus some cleanup). (#9334) 
(#9647)
    
    * Fix an error on SSL config reload (plus some cleanup). (#9334)
    
    * Fix an error on SSL config reload (plus some cleanup).
    
    This seems to have eliminated some ERROR diags we were seeing in Yahoo Prod 
when
    doing config reloads.
    
    The SSLSecret public functions no longer return pointers into the 
unorded_map of
    secrets, they return a copy of the secret data. This seemed thread unsafe. A
    periodic poll running in the background can update the secret data for an 
entry
    for a secret name in the map.
    
    To avoid exporting pointers, I had to change the prototype of 
TSSslSecretGet().
    Hopefully there are no existing plugins that are already using this TS API 
function,
    so breaking this rule will be moot.
    
    * YTSATS-4067: Fix deadlock with secret_map_mutex (#740)
    
    1. getOrLoadSecret grabbed the secret_map_mutex and called loadSecret.
    2. loadSecret dispatched to Continations that registered for the
       TS_EVENT_SSL_SECRET event. This would try to grab the Continuation's
       lock.
    3. In the meantime, those Continuations could call setSecret which would
       try to grab the secret_map_mutex. If this Continuation did this while
       holding the lock that step 2 is waiting upon, then there will be a
       deadlock between the Continuation lock and the secret_map_mutex
       between the two threads.
    
    This patch avoids the deadlock by releasing the secret_map_mutex lock
    before calling loadSecret. It also updates the secret_map when loading
    secrets from a file in loadSecret.
    
    ---------
    
    Co-authored-by: Brian Neradt <[email protected]>
    (cherry picked from commit 0c2488c7d0432698272b1433817ee62d0c263ab4)
    
    Conflicts:
    src/traffic_server/InkAPI.cc
    
    * Remove TSSslSecretXxx TS API functions.
    
    * Restore version of Au test tls_keepalive not needing 
ssl_secret_load_test.so.
---
 .../api/functions/TSLifecycleHookAdd.en.rst        |  12 +-
 .../api/functions/TSSslSecret.en.rst               |  77 ------
 doc/release-notes/whats-new.en.rst                 |   3 -
 include/ts/apidefs.h.in                            |   4 +-
 include/ts/ts.h                                    |   7 -
 iocore/cache/test/stub.cc                          |   7 +
 iocore/net/P_SSLConfig.h                           |   7 +
 iocore/net/P_SSLSecret.h                           |  13 +-
 iocore/net/SSLConfig.cc                            |  24 +-
 iocore/net/SSLSecret.cc                            | 148 +++++-----
 iocore/net/SSLUtils.cc                             |  13 +-
 iocore/net/libinknet_stub.cc                       |   7 +
 proxy/InkAPIInternal.h                             |   6 +
 src/traffic_quic/traffic_quic.cc                   |   7 +
 src/traffic_server/InkAPI.cc                       |  90 ++-----
 .../tls/tls_check_cert_select_plugin.test.py       | 166 ------------
 .../tls_check_dual_cert_selection_plugin.test.py   | 193 -------------
 .../gold_tests/tls/tls_client_cert2_plugin.test.py | 191 -------------
 .../tls/tls_client_cert_override_plugin.test.py    | 227 ----------------
 .../gold_tests/tls/tls_client_cert_plugin.test.py  | 297 ---------------------
 tests/gold_tests/tls/tls_keepalive.test.py         |   2 +-
 tests/tools/plugins/Makefile.inc                   |   3 -
 tests/tools/plugins/ssl_secret_load_test.cc        | 184 -------------
 23 files changed, 158 insertions(+), 1530 deletions(-)

diff --git a/doc/developer-guide/api/functions/TSLifecycleHookAdd.en.rst 
b/doc/developer-guide/api/functions/TSLifecycleHookAdd.en.rst
index 5a3f029bcb..d1fca4964f 100644
--- a/doc/developer-guide/api/functions/TSLifecycleHookAdd.en.rst
+++ b/doc/developer-guide/api/functions/TSLifecycleHookAdd.en.rst
@@ -40,8 +40,8 @@ specified by :arg:`id`. Lifecycle hooks are based on the 
Traffic Server
 process, not on any specific transaction or session. These will typically be
 called only once during the execution of the Traffic Server process and
 therefore should be added in :func:`TSPluginInit` (which could itself be
-considered a lifecycle hook). Unlike other hooks, lifecycle hooks may not have 
a
-well defined ordering and use of them should not assume that one of the hooks
+considered a lifecycle hook). Unlike other hooks, lifecycle hooks may not have
+a well defined ordering and use of them should not assume that one of the hooks
 is always called before another unless specifically mentioned.
 
 Types
@@ -106,14 +106,6 @@ Types
       Invoked with the event :c:data:`TS_EVENT_LIFECYCLE_TASK_THREADS_READY` 
and ``NULL``
       data.
 
-   .. cpp:enumerator:: TS_LIFECYCLE_SSL_SECRET_HOOK
-
-      Called before the data for the certificate or key is loaded.  The data 
argument to the callback is a pointer to a :type:`TSSecretID` which
-      contains a pointer to the name of the certificate or key and the 
relevant version if applicable.
-
-      This hook gives the plugin a chance to load the certificate or key from 
an alternative source and set via the :c:func:`TSSslSecretSet` API.
-      If there is no plugin override, the certificate or key will be loaded 
from disk and the secret name will be interpreted as a file path.
-
    .. cpp:enumerator:: TS_LIFECYCLE_SHUTDOWN_HOOK
 
       Called after |TS| receiving a shutdown signal, such as SIGTERM.
diff --git a/doc/developer-guide/api/functions/TSSslSecret.en.rst 
b/doc/developer-guide/api/functions/TSSslSecret.en.rst
deleted file mode 100644
index 87478e2254..0000000000
--- a/doc/developer-guide/api/functions/TSSslSecret.en.rst
+++ /dev/null
@@ -1,77 +0,0 @@
-.. Licensed to the Apache Software Foundation (ASF) under one or more
-      contributor license agreements.  See the NOTICE file distributed
-   with this work for additional information regarding copyright
-   ownership.  The ASF licenses this file to you under the Apache
-   License, Version 2.0 (the "License"); you may not use this file
-   except in compliance with the License.  You may obtain a copy of
-   the License at
-   http://www.apache.org/licenses/LICENSE-2.0
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
-   implied.  See the License for the specific language governing
-   permissions and limitations under the License.
-.. include:: /common.defs
-
-.. default-domain:: c
-
-TSSslSecretSet
-**************
-
-Set the data associated with a secret name specified in the config.
-
-Synopsis
-========
-
-.. code-block:: cpp
-
-    #include <ts/ts.h>
-
-.. function:: TSReturnCode TSSslSecretSet(const char * secret_name, int 
secret_name_length, const char * secret_data, int secret_data_len)
-
-Description
-===========
-
-:func:`TSSslSecretSet` updates the current secret map. Generally the secret 
name corresponds to the name of a certificate or a key.
-Future creation of SSL_CTX objects that use the secret will use the newly 
specified data. It can be useful to call this function
-from the :cpp:enumerator:`TS_LIFECYCLE_SSL_SECRET_HOOK`.
-
-TSSslSecretGet
-**************
-
-Get the data associated with a secret name specified in the config.
-
-Synopsis
-========
-
-.. code-block:: cpp
-
-    #include <ts/ts.h>
-
-.. function:: TSReturnCode TSSslSecretGet(const char * secret_name, int 
secret_name_length, const char ** secret_data_return, int * secret_data_len)
-
-Description
-===========
-
-:func:`TSSslSecretGet` fetches the named secret from the current secret map. 
TS_ERROR is returned if there is no entry for the secret.
-
-TSSslSecretUpdate
-*****************
-
-Tell |TS| to update the SSL objects dependent on the secret.
-
-Synopsis
-========
-
-.. code-block:: cpp
-
-    #include <ts/ts.h>
-
-.. function:: TSReturnCode TSSslSecretUpdate(const char * secret_name, int 
secret_name_length)
-
-Description
-===========
-
-:func:`TSSslSecretUpdate` causes |TS| to update the SSL objects that depend on 
the specified secret.  This enables a plugin to look for
-multiple secret updates and make calls to :func:`TSSslSecretSet` to update the 
secret table.  Then once everything is updated call
-:func:`TSSslSecretUpdate` to update the SSL objects with a consistent updated 
set of secrets.
diff --git a/doc/release-notes/whats-new.en.rst 
b/doc/release-notes/whats-new.en.rst
index dc13839018..a21181920a 100644
--- a/doc/release-notes/whats-new.en.rst
+++ b/doc/release-notes/whats-new.en.rst
@@ -547,9 +547,6 @@ make the older ``stats_over_http`` obsolete.
 Plugin APIs
 -----------
 
-A new hook for loading certificates was added, 
:cpp:enumerator:`TS_LIFECYCLE_SSL_SECRET_HOOK`. When using
-this hook, the plugin recieved a structure with a type :c:type:`TSSecretID`.
-
 The transction control APIs where refactored and promoted to that ``ts.h`` 
public APIs. This adds
 :c:func:`TSHttpTxnCntlGet` and :c:func:`TSHttpTxnCntlSet`, and the 
c:enum::`TSHttpCntlType` enum.
 
diff --git a/include/ts/apidefs.h.in b/include/ts/apidefs.h.in
index ac550a95fd..24a26a28e2 100644
--- a/include/ts/apidefs.h.in
+++ b/include/ts/apidefs.h.in
@@ -438,8 +438,8 @@ typedef enum {
   TS_LIFECYCLE_MSG_HOOK,
   TS_LIFECYCLE_TASK_THREADS_READY_HOOK,
   TS_LIFECYCLE_SHUTDOWN_HOOK,
-  TS_LIFECYCLE_SSL_SECRET_HOOK,
-  TS_LIFECYCLE_LAST_HOOK
+  // TS_LIFECYCLE_SSL_SECRET_HOOK, future release
+  TS_LIFECYCLE_LAST_HOOK = TS_LIFECYCLE_SHUTDOWN_HOOK + 2
 } TSLifecycleHookID;
 
 /**
diff --git a/include/ts/ts.h b/include/ts/ts.h
index 4d35972e7e..2a15194132 100644
--- a/include/ts/ts.h
+++ b/include/ts/ts.h
@@ -1302,13 +1302,6 @@ tsapi TSSslContext TSSslClientContextFindByName(const 
char *ca_paths, const char
 tsapi TSReturnCode TSSslClientCertUpdate(const char *cert_path, const char 
*key_path);
 tsapi TSReturnCode TSSslServerCertUpdate(const char *cert_path, const char 
*key_path);
 
-/* Update the transient secret table for SSL_CTX loading */
-tsapi TSReturnCode TSSslSecretSet(const char *secret_name, int 
secret_name_length, const char *secret_data, int secret_data_len);
-tsapi TSReturnCode TSSslSecretGet(const char *secret_name, int 
secret_name_length, const char **secret_data_return,
-                                  int *secret_data_len);
-
-tsapi TSReturnCode TSSslSecretUpdate(const char *secret_name, int 
secret_name_length);
-
 /* Create a new SSL context based on the settings in records.config */
 tsapi TSSslContext TSSslServerContextCreate(TSSslX509 cert, const char 
*certname, const char *rsp_file);
 tsapi void TSSslContextDestroy(TSSslContext ctx);
diff --git a/iocore/cache/test/stub.cc b/iocore/cache/test/stub.cc
index b247da1bb3..8c28cf028e 100644
--- a/iocore/cache/test/stub.cc
+++ b/iocore/cache/test/stub.cc
@@ -51,6 +51,13 @@ APIHook::invoke(int, void *) const
   return 0;
 }
 
+int
+APIHook::blocking_invoke(int, void *) const
+{
+  ink_assert(false);
+  return 0;
+}
+
 APIHook *
 APIHook::next() const
 {
diff --git a/iocore/net/P_SSLConfig.h b/iocore/net/P_SSLConfig.h
index c31494bc7b..3338b37da7 100644
--- a/iocore/net/P_SSLConfig.h
+++ b/iocore/net/P_SSLConfig.h
@@ -30,6 +30,8 @@
  ****************************************************************************/
 #pragma once
 
+#include <atomic>
+
 #include <openssl/rand.h>
 
 #include "tscore/ink_inet.h"
@@ -165,6 +167,11 @@ struct SSLConfigParams : public ConfigInfo {
   void cleanup();
   void reset();
   void SSLConfigInit(IpMap *global);
+
+private:
+  // c_str() of string passed to in-progess call to updateCTX().
+  //
+  mutable std::atomic<char const *> secret_for_updateCTX{nullptr};
 };
 
 /////////////////////////////////////////////////////////////
diff --git a/iocore/net/P_SSLSecret.h b/iocore/net/P_SSLSecret.h
index f6b2a27942..212ab156c6 100644
--- a/iocore/net/P_SSLSecret.h
+++ b/iocore/net/P_SSLSecret.h
@@ -19,6 +19,8 @@
   limitations under the License.
  */
 
+#pragma once
+
 #include <string>
 #include <string_view>
 #include <mutex>
@@ -28,14 +30,13 @@ class SSLSecret
 {
 public:
   SSLSecret() {}
-  bool getSecret(const std::string &name, std::string_view &data) const;
-  bool setSecret(const std::string &name, const char *data, int data_len);
-  bool getOrLoadSecret(const std::string &name, const std::string &name2, 
std::string_view &data, std::string_view &data2);
+  std::string getSecret(const std::string &name) const;
+  void setSecret(const std::string &name, std::string_view data);
+  void getOrLoadSecret(const std::string &name1, const std::string &name2, 
std::string &data, std::string &data2);
 
 private:
-  const std::string *getSecretItem(const std::string &name) const;
-  bool loadSecret(const std::string &name, const std::string &name2, 
std::string &data_item, std::string &data_item2);
-  bool loadFile(const std::string &name, std::string &data_item);
+  void loadSecret(const std::string &name1, const std::string &name2, 
std::string &data_item, std::string &data_item2);
+  std::string loadFile(const std::string &name);
 
   std::unordered_map<std::string, std::string> secret_map;
   mutable std::recursive_mutex secret_map_mutex;
diff --git a/iocore/net/SSLConfig.cc b/iocore/net/SSLConfig.cc
index 006a4e81c2..5825b0c66b 100644
--- a/iocore/net/SSLConfig.cc
+++ b/iocore/net/SSLConfig.cc
@@ -739,6 +739,24 @@ cleanup_bio(BIO *&biop)
 void
 SSLConfigParams::updateCTX(const std::string &cert_secret_name) const
 {
+  Debug("ssl_config_updateCTX", "Update cert %s, %p", 
cert_secret_name.c_str(), this);
+
+  // Instances of SSLConfigParams should access by one thread at a time only.  
secret_for_updateCTX is accessed
+  // atomically as a fail-safe.
+  //
+  char const *expected = nullptr;
+  if (!secret_for_updateCTX.compare_exchange_strong(expected, 
cert_secret_name.c_str())) {
+    if (is_debug_tag_set("ssl_config_updateCTX")) {
+      // As a fail-safe, handle it if secret_for_updateCTX doesn't or no 
longer points to a null-terminated string.
+      //
+      char const *s{expected};
+      for (; *s && (std::size_t(s - expected) < cert_secret_name.size()); ++s) 
{
+      }
+      Debug("ssl_config_updateCTX", "Update cert, indirect recusive call 
caused by call for %.*s", int(s - expected), expected);
+    }
+    return;
+  }
+
   // Clear the corresponding client CTXs.  They will be lazy loaded later
   Debug("ssl_load", "Update cert %s", cert_secret_name.c_str());
   this->clearCTX(cert_secret_name);
@@ -746,6 +764,8 @@ SSLConfigParams::updateCTX(const std::string 
&cert_secret_name) const
   // Update the server cert
   SSLMultiCertConfigLoader loader(this);
   loader.update_ssl_ctx(cert_secret_name);
+
+  secret_for_updateCTX = nullptr;
 }
 
 void
@@ -800,8 +820,8 @@ SSLConfigParams::getCTX(const std::string &client_cert, 
const std::string &key_f
 
     // Set public and private keys
     if (!client_cert.empty()) {
-      std::string_view secret_data;
-      std::string_view secret_key_data;
+      std::string secret_data;
+      std::string secret_key_data;
 
       // Fetch the client_cert data
       std::string 
completeSecretPath{Layout::get()->relative_to(this->clientCertPathOnly, 
client_cert)};
diff --git a/iocore/net/SSLSecret.cc b/iocore/net/SSLSecret.cc
index b8b01895be..b84d241dad 100644
--- a/iocore/net/SSLSecret.cc
+++ b/iocore/net/SSLSecret.cc
@@ -18,131 +18,117 @@
   See the License for the specific language governing permissions and
   limitations under the License.
  */
-#include <string>
-#include <map>
+
 #include "InkAPIInternal.h" // Added to include the ssl_hook and 
lifestyle_hook definitions
 #include "tscore/ts_file.h"
 #include "P_SSLConfig.h"
 
-bool
-SSLSecret::loadSecret(const std::string &name1, const std::string &name2, 
std::string &data_item1, std::string &data_item2)
+#include <utility>
+
+// NOTE: The secret_map_mutex should not be held by the caller of this
+// function. The implementation of this function may call a plugin's
+// TS_EVENT_SSL_SECRET handler which in turn may grab a lock for
+// secret_map_mutex via a TSSslSecretSet (proposed in a future release) call.
+// These events will result in a deadlock.
+void
+SSLSecret::loadSecret(const std::string &name1, const std::string &name2, 
std::string &data1, std::string &data2)
 {
   // Call the load secret hooks
   //
-  class APIHook *curHook = lifecycle_hooks->get(TS_LIFECYCLE_SSL_SECRET_HOOK);
+  class APIHook *curHook = lifecycle_hooks->get(/* 
TS_LIFECYCLE_SSL_SECRET_HOOK */ TS_LIFECYCLE_LAST_HOOK /* dummy value */);
   TSSecretID secret_name;
   secret_name.cert_name     = name1.data();
   secret_name.cert_name_len = name1.size();
   secret_name.key_name      = name2.data();
   secret_name.key_name_len  = name2.size();
   while (curHook) {
-    curHook->invoke(TS_EVENT_SSL_SECRET, &secret_name);
+    curHook->blocking_invoke(TS_EVENT_SSL_SECRET, &secret_name);
     curHook = curHook->next();
   }
 
-  const std::string *data1 = this->getSecretItem(name1);
-  const std::string *data2 = this->getSecretItem(name2);
-  if ((nullptr == data1 || data1->length() == 0) || (!name2.empty() && 
(nullptr == data2 || data2->length() == 0))) {
+  data1 = this->getSecret(name1);
+  data2 = name2.empty() ? std::string{} : this->getSecret(name2);
+  if (data1.empty() || (!name2.empty() && data2.empty())) {
     // If none of them loaded it, assume it is a file
-    return loadFile(name1, data_item1) && (name2.empty() || loadFile(name2, 
data_item2));
+    data1 = loadFile(name1);
+    setSecret(name1, data1);
+    if (!name2.empty()) {
+      data2 = loadFile(name2);
+      setSecret(name2, data2);
+    }
   }
-  return true;
 }
 
-bool
-SSLSecret::loadFile(const std::string &name, std::string &data_item)
+std::string
+SSLSecret::loadFile(const std::string &name)
 {
-  struct stat statdata;
-  // Load the secret and add it to the map
-  if (stat(name.c_str(), &statdata) < 0) {
-    Debug("ssl_secret", "File: %s received error: %s", name.c_str(), 
strerror(errno));
-    return false;
-  }
+  Debug("ssl_secret", "SSLSecret::loadFile(%s)", name.c_str());
   std::error_code error;
-  data_item = ts::file::load(ts::file::path(name), error);
+  std::string const data = ts::file::load(ts::file::path(name), error);
   if (error) {
+    Debug("ssl_secret_err", "SSLSecret::loadFile(%s) failed error code=%d 
message=%s", name.c_str(), error.value(),
+          error.message().c_str());
     // Loading file failed
     Debug("ssl_secret", "Loading file: %s failed ", name.c_str());
-    return false;
+    return std::string{};
   }
+  Debug("ssl_secret", "Secret data: %.50s", data.c_str());
   if (SSLConfigParams::load_ssl_file_cb) {
     SSLConfigParams::load_ssl_file_cb(name.c_str());
   }
-  return true;
+  return data;
 }
 
-bool
-SSLSecret::setSecret(const std::string &name, const char *data, int data_len)
+void
+SSLSecret::setSecret(const std::string &name, std::string_view data)
 {
   std::scoped_lock lock(secret_map_mutex);
-  auto iter = secret_map.find(name);
-  if (iter == secret_map.end()) {
-    secret_map[name] = "";
-    iter             = secret_map.find(name);
-  }
-  if (iter == secret_map.end()) {
-    return false;
-  }
-  iter->second.assign(data, data_len);
+  secret_map[name] = std::string{data};
   // The full secret data can be sensitive. Print only the first 50 bytes.
-  Debug("ssl_secret", "Set secret for %s to %.50s", name.c_str(), 
iter->second.c_str());
-  return true;
+  Debug("ssl_secret", "Set secret for %s to %.*s", name.c_str(), 
int(data.size() > 50 ? 50 : data.size()), data.data());
 }
 
-const std::string *
-SSLSecret::getSecretItem(const std::string &name) const
+std::string
+SSLSecret::getSecret(const std::string &name) const
 {
   std::scoped_lock lock(secret_map_mutex);
   auto iter = secret_map.find(name);
-  if (iter == secret_map.end()) {
-    return nullptr;
-  }
-  return &iter->second;
-}
-
-bool
-SSLSecret::getSecret(const std::string &name, std::string_view &data) const
-{
-  const std::string *data_item = this->getSecretItem(name);
-  if (data_item) {
-    // The full secret data can be sensitive. Print only the first 50 bytes.
-    Debug("ssl_secret", "Get secret for %s: %.50s", name.c_str(), 
data_item->c_str());
-    data = *data_item;
-  } else {
+  if (secret_map.end() == iter) {
     Debug("ssl_secret", "Get secret for %s: not found", name.c_str());
-    data = std::string_view{};
+    return std::string{};
   }
-  return data_item != nullptr;
+  if (iter->second.empty()) {
+    Debug("ssl_secret", "Get secret for %s: empty", name.c_str());
+    return std::string{};
+  }
+  // The full secret data can be sensitive. Print only the first 50 bytes.
+  Debug("ssl_secret", "Get secret for %s: %.50s", name.c_str(), 
iter->second.c_str());
+  return iter->second;
 }
 
-bool
-SSLSecret::getOrLoadSecret(const std::string &name1, const std::string &name2, 
std::string_view &data1, std::string_view &data2)
+void
+SSLSecret::getOrLoadSecret(const std::string &name1, const std::string &name2, 
std::string &data1, std::string &data2)
 {
-  Debug("ssl_secret", "lookup up secrets for %s and %s", name1.c_str(), 
name2.empty() ? "[empty]" : name2.c_str());
-  std::scoped_lock lock(secret_map_mutex);
-  bool found_secret1 = this->getSecret(name1, data1);
-  bool found_secret2 = name2.empty() || this->getSecret(name2, data2);
-
-  // If we can't find either secret, load them both again
-  if (!found_secret1 || !found_secret2) {
-    // Make sure each name has an entry
-    if (!found_secret1) {
-      secret_map[name1] = "";
-    }
-    if (!found_secret2) {
-      secret_map[name2] = "";
-    }
-    auto iter1 = secret_map.find(name1);
-    auto iter2 = name2.empty() ? iter1 : secret_map.find(name2);
-    if (this->loadSecret(name1, name2, iter1->second, iter2->second)) {
-      data1 = iter1->second;
-      if (!name2.empty()) {
-        data2 = iter2->second;
+  Debug("ssl_secret", "lookup up secrets for %s and %s", name1.c_str(), 
name2.c_str());
+  {
+    std::scoped_lock lock(secret_map_mutex);
+    std::string *const data1ptr = &(secret_map[name1]);
+    std::string *const data2ptr = [&]() -> std::string *const {
+      if (name2.empty()) {
+        data2.clear();
+        return &data2;
       }
-      return true;
-    }
-  } else {
-    return true;
+      return &(secret_map[name2]);
+    }();
+    data1 = *data1ptr;
+    data2 = *data2ptr;
+  }
+  // If we can't find either secret, load them both again
+  if (data1.empty() || (!name2.empty() && data2.empty())) {
+    std::string data1tmp;
+    std::string data2tmp;
+    this->loadSecret(name1, name2, data1tmp, data2tmp);
+    data1 = std::move(data1tmp);
+    data2 = std::move(data2tmp);
   }
-  return false;
 }
diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc
index d257f62040..551a1413de 100644
--- a/iocore/net/SSLUtils.cc
+++ b/iocore/net/SSLUtils.cc
@@ -1020,8 +1020,8 @@ SSLPrivateKeyHandler(SSL_CTX *ctx, const SSLConfigParams 
*params, const char *ke
     scoped_BIO bio(BIO_new_mem_buf(secret_data, secret_data_len));
     pkey = PEM_read_bio_PrivateKey(bio.get(), nullptr, nullptr, nullptr);
     if (nullptr == pkey) {
-      Debug("ssl_load", "failed to load server private key from %s",
-            (!keyPath || keyPath[0] == '\0') ? "[empty key path]" : keyPath);
+      Debug("ssl_load", "failed to load server private key (%.*s) from %s", 
secret_data_len < 50 ? secret_data_len : 50,
+            secret_data, (!keyPath || keyPath[0] == '\0') ? "[empty key path]" 
: keyPath);
       return false;
     }
     if (!SSL_CTX_use_PrivateKey(ctx, pkey)) {
@@ -2242,8 +2242,8 @@ 
SSLMultiCertConfigLoader::load_certs_and_cross_reference_names(
   }
 
   for (size_t i = 0; i < data.cert_names_list.size(); i++) {
-    std::string_view secret_data;
-    std::string_view secret_key_data;
+    std::string secret_data;
+    std::string secret_key_data;
     params->secrets.getOrLoadSecret(data.cert_names_list[i], 
data.key_list.size() > i ? data.key_list[i] : "", secret_data,
                                     secret_key_data);
     if (secret_data.empty()) {
@@ -2402,8 +2402,8 @@ SSLMultiCertConfigLoader::load_certs(SSL_CTX *ctx, const 
std::vector<std::string
 
   for (size_t i = 0; i < cert_names_list.size(); i++) {
     std::string keyPath = (i < key_list.size()) ? key_list[i] : "";
-    std::string_view secret_data;
-    std::string_view secret_key_data;
+    std::string secret_data;
+    std::string secret_key_data;
     params->secrets.getOrLoadSecret(cert_names_list[i], keyPath, secret_data, 
secret_key_data);
     if (secret_data.empty()) {
       SSLError("failed to load certificate secret for %s with key path %s", 
cert_names_list[i].c_str(),
@@ -2439,6 +2439,7 @@ SSLMultiCertConfigLoader::load_certs(SSL_CTX *ctx, const 
std::vector<std::string
     }
 
     if (secret_key_data.empty()) {
+      Note("Empty private key for public key %.*s", int(secret_data.size()), 
secret_data.data());
       secret_key_data = secret_data;
     }
     if (!SSLPrivateKeyHandler(ctx, params, keyPath.c_str(), 
secret_key_data.data(), secret_key_data.size())) {
diff --git a/iocore/net/libinknet_stub.cc b/iocore/net/libinknet_stub.cc
index 1c65bd3325..a48273372a 100644
--- a/iocore/net/libinknet_stub.cc
+++ b/iocore/net/libinknet_stub.cc
@@ -92,6 +92,13 @@ APIHook::invoke(int, void *) const
   return 0;
 }
 
+int
+APIHook::blocking_invoke(int, void *) const
+{
+  ink_assert(false);
+  return 0;
+}
+
 APIHook *
 APIHook::next() const
 {
diff --git a/proxy/InkAPIInternal.h b/proxy/InkAPIInternal.h
index 539972aac9..2c7c716bac 100644
--- a/proxy/InkAPIInternal.h
+++ b/proxy/InkAPIInternal.h
@@ -117,6 +117,12 @@ public:
   APIHook *next() const;
   APIHook *prev() const;
   LINK(APIHook, m_link);
+
+  // This is like invoke(), but allows for blocking on continuation mutexes.  
It is a hack, calling it can block
+  // the calling thread.  Hooks that require this should be reimplemented, 
modeled on the hook handling in HttpSM.cc .
+  // That is, try to lock the mutex, and reschedule the contination if the 
mutex cannot be locked.
+  //
+  int blocking_invoke(int event, void *edata) const;
 };
 
 /// A collection of API hooks.
diff --git a/src/traffic_quic/traffic_quic.cc b/src/traffic_quic/traffic_quic.cc
index 8f42a9bb2b..1533224184 100644
--- a/src/traffic_quic/traffic_quic.cc
+++ b/src/traffic_quic/traffic_quic.cc
@@ -213,6 +213,13 @@ APIHook::invoke(int, void *) const
   return 0;
 }
 
+int
+APIHook::blocking_invoke(int, void *) const
+{
+  ink_assert(false);
+  return 0;
+}
+
 APIHook *
 APIHooks::head() const
 {
diff --git a/src/traffic_server/InkAPI.cc b/src/traffic_server/InkAPI.cc
index d5bc38a3cd..8785b24360 100644
--- a/src/traffic_server/InkAPI.cc
+++ b/src/traffic_server/InkAPI.cc
@@ -21,12 +21,9 @@
   limitations under the License.
  */
 
-#include <cstdio>
 #include <atomic>
 #include <string_view>
-#include <tuple>
-#include <unordered_map>
-#include <string_view>
+#include <string>
 
 #include "tscore/ink_platform.h"
 #include "tscore/ink_base64.h"
@@ -1362,7 +1359,7 @@ APIHook::prev() const
 int
 APIHook::invoke(int event, void *edata) const
 {
-  if ((event == EVENT_IMMEDIATE) || (event == EVENT_INTERVAL) || event == 
TS_EVENT_HTTP_TXN_CLOSE) {
+  if (event == EVENT_IMMEDIATE || event == EVENT_INTERVAL || event == 
TS_EVENT_HTTP_TXN_CLOSE) {
     if (ink_atomic_increment((int *)&m_cont->m_event_count, 1) < 0) {
       ink_assert(!"not reached");
     }
@@ -1375,6 +1372,20 @@ APIHook::invoke(int event, void *edata) const
   return m_cont->handleEvent(event, edata);
 }
 
+int
+APIHook::blocking_invoke(int event, void *edata) const
+{
+  if (event == EVENT_IMMEDIATE || event == EVENT_INTERVAL || event == 
TS_EVENT_HTTP_TXN_CLOSE) {
+    if (ink_atomic_increment((int *)&m_cont->m_event_count, 1) < 0) {
+      ink_assert(!"not reached");
+    }
+  }
+
+  WEAK_SCOPED_MUTEX_LOCK(lock, m_cont->mutex, this_ethread());
+
+  return m_cont->handleEvent(event, edata);
+}
+
 APIHook *
 APIHooks::head() const
 {
@@ -9616,75 +9627,6 @@ TSSslContextFindByAddr(struct sockaddr const *addr)
   return ret;
 }
 
-/**
- * This function sets the secret cache value for a given secret name.  This 
allows
- * plugins to load cert/key PEM information on for use by the TLS core
- */
-tsapi TSReturnCode
-TSSslSecretSet(const char *secret_name, int secret_name_length, const char 
*secret_data, int secret_data_len)
-{
-  TSReturnCode retval          = TS_SUCCESS;
-  SSLConfigParams *load_params = SSLConfig::load_acquire();
-  SSLConfigParams *params      = SSLConfig::acquire();
-  if (load_params != nullptr) { // Update the current data structure
-    Debug("ssl.cert_update", "Setting secrets in SSLConfig load for: %.*s", 
secret_name_length, secret_name);
-    if (!load_params->secrets.setSecret(std::string(secret_name, 
secret_name_length), secret_data, secret_data_len)) {
-      retval = TS_ERROR;
-    }
-    load_params->updateCTX(std::string(secret_name, secret_name_length));
-    SSLConfig::load_release(load_params);
-  }
-  if (params != nullptr) {
-    Debug("ssl.cert_update", "Setting secrets in SSLConfig for: %.*s", 
secret_name_length, secret_name);
-    if (!params->secrets.setSecret(std::string(secret_name, 
secret_name_length), secret_data, secret_data_len)) {
-      retval = TS_ERROR;
-    }
-    params->updateCTX(std::string(secret_name, secret_name_length));
-    SSLConfig::release(params);
-  }
-  return retval;
-}
-
-tsapi TSReturnCode
-TSSslSecretUpdate(const char *secret_name, int secret_name_length)
-{
-  TSReturnCode retval     = TS_SUCCESS;
-  SSLConfigParams *params = SSLConfig::acquire();
-  if (params != nullptr) {
-    params->updateCTX(std::string(secret_name, secret_name_length));
-  }
-  SSLConfig::release(params);
-  return retval;
-}
-
-tsapi TSReturnCode
-TSSslSecretGet(const char *secret_name, int secret_name_length, const char 
**secret_data_return, int *secret_data_len)
-{
-  bool loading            = true;
-  TSReturnCode retval     = TS_SUCCESS;
-  SSLConfigParams *params = SSLConfig::load_acquire();
-  if (params == nullptr) {
-    params  = SSLConfig::acquire();
-    loading = false;
-  }
-  std::string_view secret_data;
-  if (!params->secrets.getSecret(std::string(secret_name, secret_name_length), 
secret_data)) {
-    retval = TS_ERROR;
-  }
-  if (secret_data_return) {
-    *secret_data_return = secret_data.data();
-  }
-  if (secret_data_len) {
-    *secret_data_len = secret_data.size();
-  }
-  if (loading) {
-    SSLConfig::load_release(params);
-  } else {
-    SSLConfig::release(params);
-  }
-  return retval;
-}
-
 /**
  * This function retrieves an array of lookup keys for client contexts loaded 
in
  * traffic server. Given a 2-level mapping for client contexts, every 2 lookup 
keys
diff --git a/tests/gold_tests/tls/tls_check_cert_select_plugin.test.py 
b/tests/gold_tests/tls/tls_check_cert_select_plugin.test.py
deleted file mode 100644
index b189060d1a..0000000000
--- a/tests/gold_tests/tls/tls_check_cert_select_plugin.test.py
+++ /dev/null
@@ -1,166 +0,0 @@
-'''
-'''
-#  Licensed to the Apache Software Foundation (ASF) under one
-#  or more contributor license agreements.  See the NOTICE file
-#  distributed with this work for additional information
-#  regarding copyright ownership.  The ASF licenses this file
-#  to you under the Apache License, Version 2.0 (the
-#  "License"); you may not use this file except in compliance
-#  with the License.  You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-#  Unless required by applicable law or agreed to in writing, software
-#  distributed under the License is distributed on an "AS IS" BASIS,
-#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-#  See the License for the specific language governing permissions and
-#  limitations under the License.
-
-import os
-
-Test.Summary = '''
-Test ATS offering different certificates based on SNI. Load via plugin
-'''
-
-# Define default ATS
-ts = Test.MakeATSProcess("ts", command="traffic_manager", select_ports=True, 
enable_tls=True)
-server = Test.MakeOriginServer("server", ssl=True)
-dns = Test.MakeDNServer("dns")
-
-request_header = {"headers": "GET / HTTP/1.1\r\n\r\n", "timestamp": 
"1469733493.993", "body": ""}
-response_header = {"headers": "HTTP/1.1 200 OK\r\nConnection: close\r\n\r\n", 
"timestamp": "1469733493.993", "body": ""}
-server.addResponse("sessionlog.json", request_header, response_header)
-
-# add ssl materials like key, certificates for the server
-ts.addSSLfile("ssl/signed-foo.pem")
-ts.addSSLfile("ssl/signed-foo.key")
-ts.addSSLfile("ssl/signed-bar.pem")
-ts.addSSLfile("ssl/signed2-bar.pem")
-ts.addSSLfile("ssl/signed-bar.key")
-ts.addSSLfile("ssl/server.pem")
-ts.addSSLfile("ssl/server.key")
-ts.addSSLfile("ssl/signer.pem")
-ts.addSSLfile("ssl/signer.key")
-
-ts.Disk.remap_config.AddLine(
-    'map / https://foo.com:{1}'.format(ts.Variables.ssl_port, 
server.Variables.SSL_Port))
-
-ts.Disk.ssl_multicert_config.AddLines([
-    'dest_ip=127.0.0.1 ssl_cert_name=signed-foo.pem 
ssl_key_name=signed-foo.key',
-    'ssl_cert_name=signed2-bar.pem ssl_key_name=signed-bar.key',
-    'dest_ip=* ssl_cert_name=server.pem ssl_key_name=server.key'
-])
-
-Test.PrepareTestPlugin(os.path.join(Test.Variables.AtsTestPluginsDir, 
'ssl_secret_load_test.so'), ts)
-
-# Case 1, global config policy=permissive properties=signature
-#         override for foo.com policy=enforced properties=all
-ts.Disk.records_config.update({
-    'proxy.config.diags.debug.tags': 'ssl_secret_load_test|ssl',
-    'proxy.config.diags.debug.enabled': 1,
-    'proxy.config.ssl.server.cert.path': '{0}/../'.format(ts.Variables.SSLDir),
-    'proxy.config.ssl.server.private_key.path': 
'{0}/../'.format(ts.Variables.SSLDir),
-    'proxy.config.ssl.server.cipher_suite': 
'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:AES128-GCM-SHA256:AES256-GCM-SHA384:ECDHE-RSA-RC4-SHA:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:RC4-SHA:RC4-MD5:AES128-SHA:AES256-SHA:DES-CBC3-SHA!SRP:!DSS:!PSK:!aNULL:!eNULL:!SSLv2',
-    'proxy.config.url_remap.pristine_host_hdr': 1,
-    'proxy.config.dns.nameservers': '127.0.0.1:{0}'.format(dns.Variables.Port),
-    'proxy.config.exec_thread.autoconfig.scale': 1.0,
-    'proxy.config.dns.resolv_conf': 'NULL',
-    'proxy.config.ssl.client.verify.server.policy': 'PERMISSIVE',
-})
-
-dns.addRecords(records={"foo.com.": ["127.0.0.1"]})
-dns.addRecords(records={"bar.com.": ["127.0.0.1"]})
-
-# Should receive a bar.com cert
-tr = Test.AddTestRun("bar.com cert")
-tr.Setup.Copy("ssl/signer.pem")
-tr.Setup.Copy("ssl/signer2.pem")
-tr.Processes.Default.Command = "curl -v --cacert ./signer2.pem  --resolve 
'bar.com:{0}:127.0.0.1' https://bar.com:{0}".format(
-    ts.Variables.ssl_port)
-tr.ReturnCode = 0
-tr.Processes.Default.StartBefore(server)
-tr.Processes.Default.StartBefore(dns)
-tr.Processes.Default.StartBefore(Test.Processes.ts, 
ready=When.PortOpen(ts.Variables.ssl_port))
-tr.StillRunningAfter = server
-tr.StillRunningAfter = ts
-tr.Processes.Default.Streams.All = Testers.ExcludesExpression("Could Not 
Connect", "Curl attempt should have succeeded")
-tr.Processes.Default.Streams.All += Testers.ContainsExpression("CN=bar.com", 
"Cert should contain bar.com")
-tr.Processes.Default.Streams.All += Testers.ExcludesExpression("CN=foo.com", 
"Cert should not contain foo.com")
-tr.Processes.Default.Streams.All += Testers.ContainsExpression("404", "Should 
make an exchange")
-
-# Should receive a foo.com cert
-tr2 = Test.AddTestRun("foo.com cert")
-tr2.Processes.Default.Command = "curl -v --cacert ./signer.pem --resolve 
'foo.com:{0}:127.0.0.1' https://foo.com:{0}".format(
-    ts.Variables.ssl_port)
-tr2.ReturnCode = 0
-tr2.StillRunningAfter = server
-tr2.StillRunningAfter = ts
-tr2.Processes.Default.Streams.All = Testers.ExcludesExpression("Could Not 
Connect", "Curl attempt should have succeeded")
-tr2.Processes.Default.Streams.All += Testers.ContainsExpression("CN=foo.com", 
"Cert should contain foo.com")
-tr2.Processes.Default.Streams.All += Testers.ExcludesExpression("CN=bar.com", 
"Cert should not contain bar.com")
-tr.Processes.Default.Streams.All += Testers.ContainsExpression("404", "Should 
make an exchange")
-
-# Should receive random.server.com
-tr2 = Test.AddTestRun("random.server.com cert")
-tr2.Processes.Default.Command = "curl -v -k --resolve 
'random.server.com:{0}:127.0.0.1' https://random.server.com:{0}".format(
-    ts.Variables.ssl_port)
-tr2.ReturnCode = 0
-tr2.StillRunningAfter = server
-tr2.StillRunningAfter = ts
-tr2.Processes.Default.Streams.All = Testers.ExcludesExpression("Could Not 
Connect", "Curl attempt should have succeeded")
-tr2.Processes.Default.Streams.All += 
Testers.ContainsExpression("CN=random.server.com", "Cert should contain 
random.server.com")
-tr2.Processes.Default.Streams.All += Testers.ExcludesExpression("CN=foo.com", 
"Cert should not contain foo.com")
-tr2.Processes.Default.Streams.All += Testers.ExcludesExpression("CN=bar.com", 
"Cert should not contain bar.com")
-tr.Processes.Default.Streams.All += Testers.ContainsExpression("404", "Should 
make an exchange")
-
-# No SNI match should match specific IP address, foo.com
-# SNI name and returned cert name will not match, so must use -k to avoid cert 
verification
-tr2 = Test.AddTestRun("Bad SNI")
-tr2.Processes.Default.Command = "curl -v -k --cacert ./signer.pem --resolve 
'bad.sni.com:{0}:127.0.0.1' https://bad.sni.com:{0}".format(
-    ts.Variables.ssl_port)
-tr2.ReturnCode = 0
-tr2.StillRunningAfter = server
-tr2.StillRunningAfter = ts
-tr2.Processes.Default.Streams.All = Testers.ExcludesExpression("Could Not 
Connect", "Curl attempt should have succeeded")
-tr2.Processes.Default.Streams.All += Testers.ContainsExpression("CN=foo.com", 
"Cert should contain foo.com")
-tr2.Processes.Default.Streams.All += Testers.ExcludesExpression("CN=bar.com", 
"Cert should not contain bar.com")
-tr.Processes.Default.Streams.All += Testers.ContainsExpression("404", "Should 
make an exchange")
-
-# Copy in a new version of the bar.com cert.  Replace it with the version
-# signed by signer 1.  Wait at least a second to sure the file update time
-# differs
-trupdate = Test.AddTestRun("Update server bar cert file in place")
-trupdate.StillRunningAfter = ts
-trupdate.StillRunningAfter = server
-trupdate.Setup.CopyAs("ssl/signed-bar.pem", ".", 
"{0}/signed2-bar.pem".format(ts.Variables.SSLDir))
-# For some reason the Setup.CopyAs does not change the modification time, so 
we touch
-trupdate.Processes.Default.Command = 'touch 
{0}/signed2-bar.pem'.format(ts.Variables.SSLDir)
-# Need to copy over the environment so traffic_ctl knows where to find the 
unix domain socket
-trupdate.Processes.Default.Env = ts.Env
-trupdate.Processes.Default.ReturnCode = 0
-
-# The plugin will pull every 3 seconds.  So wait 4 seconds and test again.
-# Request with CA=signer.pem should work.  Request with CA=signer2.pem
-# should fail
-tr = Test.AddTestRun("Test new version of bar cert with good CA")
-tr.DelayStart = 4
-tr.Processes.Default.Command = "date; curl -v --cacert ./signer.pem  --resolve 
'bar.com:{0}:127.0.0.1' https://bar.com:{0}".format(
-    ts.Variables.ssl_port)
-tr.ReturnCode = 0
-tr.StillRunningAfter = server
-tr.StillRunningAfter = ts
-tr.Processes.Default.Streams.All = Testers.ExcludesExpression("Could Not 
Connect", "Curl attempt should have succeeded")
-tr.Processes.Default.Streams.All += Testers.ContainsExpression("CN=bar.com", 
"Cert should contain bar.com")
-tr.Processes.Default.Streams.All += Testers.ExcludesExpression("CN=foo.com", 
"Cert should not contain foo.com")
-tr.Processes.Default.Streams.All += Testers.ContainsExpression("404", "Should 
make an exchange")
-
-tr = Test.AddTestRun("Test new version of bar cert with bad CA")
-tr.Processes.Default.Command = "curl -v --cacert ./signer2.pem  --resolve 
'bar.com:{0}:127.0.0.1' https://bar.com:{0}".format(
-    ts.Variables.ssl_port)
-tr.ReturnCode = 60
-tr.StillRunningAfter = server
-tr.StillRunningAfter = ts
-tr.Processes.Default.Streams.All = Testers.ContainsExpression("unknown CA", 
"Failed handshake")
-tr.Processes.Default.Streams.All += Testers.ExcludesExpression("CN=bar.com", 
"Cert should contain bar.com")
-tr.Processes.Default.Streams.All += Testers.ExcludesExpression("CN=foo.com", 
"Cert should not contain foo.com")
-tr.Processes.Default.Streams.All += Testers.ExcludesExpression("404", "Should 
make an exchange")
diff --git a/tests/gold_tests/tls/tls_check_dual_cert_selection_plugin.test.py 
b/tests/gold_tests/tls/tls_check_dual_cert_selection_plugin.test.py
deleted file mode 100644
index 0532dbe7cc..0000000000
--- a/tests/gold_tests/tls/tls_check_dual_cert_selection_plugin.test.py
+++ /dev/null
@@ -1,193 +0,0 @@
-'''
-'''
-#  Licensed to the Apache Software Foundation (ASF) under one
-#  or more contributor license agreements.  See the NOTICE file
-#  distributed with this work for additional information
-#  regarding copyright ownership.  The ASF licenses this file
-#  to you under the Apache License, Version 2.0 (the
-#  "License"); you may not use this file except in compliance
-#  with the License.  You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-#  Unless required by applicable law or agreed to in writing, software
-#  distributed under the License is distributed on an "AS IS" BASIS,
-#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-#  See the License for the specific language governing permissions and
-#  limitations under the License.
-
-import os
-
-Test.Summary = '''
-Test ATS offering both RSA and EC certificates loaded via plugin
-'''
-
-Test.SkipUnless(Condition.HasOpenSSLVersion('1.1.1'))
-
-# Define default ATS
-ts = Test.MakeATSProcess("ts", select_ports=True, enable_tls=True)
-server = Test.MakeOriginServer("server", ssl=True)
-dns = Test.MakeDNServer("dns")
-
-request_header = {"headers": "GET / HTTP/1.1\r\n\r\n", "timestamp": 
"1469733493.993", "body": ""}
-response_header = {"headers": "HTTP/1.1 200 OK\r\nConnection: close\r\n\r\n", 
"timestamp": "1469733493.993", "body": ""}
-server.addResponse("sessionlog.json", request_header, response_header)
-
-# add ssl materials like key, certificates for the server
-ts.addSSLfile("ssl/signed-foo.pem")
-ts.addSSLfile("ssl/signed2-foo.pem")
-ts.addSSLfile("ssl/signed-foo.key")
-ts.addSSLfile("ssl/signed-foo-ec.pem")
-ts.addSSLfile("ssl/signed-foo-ec.key")
-ts.addSSLfile("ssl/signed-san.pem")
-ts.addSSLfile("ssl/signed-san.key")
-ts.addSSLfile("ssl/signed-san-ec.pem")
-ts.addSSLfile("ssl/signed-san-ec.key")
-ts.addSSLfile("ssl/signer.pem")
-ts.addSSLfile("ssl/signer.key")
-ts.addSSLfile("ssl/server.pem")
-ts.addSSLfile("ssl/server.key")
-
-ts.Disk.remap_config.AddLine(
-    'map / https://foo.com:{1}'.format(ts.Variables.ssl_port, 
server.Variables.SSL_Port))
-
-ts.Disk.ssl_multicert_config.AddLines([
-    'ssl_cert_name=signed-foo-ec.pem,signed-foo.pem 
ssl_key_name=signed-foo-ec.key,signed-foo.key',
-    'ssl_cert_name=signed-san-ec.pem,signed-san.pem 
ssl_key_name=signed-san-ec.key,signed-san.key',
-    'dest_ip=* ssl_cert_name=server.pem ssl_key_name=server.key'
-])
-
-Test.PrepareTestPlugin(os.path.join(Test.Variables.AtsTestPluginsDir, 
'ssl_secret_load_test.so'), ts)
-
-# Case 1, global config policy=permissive properties=signature
-#         override for foo.com policy=enforced properties=all
-ts.Disk.records_config.update({
-    'proxy.config.ssl.server.cert.path': '{0}/../'.format(ts.Variables.SSLDir),
-    'proxy.config.ssl.server.private_key.path': 
'{0}/../'.format(ts.Variables.SSLDir),
-    'proxy.config.ssl.server.cipher_suite': 
'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:AES128-GCM-SHA256:AES256-GCM-SHA384:ECDHE-RSA-RC4-SHA:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:RC4-SHA:RC4-MD5:AES128-SHA:AES256-SHA:DES-CBC3-SHA!SRP:!DSS:!PSK:!aNULL:!eNULL:!SSLv2',
-    'proxy.config.url_remap.pristine_host_hdr': 1,
-    'proxy.config.dns.nameservers': '127.0.0.1:{0}'.format(dns.Variables.Port),
-    'proxy.config.exec_thread.autoconfig.scale': 1.0,
-    'proxy.config.dns.resolv_conf': 'NULL',
-    'proxy.config.diags.debug.tags': 'ssl_secret_load_test',
-    'proxy.config.diags.debug.enabled': 1
-})
-
-dns.addRecords(records={"foo.com.": ["127.0.0.1"]})
-dns.addRecords(records={"bar.com.": ["127.0.0.1"]})
-
-# Should receive a EC cert
-tr = Test.AddTestRun("Default for foo should return EC cert")
-tr.Setup.Copy("ssl/signer.pem")
-tr.Setup.Copy("ssl/signer2.pem")
-tr.Processes.Default.Command = "echo foo | openssl s_client  -CAfile 
signer.pem -servername foo.com -connect 127.0.0.1:{0}".format(
-    ts.Variables.ssl_port)
-tr.ReturnCode = 0
-tr.Processes.Default.StartBefore(server)
-tr.Processes.Default.StartBefore(dns)
-tr.Processes.Default.StartBefore(Test.Processes.ts, 
ready=When.PortOpen(ts.Variables.ssl_port))
-tr.StillRunningAfter = server
-tr.StillRunningAfter = ts
-tr.Processes.Default.Streams.All += Testers.ContainsExpression("Peer signature 
type: ECDSA", "Should select EC cert")
-tr.Processes.Default.Streams.All += Testers.ExcludesExpression("unable to 
verify the first certificate", "Correct signer")
-
-# Should receive a RSA cert
-tr = Test.AddTestRun("Only offer RSA ciphers, should receive RSA cert")
-tr.Processes.Default.Command = "echo foo | openssl s_client  -CAfile 
signer.pem -servername foo.com -sigalgs 'RSA-PSS+SHA256' -connect 
127.0.0.1:{0}".format(
-    ts.Variables.ssl_port)
-tr.ReturnCode = 0
-tr.StillRunningAfter = server
-tr.StillRunningAfter = ts
-tr.Processes.Default.Streams.All += Testers.ContainsExpression("Peer signature 
type: RSA-PSS", "Should select RSA cert")
-tr.Processes.Default.Streams.All += Testers.ExcludesExpression("unable to 
verify the first certificate", "Correct signer")
-
-# Should receive a EC cert
-tr = Test.AddTestRun("Default for one.com should return EC cert")
-tr.Processes.Default.Command = "echo foo | openssl s_client  -CAfile 
signer.pem -servername one.com -connect 127.0.0.1:{0}".format(
-    ts.Variables.ssl_port)
-tr.ReturnCode = 0
-tr.StillRunningAfter = server
-tr.StillRunningAfter = ts
-tr.Processes.Default.Streams.All += Testers.ContainsExpression("Peer signature 
type: ECDSA", "Should select EC cert")
-tr.Processes.Default.Streams.All += Testers.ContainsExpression("CN = 
group.com", "Should select a group SAN")
-tr.Processes.Default.Streams.All += Testers.ExcludesExpression("unable to 
verify the first certificate", "Correct signer")
-
-# Should receive a RSA cert
-tr = Test.AddTestRun("Only offer RSA ciphers, should receive RSA cert")
-tr.Processes.Default.Command = "echo foo | openssl s_client  -CAfile 
signer.pem -servername one.com -sigalgs 'RSA-PSS+SHA256' -connect 
127.0.0.1:{0}".format(
-    ts.Variables.ssl_port)
-tr.ReturnCode = 0
-tr.StillRunningAfter = server
-tr.StillRunningAfter = ts
-tr.Processes.Default.Streams.All += Testers.ContainsExpression("Peer signature 
type: RSA-PSS", "Should select RSA cert")
-tr.Processes.Default.Streams.All += Testers.ContainsExpression("CN = 
group.com", "Should select a group SAN")
-tr.Processes.Default.Streams.All += Testers.ExcludesExpression("unable to 
verify the first certificate", "Correct signer")
-
-# Should receive a RSA cert
-tr = Test.AddTestRun("rsa.com only in rsa cert")
-tr.Processes.Default.Command = "echo foo | openssl s_client  -CAfile 
signer.pem -servername rsa.com -connect 127.0.0.1:{0}".format(
-    ts.Variables.ssl_port)
-tr.ReturnCode = 0
-tr.StillRunningAfter = server
-tr.StillRunningAfter = ts
-tr.Processes.Default.Streams.All += Testers.ContainsExpression("Peer signature 
type: RSA-PSS", "Should select RSA cert")
-tr.Processes.Default.Streams.All += Testers.ContainsExpression("CN = 
group.com", "Should select a group SAN")
-tr.Processes.Default.Streams.All += Testers.ExcludesExpression("unable to 
verify the first certificate", "Correct signer")
-
-# Should receive a EC cert
-tr = Test.AddTestRun("ec.com only in ec cert")
-tr.Processes.Default.Command = "echo foo | openssl s_client  -CAfile 
signer.pem -servername ec.com -connect 127.0.0.1:{0}".format(
-    ts.Variables.ssl_port)
-tr.ReturnCode = 0
-tr.StillRunningAfter = server
-tr.StillRunningAfter = ts
-tr.Processes.Default.Streams.All += Testers.ContainsExpression("Peer signature 
type: ECDSA", "Should select EC cert")
-tr.Processes.Default.Streams.All += Testers.ContainsExpression("CN = 
group.com", "Should select a group SAN")
-tr.Processes.Default.Streams.All += Testers.ExcludesExpression("unable to 
verify the first certificate", "Correct signer")
-
-# Copy in a new version of the foo.com cert.  Replace it with the version
-# signed by signer 2.  Wait at least a second to sure the file update time
-# differs
-trupdate = Test.AddTestRun("Update server bar cert file in place")
-trupdate.StillRunningAfter = ts
-trupdate.StillRunningAfter = server
-trupdate.Setup.CopyAs("ssl/signed2-foo.pem", ".", 
"{0}/signed-foo.pem".format(ts.Variables.SSLDir))
-# For some reason the Setup.CopyAs does not change the modification time, so 
we touch
-trupdate.Processes.Default.Command = 'touch 
{0}/signed-foo.pem'.format(ts.Variables.SSLDir)
-# Need to copy over the environment so traffic_ctl knows where to find the 
unix domain socket
-trupdate.Processes.Default.Env = ts.Env
-trupdate.Processes.Default.ReturnCode = 0
-
-# The plugin will pull every 3 seconds.  So wait 4 seconds and test again.  
Request with CA=signer2.pem should work.  Request with CA=signer.pem should fail
-# Should receive a RSA cert
-tr = Test.AddTestRun("Only offer RSA ciphers, should receive RSA cert")
-tr.Processes.Default.Command = "echo foo | openssl s_client -CAfile signer.pem 
 -servername foo.com -sigalgs 'RSA-PSS+SHA256' -connect 127.0.0.1:{0}".format(
-    ts.Variables.ssl_port)
-tr.DelayStart = 4
-tr.ReturnCode = 0
-tr.StillRunningAfter = server
-tr.StillRunningAfter = ts
-tr.Processes.Default.Streams.All += Testers.ContainsExpression("Peer signature 
type: RSA-PSS", "Should select RSA cert")
-tr.Processes.Default.Streams.All += Testers.ContainsExpression("CN = foo.com", 
"Should select foo.com")
-tr.Processes.Default.Streams.All += Testers.ContainsExpression("unable to 
verify the first certificate", "Different signer")
-
-tr = Test.AddTestRun("Only offer RSA ciphers, should receive RSA cert with 
correct CA")
-tr.Processes.Default.Command = "echo foo | openssl s_client -CAfile 
signer2.pem  -servername foo.com -sigalgs 'RSA-PSS+SHA256' -connect 
127.0.0.1:{0}".format(
-    ts.Variables.ssl_port)
-tr.ReturnCode = 0
-tr.StillRunningAfter = server
-tr.StillRunningAfter = ts
-tr.Processes.Default.Streams.All += Testers.ContainsExpression("Peer signature 
type: RSA-PSS", "Should select RSA cert")
-tr.Processes.Default.Streams.All += Testers.ContainsExpression("CN = foo.com", 
"Should select foo.com")
-tr.Processes.Default.Streams.All += Testers.ExcludesExpression("unable to 
verify the first certificate", "Correct signer")
-
-# The EC case should be unchanged
-tr = Test.AddTestRun("Offer any cipher")
-tr.Processes.Default.Command = "echo foo | openssl s_client -CAfile signer.pem 
 -servername foo.com  -connect 127.0.0.1:{0}".format(
-    ts.Variables.ssl_port)
-tr.ReturnCode = 0
-tr.StillRunningAfter = server
-tr.StillRunningAfter = ts
-tr.Processes.Default.Streams.All += Testers.ContainsExpression("Peer signature 
type: ECDSA", "Should select EC cert")
-tr.Processes.Default.Streams.All += Testers.ContainsExpression("CN = foo.com", 
"Should select foo.com")
-tr.Processes.Default.Streams.All += Testers.ExcludesExpression("unable to 
verify the first certificate", "Correct signer")
diff --git a/tests/gold_tests/tls/tls_client_cert2_plugin.test.py 
b/tests/gold_tests/tls/tls_client_cert2_plugin.test.py
deleted file mode 100644
index 4410e27c37..0000000000
--- a/tests/gold_tests/tls/tls_client_cert2_plugin.test.py
+++ /dev/null
@@ -1,191 +0,0 @@
-'''
-Test offering client cert to origin, but using plugin for cert loading
-'''
-#  Licensed to the Apache Software Foundation (ASF) under one
-#  or more contributor license agreements.  See the NOTICE file
-#  distributed with this work for additional information
-#  regarding copyright ownership.  The ASF licenses this file
-#  to you under the Apache License, Version 2.0 (the
-#  "License"); you may not use this file except in compliance
-#  with the License.  You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-#  Unless required by applicable law or agreed to in writing, software
-#  distributed under the License is distributed on an "AS IS" BASIS,
-#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-#  See the License for the specific language governing permissions and
-#  limitations under the License.
-
-import subprocess
-import os
-
-Test.Summary = '''
-Test offering client cert to origin, but using plugin for cert loading
-'''
-
-ts = Test.MakeATSProcess("ts", command="traffic_server", select_ports=True)
-cafile = "{0}/signer.pem".format(Test.RunDirectory)
-cafile2 = "{0}/signer2.pem".format(Test.RunDirectory)
-server = Test.MakeOriginServer("server",
-                               ssl=True,
-                               options={"--clientCA": cafile,
-                                        "--clientverify": ""},
-                               
clientcert="{0}/signed-foo.pem".format(Test.RunDirectory),
-                               
clientkey="{0}/signed-foo.key".format(Test.RunDirectory))
-server2 = Test.MakeOriginServer("server2",
-                                ssl=True,
-                                options={"--clientCA": cafile2,
-                                         "--clientverify": ""},
-                                
clientcert="{0}/signed2-bar.pem".format(Test.RunDirectory),
-                                
clientkey="{0}/signed-bar.key".format(Test.RunDirectory))
-server.Setup.Copy("ssl/signer.pem")
-server.Setup.Copy("ssl/signer2.pem")
-server.Setup.Copy("ssl/signed-foo.pem")
-server.Setup.Copy("ssl/signed-foo.key")
-server.Setup.Copy("ssl/signed2-foo.pem")
-server.Setup.Copy("ssl/signed2-bar.pem")
-server.Setup.Copy("ssl/signed-bar.key")
-server2.Setup.Copy("ssl/signer.pem")
-server2.Setup.Copy("ssl/signer2.pem")
-server2.Setup.Copy("ssl/signed-foo.pem")
-server2.Setup.Copy("ssl/signed-foo.key")
-server2.Setup.Copy("ssl/signed2-foo.pem")
-server2.Setup.Copy("ssl/signed2-bar.pem")
-server2.Setup.Copy("ssl/signed-bar.key")
-
-request_header = {"headers": "GET / HTTP/1.1\r\nHost: example.com\r\n\r\n", 
"timestamp": "1469733493.993", "body": ""}
-response_header = {"headers": "HTTP/1.1 200 OK\r\nConnection: close\r\n\r\n", 
"timestamp": "1469733493.993", "body": ""}
-server.addResponse("sessionlog.json", request_header, response_header)
-request_header = {"headers": "GET / HTTP/1.1\r\nHost: bar.com\r\n\r\n", 
"timestamp": "1469733493.993", "body": ""}
-response_header = {"headers": "HTTP/1.1 200 OK\r\nConnection: close\r\n\r\n", 
"timestamp": "1469733493.993", "body": ""}
-server.addResponse("sessionlog.json", request_header, response_header)
-
-#
-# Certs and keys loaded into the ts/ssl directory, but the paths in the
-# configs omit the ssl subdirectory.  The ssl_secret_load_test plugin adds 
this back in
-ts.addSSLfile("ssl/server.pem")
-ts.addSSLfile("ssl/server.key")
-ts.addSSLfile("ssl/combo-signed-foo.pem")
-ts.addSSLfile("ssl/signed-foo.pem")
-ts.addSSLfile("ssl/signed-foo.key")
-ts.addSSLfile("ssl/signed2-foo.pem")
-ts.addSSLfile("ssl/signed-bar.pem")
-ts.addSSLfile("ssl/signed2-bar.pem")
-ts.addSSLfile("ssl/signed-bar.key")
-
-Test.PrepareTestPlugin(os.path.join(Test.Variables.AtsTestPluginsDir, 
'ssl_secret_load_test.so'), ts)
-
-ts.Disk.records_config.update({
-    'proxy.config.diags.debug.enabled': 1,
-    'proxy.config.diags.debug.tags': 'ssl_secret_load_test',
-    'proxy.config.ssl.server.cert.path': '{0}'.format(ts.Variables.SSLDir),
-    'proxy.config.ssl.server.private_key.path': 
'{0}'.format(ts.Variables.SSLDir),
-    'proxy.config.ssl.client.cert.path': '{0}/../'.format(ts.Variables.SSLDir),
-    'proxy.config.ssl.client.private_key.path': 
'{0}'.format(ts.Variables.SSLDir),
-    'proxy.config.ssl.client.verify.server.policy': 'PERMISSIVE',
-    'proxy.config.exec_thread.autoconfig.scale': 1.0,
-    'proxy.config.url_remap.pristine_host_hdr': 1,
-})
-
-ts.Disk.ssl_multicert_config.AddLine(
-    'dest_ip=* ssl_cert_name=server.pem ssl_key_name=server.key'
-)
-
-ts.Disk.remap_config.AddLine(
-    'map /case1 https://127.0.0.1:{0}/'.format(server.Variables.SSL_Port)
-)
-ts.Disk.remap_config.AddLine(
-    'map /case2 https://127.0.0.1:{0}/'.format(server2.Variables.SSL_Port)
-)
-
-ts.Disk.sni_yaml.AddLines([
-    'sni:',
-    '- fqdn: bob.bar.com',
-    '  client_cert: {0}/../signed-bar.pem'.format(ts.Variables.SSLDir),
-    '  client_key: {0}/../signed-bar.key'.format(ts.Variables.SSLDir),
-    '- fqdn: bob.*.com',
-    '  client_cert: {0}/../combo-signed-foo.pem'.format(ts.Variables.SSLDir),
-    '- fqdn: "*bar.com"',
-    '  client_cert: {0}/../signed2-bar.pem'.format(ts.Variables.SSLDir),
-    '  client_key: {0}/../signed-bar.key'.format(ts.Variables.SSLDir),
-    '- fqdn: "foo.com"',
-    '  client_cert: {0}/../signed2-foo.pem'.format(ts.Variables.SSLDir),
-    '  client_key: {0}/../signed-foo.key'.format(ts.Variables.SSLDir),
-])
-
-
-# Should succeed
-tr = Test.AddTestRun("bob.bar.com to server 1")
-tr.Processes.Default.StartBefore(Test.Processes.ts, 
ready=When.PortOpen(ts.Variables.port))
-tr.Processes.Default.StartBefore(server)
-tr.Processes.Default.StartBefore(server2)
-tr.StillRunningAfter = ts
-tr.StillRunningAfter = server
-tr.StillRunningAfter = server2
-tr.Processes.Default.Command = "curl -H host:bob.bar.com  
http://127.0.0.1:{0}/case1".format(ts.Variables.port)
-tr.Processes.Default.ReturnCode = 0
-tr.Processes.Default.Streams.stdout = Testers.ExcludesExpression("Could Not 
Connect", "Check response")
-
-# Should fail
-trfail = Test.AddTestRun("bob.bar.com to server 2")
-trfail.StillRunningAfter = ts
-trfail.StillRunningAfter = server
-trfail.StillRunningAfter = server2
-trfail.Processes.Default.Command = 'curl -H host:bob.bar.com  
http://127.0.0.1:{0}/case2'.format(ts.Variables.port)
-trfail.Processes.Default.ReturnCode = 0
-trfail.Processes.Default.Streams.stdout = Testers.ContainsExpression("Could 
Not Connect", "Check response")
-
-# Should succeed
-tr = Test.AddTestRun("bob.foo.com to server 1")
-tr.StillRunningAfter = ts
-tr.StillRunningAfter = server
-tr.StillRunningAfter = server2
-tr.Processes.Default.Command = "curl -H host:bob.foo.com  
http://127.0.0.1:{0}/case1".format(ts.Variables.port)
-tr.Processes.Default.ReturnCode = 0
-tr.Processes.Default.Streams.stdout = Testers.ExcludesExpression("Could Not 
Connect", "Check response")
-
-# Should fail
-trfail = Test.AddTestRun("bob.foo.com to server 2")
-trfail.StillRunningAfter = ts
-trfail.StillRunningAfter = server
-trfail.StillRunningAfter = server2
-trfail.Processes.Default.Command = 'curl -H host:bob.foo.com  
http://127.0.0.1:{0}/case2'.format(ts.Variables.port)
-trfail.Processes.Default.ReturnCode = 0
-trfail.Processes.Default.Streams.stdout = Testers.ContainsExpression("Could 
Not Connect", "Check response")
-
-# Should succeed
-tr = Test.AddTestRun("random.bar.com to server 2")
-tr.StillRunningAfter = ts
-tr.StillRunningAfter = server
-tr.StillRunningAfter = server2
-tr.Processes.Default.Command = "curl -H host:random.bar.com  
http://127.0.0.1:{0}/case2".format(ts.Variables.port)
-tr.Processes.Default.ReturnCode = 0
-tr.Processes.Default.Streams.stdout = Testers.ExcludesExpression("Could Not 
Connect", "Check response")
-
-# Should fail
-trfail = Test.AddTestRun("random.bar.com to server 1")
-trfail.StillRunningAfter = ts
-trfail.StillRunningAfter = server
-trfail.StillRunningAfter = server2
-trfail.Processes.Default.Command = 'curl -H host:random.bar.com  
http://127.0.0.1:{0}/case1'.format(ts.Variables.port)
-trfail.Processes.Default.ReturnCode = 0
-trfail.Processes.Default.Streams.stdout = Testers.ContainsExpression("Could 
Not Connect", "Check response")
-
-# Should fail
-tr = Test.AddTestRun("random.foo.com to server 2")
-tr.StillRunningAfter = ts
-tr.StillRunningAfter = server
-tr.StillRunningAfter = server2
-tr.Processes.Default.Command = "curl -H host:random.foo.com  
http://127.0.0.1:{0}/case2".format(ts.Variables.port)
-tr.Processes.Default.ReturnCode = 0
-tr.Processes.Default.Streams.stdout = Testers.ContainsExpression("Could Not 
Connect", "Check response")
-
-# Should fail
-trfail = Test.AddTestRun("random.foo.com to server 1")
-trfail.StillRunningAfter = ts
-trfail.StillRunningAfter = server
-trfail.StillRunningAfter = server2
-trfail.Processes.Default.Command = 'curl -H host:random.foo.com  
http://127.0.0.1:{0}/case1'.format(ts.Variables.port)
-trfail.Processes.Default.ReturnCode = 0
-trfail.Processes.Default.Streams.stdout = Testers.ContainsExpression("Could 
Not Connect", "Check response")
diff --git a/tests/gold_tests/tls/tls_client_cert_override_plugin.test.py 
b/tests/gold_tests/tls/tls_client_cert_override_plugin.test.py
deleted file mode 100644
index 1aa46fa48d..0000000000
--- a/tests/gold_tests/tls/tls_client_cert_override_plugin.test.py
+++ /dev/null
@@ -1,227 +0,0 @@
-#  Licensed to the Apache Software Foundation (ASF) under one
-#  or more contributor license agreements.  See the NOTICE file
-#  distributed with this work for additional information
-#  regarding copyright ownership.  The ASF licenses this file
-#  to you under the Apache License, Version 2.0 (the
-#  "License"); you may not use this file except in compliance
-#  with the License.  You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-#  Unless required by applicable law or agreed to in writing, software
-#  distributed under the License is distributed on an "AS IS" BASIS,
-#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-#  See the License for the specific language governing permissions and
-#  limitations under the License.
-
-import subprocess
-import os
-Test.Summary = '''
-Test conf_remp to specify different client certificates to offer to the 
origin.  Loading certs/keys via plugin.
-'''
-
-
-ts = Test.MakeATSProcess("ts", command="traffic_manager", select_ports=True)
-cafile = "{0}/signer.pem".format(Test.RunDirectory)
-cafile2 = "{0}/signer2.pem".format(Test.RunDirectory)
-server = Test.MakeOriginServer("server",
-                               ssl=True,
-                               options={"--clientCA": cafile,
-                                        "--clientverify": ""},
-                               
clientcert="{0}/signed-foo.pem".format(Test.RunDirectory),
-                               
clientkey="{0}/signed-foo.key".format(Test.RunDirectory))
-server2 = Test.MakeOriginServer("server2",
-                                ssl=True,
-                                options={"--clientCA": cafile2,
-                                         "--clientverify": ""},
-                                
clientcert="{0}/signed2-bar.pem".format(Test.RunDirectory),
-                                
clientkey="{0}/signed-bar.key".format(Test.RunDirectory))
-server3 = Test.MakeOriginServer("server3")
-server.Setup.Copy("ssl/signer.pem")
-server.Setup.Copy("ssl/signer2.pem")
-server.Setup.Copy("ssl/signed-foo.pem")
-server.Setup.Copy("ssl/signed-foo.key")
-server.Setup.Copy("ssl/signed2-foo.pem")
-server.Setup.Copy("ssl/signed2-bar.pem")
-server.Setup.Copy("ssl/signed-bar.key")
-server2.Setup.Copy("ssl/signer.pem")
-server2.Setup.Copy("ssl/signer2.pem")
-server2.Setup.Copy("ssl/signed-foo.pem")
-server2.Setup.Copy("ssl/signed-foo.key")
-server2.Setup.Copy("ssl/signed2-foo.pem")
-server2.Setup.Copy("ssl/signed2-bar.pem")
-server2.Setup.Copy("ssl/signed-bar.key")
-
-request_header = {"headers": "GET / HTTP/1.1\r\nHost: example.com\r\n\r\n", 
"timestamp": "1469733493.993", "body": ""}
-response_header = {
-    "headers": "HTTP/1.1 200 OK\r\nConnection: close\r\nCache-Control: 
no-cache\r\n\r\n",
-    "timestamp": "1469733493.993",
-    "body": ""}
-server.addResponse("sessionlog.json", request_header, response_header)
-request_header = {"headers": "GET / HTTP/1.1\r\nHost: bar.com\r\n\r\n", 
"timestamp": "1469733493.993", "body": ""}
-response_header = {
-    "headers": "HTTP/1.1 200 OK\r\nCache-Control: no-cache\r\nConnection: 
close\r\n\r\n",
-    "timestamp": "1469733493.993",
-    "body": ""}
-server.addResponse("sessionlog.json", request_header, response_header)
-
-ts.addSSLfile("ssl/server.pem")
-ts.addSSLfile("ssl/server.key")
-ts.addSSLfile("ssl/signed-foo.pem")
-ts.addSSLfile("ssl/signed-foo.key")
-ts.addSSLfile("ssl/signed2-foo.pem")
-ts.addSSLfile("ssl/signed-bar.pem")
-ts.addSSLfile("ssl/signed2-bar.pem")
-ts.addSSLfile("ssl/signed-bar.key")
-
-ts.Disk.sni_yaml.AddLine('sni:')
-ts.Disk.sni_yaml.AddLine('- fqdn: random')
-ts.Disk.sni_yaml.AddLine('  verify_server_properties: NONE')
-snipath = ts.Disk.sni_yaml.AbsPath
-
-Test.PrepareTestPlugin(os.path.join(Test.Variables.AtsTestPluginsDir, 
'ssl_secret_load_test.so'), ts)
-
-shortdir = ts.Variables.SSLDir[0:ts.Variables.SSLDir.rfind("/")]
-
-ts.Disk.records_config.update({
-    'proxy.config.ssl.server.cert.path': '{0}'.format(ts.Variables.SSLDir),
-    'proxy.config.ssl.server.private_key.path': 
'{0}'.format(ts.Variables.SSLDir),
-    'proxy.config.ssl.client.verify.server.policy': 'PERMISSIVE',
-    'proxy.config.ssl.client.cert.path': '{0}'.format(shortdir),
-    'proxy.config.ssl.client.cert.filename': 'signed-foo.pem',
-    'proxy.config.ssl.client.private_key.path': '{0}'.format(shortdir),
-    'proxy.config.ssl.client.private_key.filename': 'signed-foo.key',
-    'proxy.config.exec_thread.autoconfig.scale': 1.0,
-    'proxy.config.url_remap.pristine_host_hdr': 1,
-    'proxy.config.diags.debug.enabled': 1,
-    'proxy.config.diags.debug.tags': 'ssl_secret_load|http|ssl',
-})
-
-ts.Disk.ssl_multicert_config.AddLine(
-    'dest_ip=* ssl_cert_name=server.pem ssl_key_name=server.key'
-)
-
-ts.Disk.remap_config.AddLine(
-    'map /case1 https://127.0.0.1:{0}/ @plugin=conf_remap.so 
@pparam=proxy.config.ssl.client.cert.filename={1} plugin=conf_remap.so 
@pparam=proxy.config.ssl.client.private_key.filename={2}'.format(
-        server.Variables.SSL_Port,
-        "signed-foo.pem",
-        "signed-foo.key"))
-ts.Disk.remap_config.AddLine(
-    'map /badcase1 https://127.0.0.1:{0}/ @plugin=conf_remap.so 
@pparam=proxy.config.ssl.client.cert.filename={1} plugin=conf_remap.so 
@pparam=proxy.config.ssl.client.private_key.filename={2}'.format(
-        server.Variables.SSL_Port,
-        "signed2-foo.pem",
-        "signed-foo.key"))
-ts.Disk.remap_config.AddLine(
-    'map /case2 https://127.0.0.1:{0}/ @plugin=conf_remap.so 
@pparam=proxy.config.ssl.client.cert.filename={1} plugin=conf_remap.so 
@pparam=proxy.config.ssl.client.private_key.filename={2}'.format(
-        server2.Variables.SSL_Port,
-        "signed2-foo.pem",
-        "signed-foo.key"))
-ts.Disk.remap_config.AddLine(
-    'map /badcase2 https://127.0.0.1:{0}/ @plugin=conf_remap.so 
@pparam=proxy.config.ssl.client.cert.filename={1} plugin=conf_remap.so 
@pparam=proxy.config.ssl.client.private_key.filename={2}'.format(
-        server2.Variables.SSL_Port,
-        "signed-foo.pem",
-        "signed-foo.key"))
-
-# Should succeed
-tr = Test.AddTestRun("Connect with correct client cert to first server")
-tr.Processes.Default.StartBefore(Test.Processes.ts)
-tr.Processes.Default.StartBefore(server)
-tr.Processes.Default.StartBefore(server2)
-tr.StillRunningAfter = ts
-tr.StillRunningAfter = server
-tr.StillRunningAfter = server2
-tr.Processes.Default.Command = "curl -H host:example.com  
http://127.0.0.1:{0}/case1".format(ts.Variables.port)
-tr.Processes.Default.ReturnCode = 0
-tr.Processes.Default.Streams.stdout = Testers.ExcludesExpression("Could Not 
Connect", "Check response")
-
-# Should fail
-trfail = Test.AddTestRun("Connect with bad client cert to first server")
-trfail.StillRunningAfter = ts
-trfail.StillRunningAfter = server
-trfail.StillRunningAfter = server2
-trfail.Processes.Default.Command = 'curl -H host:example.com  
http://127.0.0.1:{0}/badcase1'.format(ts.Variables.port)
-trfail.Processes.Default.ReturnCode = 0
-trfail.Processes.Default.Streams.stdout = Testers.ContainsExpression("Could 
Not Connect", "Check response")
-
-# Should succeed
-trbar = Test.AddTestRun("Connect with correct client cert to second server")
-trbar.StillRunningAfter = ts
-trbar.StillRunningAfter = server
-trbar.StillRunningAfter = server2
-trbar.Processes.Default.Command = "curl -H host:bar.com  
http://127.0.0.1:{0}/case2".format(ts.Variables.port)
-trbar.Processes.Default.ReturnCode = 0
-trbar.Processes.Default.Streams.stdout = Testers.ExcludesExpression("Could Not 
Connect", "Check response")
-
-# Should fail
-trbarfail = Test.AddTestRun("Connect with bad client cert to second server")
-trbarfail.StillRunningAfter = ts
-trbarfail.StillRunningAfter = server
-trbarfail.StillRunningAfter = server2
-trbarfail.Processes.Default.Command = 'curl -H host:bar.com  
http://127.0.0.1:{0}/badcase2'.format(ts.Variables.port)
-trbarfail.Processes.Default.ReturnCode = 0
-trbarfail.Processes.Default.Streams.stdout = Testers.ContainsExpression("Could 
Not Connect", "Check response")
-
-# Test the case of updating certificate contents without changing file name.
-trupdate = Test.AddTestRun("Update client cert file in place")
-trupdate.StillRunningAfter = ts
-trupdate.StillRunningAfter = server
-trupdate.StillRunningAfter = server2
-# in the config/ssl directory for records.config
-trupdate.Setup.CopyAs("ssl/signed-foo.pem", ".", 
"{0}/signed2-foo.pem".format(ts.Variables.SSLDir))
-trupdate.Processes.Default.Command = 'traffic_ctl config set 
proxy.config.ssl.client.cert.path {0}/; touch {1}'.format(
-    shortdir, snipath)
-# Need to copy over the environment so traffic_ctl knows where to find the 
unix domain socket
-trupdate.Processes.Default.Env = ts.Env
-trupdate.Processes.Default.ReturnCode = 0
-
-tr2reload = Test.AddTestRun("Reload config")
-tr2reload.StillRunningAfter = ts
-tr2reload.StillRunningAfter = server
-tr2reload.StillRunningAfter = server2
-tr2reload.Processes.Default.Command = 'traffic_ctl config reload'
-# Need to copy over the environment so traffic_ctl knows where to find the 
unix domain socket
-tr2reload.Processes.Default.Env = ts.Env
-tr2reload.Processes.Default.ReturnCode = 0
-
-tr3bar = Test.AddTestRun("Make request with other foo.  badcase1 should now 
work")
-# Wait for the reload to complete
-tr3bar.Processes.Default.StartBefore(server3, 
ready=When.FileContains(ts.Disk.diags_log.Name, 'sni.yaml finished loading', 3))
-tr3bar.StillRunningAfter = ts
-tr3bar.StillRunningAfter = server
-tr3bar.StillRunningAfter = server2
-tr3bar.Processes.Default.Command = 'curl  -H host:foo.com 
http://127.0.0.1:{0}/badcase1'.format(ts.Variables.port)
-tr3bar.Processes.Default.ReturnCode = 0
-tr3bar.Processes.Default.Streams.stdout = Testers.ExcludesExpression("Could 
Not Connect", "Check response")
-
-# Test the hot-reload feature.  Update cert file and wait about.  Should 
update without reload
-trupdate = Test.AddTestRun("Update signed-foo cert file in place")
-trupdate.StillRunningAfter = ts
-trupdate.StillRunningAfter = server
-trupdate.Setup.CopyAs("ssl/signed2-foo.pem", ".", 
"{0}/signed-foo.pem".format(ts.Variables.SSLDir))
-# For some reason the Setup.CopyAs does not change the modification time, so 
we touch
-trupdate.Processes.Default.Command = 'touch {0}/signed-foo.pem 
{0}/signed-foo.key'.format(ts.Variables.SSLDir)
-# Need to copy over the environment so traffic_ctl knows where to find the 
unix domain socket
-trupdate.Processes.Default.Env = ts.Env
-trupdate.Processes.Default.ReturnCode = 0
-
-# The plugin will pull every 3 seconds.  So wait 4 seconds and test again.
-# case1 should fail
-# badcase1 should succeed
-tr = Test.AddTestRun("Retest case1")
-tr.DelayStart = 4
-tr.ReturnCode = 0
-tr.StillRunningAfter = server
-tr.StillRunningAfter = server2
-tr.StillRunningAfter = ts
-tr.Processes.Default.Command = "curl -H host:example.com  
http://127.0.0.1:{0}/case1".format(ts.Variables.port)
-tr.Processes.Default.ReturnCode = 0
-tr.Processes.Default.Streams.stdout = Testers.ContainsExpression("Could Not 
Connect", "Check response")
-
-tr = Test.AddTestRun("Retest badcase1")
-tr.ReturnCode = 0
-tr.StillRunningAfter = server
-tr.StillRunningAfter = server2
-tr.StillRunningAfter = ts
-tr.Processes.Default.Command = "curl -H host:example.com  
http://127.0.0.1:{0}/badcase1".format(ts.Variables.port)
-tr.Processes.Default.ReturnCode = 0
-tr.Processes.Default.Streams.stdout = Testers.ExcludesExpression("Could Not 
Connect", "Check response")
diff --git a/tests/gold_tests/tls/tls_client_cert_plugin.test.py 
b/tests/gold_tests/tls/tls_client_cert_plugin.test.py
deleted file mode 100644
index e493b97a65..0000000000
--- a/tests/gold_tests/tls/tls_client_cert_plugin.test.py
+++ /dev/null
@@ -1,297 +0,0 @@
-'''
-Test offering client cert to origin, but using plugin for cert loading
-'''
-#  Licensed to the Apache Software Foundation (ASF) under one
-#  or more contributor license agreements.  See the NOTICE file
-#  distributed with this work for additional information
-#  regarding copyright ownership.  The ASF licenses this file
-#  to you under the Apache License, Version 2.0 (the
-#  "License"); you may not use this file except in compliance
-#  with the License.  You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-#  Unless required by applicable law or agreed to in writing, software
-#  distributed under the License is distributed on an "AS IS" BASIS,
-#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-#  See the License for the specific language governing permissions and
-#  limitations under the License.
-
-import subprocess
-import os
-
-Test.Summary = '''
-Test offering client cert to origin, but using plugin for cert loading
-'''
-
-ts = Test.MakeATSProcess("ts", command="traffic_manager", select_ports=True)
-cafile = "{0}/signer.pem".format(Test.RunDirectory)
-cafile2 = "{0}/signer2.pem".format(Test.RunDirectory)
-# --clientverify: "" empty string because microserver does store_true for 
argparse, but options is a dictionary
-server = Test.MakeOriginServer("server",
-                               ssl=True,
-                               options={"--clientCA": cafile,
-                                        "--clientverify": ""},
-                               
clientcert="{0}/signed-foo.pem".format(Test.RunDirectory),
-                               
clientkey="{0}/signed-foo.key".format(Test.RunDirectory))
-server2 = Test.MakeOriginServer("server2",
-                                ssl=True,
-                                options={"--clientCA": cafile2,
-                                         "--clientverify": ""},
-                                
clientcert="{0}/signed2-bar.pem".format(Test.RunDirectory),
-                                
clientkey="{0}/signed-bar.key".format(Test.RunDirectory))
-server3 = Test.MakeOriginServer("server3")
-server.Setup.Copy("ssl/signer.pem")
-server.Setup.Copy("ssl/signer2.pem")
-server.Setup.Copy("ssl/signed-foo.pem")
-server.Setup.Copy("ssl/signed-foo.key")
-server.Setup.Copy("ssl/signed2-foo.pem")
-server.Setup.Copy("ssl/signed2-bar.pem")
-server.Setup.Copy("ssl/signed-bar.key")
-server2.Setup.Copy("ssl/signer.pem")
-server2.Setup.Copy("ssl/signer2.pem")
-server2.Setup.Copy("ssl/signed-foo.pem")
-server2.Setup.Copy("ssl/signed-foo.key")
-server2.Setup.Copy("ssl/signed2-foo.pem")
-server2.Setup.Copy("ssl/signed2-bar.pem")
-server2.Setup.Copy("ssl/signed-bar.key")
-
-request_header = {"headers": "GET / HTTP/1.1\r\nHost: example.com\r\n\r\n", 
"timestamp": "1469733493.993", "body": ""}
-response_header = {"headers": "HTTP/1.1 200 OK\r\nConnection: close\r\n\r\n", 
"timestamp": "1469733493.993", "body": ""}
-server.addResponse("sessionlog.json", request_header, response_header)
-request_header = {"headers": "GET / HTTP/1.1\r\nHost: bar.com\r\n\r\n", 
"timestamp": "1469733493.993", "body": ""}
-response_header = {"headers": "HTTP/1.1 200 OK\r\nConnection: close\r\n\r\n", 
"timestamp": "1469733493.993", "body": ""}
-server.addResponse("sessionlog.json", request_header, response_header)
-
-#
-# Certs and keys loaded into the ts/ssl directory, but the paths in the
-# configs omit the ssl subdirectory.  The ssl_secret_load_test plugin adds 
this back in
-ts.addSSLfile("ssl/server.pem")
-ts.addSSLfile("ssl/server.key")
-ts.addSSLfile("ssl/signed-foo.pem")
-ts.addSSLfile("ssl/signed-foo.key")
-ts.addSSLfile("ssl/signed2-foo.pem")
-ts.addSSLfile("ssl/signed-bar.pem")
-ts.addSSLfile("ssl/signed2-bar.pem")
-ts.addSSLfile("ssl/signed-bar.key")
-
-Test.PrepareTestPlugin(os.path.join(Test.Variables.AtsTestPluginsDir, 
'ssl_secret_load_test.so'), ts)
-
-ts.Disk.records_config.update({
-    'proxy.config.diags.debug.enabled': 1,
-    'proxy.config.diags.debug.tags': 'ssl_secret_load_test|ssl',
-    'proxy.config.ssl.server.cert.path': '{0}'.format(ts.Variables.SSLDir),
-    'proxy.config.ssl.server.private_key.path': 
'{0}'.format(ts.Variables.SSLDir),
-    'proxy.config.ssl.client.verify.server.policy': 'PERMISSIVE',
-    'proxy.config.ssl.client.cert.path': '{0}/../'.format(ts.Variables.SSLDir),
-    'proxy.config.ssl.client.cert.filename': 'signed-foo.pem',
-    'proxy.config.ssl.client.private_key.path': 
'{0}/../'.format(ts.Variables.SSLDir),
-    'proxy.config.ssl.client.private_key.filename': 'signed-foo.key',
-    'proxy.config.exec_thread.autoconfig.scale': 1.0,
-    'proxy.config.url_remap.pristine_host_hdr': 1,
-})
-
-ts.Disk.ssl_multicert_config.AddLine(
-    'dest_ip=* ssl_cert_name=server.pem ssl_key_name=server.key'
-)
-
-ts.Disk.remap_config.AddLine(
-    'map /case1 https://127.0.0.1:{0}/'.format(server.Variables.SSL_Port)
-)
-ts.Disk.remap_config.AddLine(
-    'map /case2 https://127.0.0.1:{0}/'.format(server2.Variables.SSL_Port)
-)
-
-ts.Disk.sni_yaml.AddLine(
-    'sni:')
-ts.Disk.sni_yaml.AddLine(
-    '- fqdn: bar.com')
-ts.Disk.sni_yaml.AddLine(
-    '  client_cert: {0}/../signed2-bar.pem'.format(ts.Variables.SSLDir))
-ts.Disk.sni_yaml.AddLine(
-    '  client_key: {0}/../signed-bar.key'.format(ts.Variables.SSLDir))
-
-
-# Should succeed
-tr = Test.AddTestRun("Connect with first client cert to first server")
-tr.Processes.Default.StartBefore(Test.Processes.ts)
-tr.Processes.Default.StartBefore(server)
-tr.Processes.Default.StartBefore(server2)
-tr.StillRunningAfter = ts
-tr.StillRunningAfter = server
-tr.StillRunningAfter = server2
-tr.Processes.Default.Command = "curl -H host:example.com  
http://127.0.0.1:{0}/case1".format(ts.Variables.port)
-tr.Processes.Default.ReturnCode = 0
-tr.Processes.Default.Streams.stdout = Testers.ExcludesExpression("Could Not 
Connect", "Check response")
-
-# Should fail
-trfail = Test.AddTestRun("Connect with first client cert to second server")
-trfail.StillRunningAfter = ts
-trfail.StillRunningAfter = server
-trfail.StillRunningAfter = server2
-trfail.Processes.Default.Command = 'curl -H host:example.com  
http://127.0.0.1:{0}/case2'.format(ts.Variables.port)
-trfail.Processes.Default.ReturnCode = 0
-trfail.Processes.Default.Streams.stdout = Testers.ContainsExpression("Could 
Not Connect", "Check response")
-
-# Should succeed
-trbar = Test.AddTestRun("Connect with signed2 bar to second server")
-trbar.StillRunningAfter = ts
-trbar.StillRunningAfter = server
-trbar.StillRunningAfter = server2
-trbar.Processes.Default.Command = "curl -H host:bar.com  
http://127.0.0.1:{0}/case2".format(ts.Variables.port)
-trbar.Processes.Default.ReturnCode = 0
-trbar.Processes.Default.Streams.stdout = Testers.ExcludesExpression("Could Not 
Connect", "Check response")
-
-# Should fail
-trbarfail = Test.AddTestRun("Connect with signed2 bar cert to first server")
-trbarfail.StillRunningAfter = ts
-trbarfail.StillRunningAfter = server
-trbarfail.StillRunningAfter = server2
-trbarfail.Processes.Default.Command = 'curl -H host:bar.com  
http://127.0.0.1:{0}/case1'.format(ts.Variables.port)
-trbarfail.Processes.Default.ReturnCode = 0
-trbarfail.Processes.Default.Streams.stdout = Testers.ContainsExpression("Could 
Not Connect", "Check response")
-
-tr2 = Test.AddTestRun("Update config files")
-# Update the SNI config
-snipath = ts.Disk.sni_yaml.AbsPath
-recordspath = ts.Disk.records_config.AbsPath
-tr2.Disk.File(snipath, id="sni_yaml", typename="ats:config"),
-tr2.Disk.sni_yaml.AddLine(
-    'sni:')
-tr2.Disk.sni_yaml.AddLine(
-    '- fqdn: bar.com')
-tr2.Disk.sni_yaml.AddLine(
-    '  client_cert: {0}/../signed-bar.pem'.format(ts.Variables.SSLDir))
-tr2.Disk.sni_yaml.AddLine(
-    '  client_key: {0}/../signed-bar.key'.format(ts.Variables.SSLDir))
-# recreate the records.config with the cert filename changed
-tr2.Disk.File(recordspath, id="records_config", typename="ats:config:records"),
-tr2.Disk.records_config.update({
-    'proxy.config.ssl.server.cert.path': '{0}'.format(ts.Variables.SSLDir),
-    'proxy.config.ssl.server.private_key.path': 
'{0}'.format(ts.Variables.SSLDir),
-    'proxy.config.ssl.client.verify.server.policy': 'PERMISSIVE',
-    'proxy.config.ssl.client.cert.path': '{0}/../'.format(ts.Variables.SSLDir),
-    'proxy.config.ssl.client.cert.filename': 'signed2-foo.pem',
-    'proxy.config.ssl.client.private_key.path': 
'{0}/../'.format(ts.Variables.SSLDir),
-    'proxy.config.ssl.client.private_key.filename': 'signed-foo.key',
-    'proxy.config.url_remap.pristine_host_hdr': 1,
-    'proxy.config.diags.debug.enabled': 1,
-    'proxy.config.diags.debug.tags': 'ssl_secret_load_test|ssl',
-})
-tr2.StillRunningAfter = ts
-tr2.StillRunningAfter = server
-tr2.StillRunningAfter = server2
-tr2.Processes.Default.Command = 'echo Updated configs'
-# Need to copy over the environment so traffic_ctl knows where to find the 
unix domain socket
-tr2.Processes.Default.Env = ts.Env
-tr2.Processes.Default.ReturnCode = 0
-
-tr2reload = Test.AddTestRun("Reload config")
-tr2reload.StillRunningAfter = ts
-tr2reload.StillRunningAfter = server
-tr2reload.StillRunningAfter = server2
-tr2reload.Processes.Default.Command = 'traffic_ctl config reload'
-# Need to copy over the environment so traffic_ctl knows where to find the 
unix domain socket
-tr2reload.Processes.Default.Env = ts.Env
-tr2reload.Processes.Default.ReturnCode = 0
-
-# Should succeed
-tr3bar = Test.AddTestRun("Make request with other bar cert to first server")
-# Wait for the reload to complete
-tr3bar.Processes.Default.StartBefore(server3, 
ready=When.FileContains(ts.Disk.diags_log.Name, 'sni.yaml finished loading', 3))
-tr3bar.StillRunningAfter = ts
-tr3bar.StillRunningAfter = server
-tr3bar.StillRunningAfter = server2
-tr3bar.Processes.Default.Command = 'curl  -H host:bar.com 
http://127.0.0.1:{0}/case1'.format(ts.Variables.port)
-tr3bar.Processes.Default.ReturnCode = 0
-tr3bar.Processes.Default.Streams.stdout = Testers.ExcludesExpression("Could 
Not Connect", "Check response")
-
-# Should fail
-tr3barfail = Test.AddTestRun("Make request with other bar cert to second 
server")
-tr3barfail.StillRunningAfter = ts
-tr3barfail.StillRunningAfter = server
-tr3barfail.StillRunningAfter = server2
-tr3barfail.Processes.Default.Command = 'curl  -H host:bar.com 
http://127.0.0.1:{0}/case2'.format(ts.Variables.port)
-tr3barfail.Processes.Default.ReturnCode = 0
-tr3barfail.Processes.Default.Streams.stdout = 
Testers.ContainsExpression("Could Not Connect", "Check response")
-
-# Should succeed
-tr3 = Test.AddTestRun("Make request with other cert to second server")
-# Wait for the reload to complete
-tr3.StillRunningAfter = ts
-tr3.StillRunningAfter = server
-tr3.StillRunningAfter = server2
-tr3.Processes.Default.Command = 'curl  -H host:example.com 
http://127.0.0.1:{0}/case2'.format(ts.Variables.port)
-tr3.Processes.Default.ReturnCode = 0
-tr3.Processes.Default.Streams.stdout = Testers.ExcludesExpression("Could Not 
Connect", "Check response")
-
-# Should fail
-tr3fail = Test.AddTestRun("Make request with other cert to first server")
-tr3fail.StillRunningAfter = ts
-tr3fail.StillRunningAfter = server
-tr3fail.StillRunningAfter = server2
-tr3fail.Processes.Default.Command = 'curl  -H host:example.com 
http://127.0.0.1:{0}/case1'.format(ts.Variables.port)
-tr3fail.Processes.Default.ReturnCode = 0
-tr3fail.Processes.Default.Streams.stdout = Testers.ContainsExpression("Could 
Not Connect", "Check response")
-
-
-# Test the case of updating certificate contents without changing file name.
-trupdate = Test.AddTestRun("Update client cert file in place")
-trupdate.StillRunningAfter = ts
-trupdate.StillRunningAfter = server
-trupdate.StillRunningAfter = server2
-# Make a meaningless config change on the path so the records.config reload 
logic will trigger
-trupdate.Setup.CopyAs("ssl/signed2-bar.pem", ".", 
"{0}/signed-bar.pem".format(ts.Variables.SSLDir))
-# in the config/ssl directory for records.config
-trupdate.Setup.CopyAs("ssl/signed-foo.pem", ".", 
"{0}/signed2-foo.pem".format(ts.Variables.SSLDir))
-trupdate.Processes.Default.Command = 'traffic_ctl config set 
proxy.config.ssl.client.cert.path {0}/../; touch {1}; touch {2}'.format(
-    ts.Variables.SSLDir, snipath, recordspath)
-# Need to copy over the environment so traffic_ctl knows where to find the 
unix domain socket
-trupdate.Processes.Default.Env = ts.Env
-trupdate.Processes.Default.ReturnCode = 0
-
-trreload = Test.AddTestRun("Reload config after renaming certs")
-trreload.StillRunningAfter = ts
-trreload.StillRunningAfter = server
-trreload.StillRunningAfter = server2
-trreload.Processes.Default.Command = 'traffic_ctl config reload'
-trreload.Processes.Default.Env = ts.Env
-trreload.Processes.Default.ReturnCode = 0
-
-# Should succeed
-tr4bar = Test.AddTestRun("Make request with renamed bar cert to second server")
-# Wait for the reload to complete
-tr4bar.DelayStart = 10
-tr4bar.StillRunningAfter = ts
-tr4bar.StillRunningAfter = server
-tr4bar.StillRunningAfter = server2
-tr4bar.Processes.Default.Command = 'curl  -H host:bar.com 
http://127.0.0.1:{0}/case2'.format(ts.Variables.port)
-tr4bar.Processes.Default.ReturnCode = 0
-tr4bar.Processes.Default.Streams.stdout = Testers.ExcludesExpression("Could 
Not Connect", "Check response")
-
-# Should fail
-tr4barfail = Test.AddTestRun("Make request with renamed bar cert to first 
server")
-tr4barfail.StillRunningAfter = ts
-tr4barfail.StillRunningAfter = server
-tr4barfail.StillRunningAfter = server2
-tr4barfail.Processes.Default.Command = 'curl  -H host:bar.com 
http://127.0.0.1:{0}/case1'.format(ts.Variables.port)
-tr4barfail.Processes.Default.ReturnCode = 0
-tr4barfail.Processes.Default.Streams.stdout = 
Testers.ContainsExpression("Could Not Connect", "Check response")
-
-# Should succeed
-tr4 = Test.AddTestRun("Make request with renamed foo cert to first server")
-tr4.StillRunningAfter = ts
-tr4.StillRunningAfter = server
-tr4.StillRunningAfter = server2
-tr4.Processes.Default.Command = 'curl  -H host:example.com 
http://127.0.0.1:{0}/case1'.format(ts.Variables.port)
-tr4.Processes.Default.ReturnCode = 0
-tr4.Processes.Default.Streams.stdout = Testers.ExcludesExpression("Could Not 
Connect", "Check response")
-
-# Should fail
-tr4fail = Test.AddTestRun("Make request with renamed foo cert to second 
server")
-tr4fail.StillRunningAfter = ts
-tr4fail.StillRunningAfter = server
-tr4fail.StillRunningAfter = server2
-tr4fail.Processes.Default.Command = 'curl  -H host:example.com 
http://127.0.0.1:{0}/case2'.format(ts.Variables.port)
-tr4fail.Processes.Default.ReturnCode = 0
-tr4fail.Processes.Default.Streams.stdout = Testers.ContainsExpression("Could 
Not Connect", "Check response")
diff --git a/tests/gold_tests/tls/tls_keepalive.test.py 
b/tests/gold_tests/tls/tls_keepalive.test.py
index a18d7bba95..ac51424781 100644
--- a/tests/gold_tests/tls/tls_keepalive.test.py
+++ b/tests/gold_tests/tls/tls_keepalive.test.py
@@ -66,7 +66,7 @@ logging:
 '''.split("\n")
 )
 
-Test.PrepareTestPlugin(os.path.join(Test.Variables.AtsTestPluginsDir, 
'ssl_secret_load_test.so'), ts)
+Test.PrepareTestPlugin(os.path.join(Test.Variables.AtsTestPluginsDir, 
'ssl_hook_test.so'), ts, '-preaccept=1')
 
 tr = Test.AddTestRun("Test two HTTP/1.1 requests over one TLS connection")
 tr.Processes.Default.StartBefore(server)
diff --git a/tests/tools/plugins/Makefile.inc b/tests/tools/plugins/Makefile.inc
index fd154a4a0e..e67b2ca6f3 100644
--- a/tests/tools/plugins/Makefile.inc
+++ b/tests/tools/plugins/Makefile.inc
@@ -58,9 +58,6 @@ tools_plugins_ssl_client_verify_test_la_SOURCES = 
tools/plugins/ssl_client_verif
 noinst_LTLIBRARIES += tools/plugins/ssl_hook_test.la
 tools_plugins_ssl_hook_test_la_SOURCES = tools/plugins/ssl_hook_test.cc
 
-noinst_LTLIBRARIES += tools/plugins/ssl_secret_load_test.la
-tools_plugins_ssl_secret_load_test_la_SOURCES = 
tools/plugins/ssl_secret_load_test.cc
-
 noinst_LTLIBRARIES += tools/plugins/ssl_verify_test.la
 tools_plugins_ssl_verify_test_la_SOURCES = tools/plugins/ssl_verify_test.cc
 
diff --git a/tests/tools/plugins/ssl_secret_load_test.cc 
b/tests/tools/plugins/ssl_secret_load_test.cc
deleted file mode 100644
index 654a717dbb..0000000000
--- a/tests/tools/plugins/ssl_secret_load_test.cc
+++ /dev/null
@@ -1,184 +0,0 @@
-/** @file
-
-  SSL Preaccept test plugin
-  Implements blind tunneling based on the client IP address
-  The client ip addresses are specified in the plugin's
-  config file as an array of IP addresses or IP address ranges under the
-  key "client-blind-tunnel"
-
-  @section license License
-
-  Licensed to the Apache Software Foundation (ASF) under one
-  or more contributor license agreements.  See the NOTICE file
-  distributed with this work for additional information
-  regarding copyright ownership.  The ASF licenses this file
-  to you under the Apache License, Version 2.0 (the
-  "License"); you may not use this file except in compliance
-  with the License.  You may obtain a copy of the License at
-
-      http://www.apache.org/licenses/LICENSE-2.0
-
-  Unless required by applicable law or agreed to in writing, software
-  distributed under the License is distributed on an "AS IS" BASIS,
-  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-  See the License for the specific language governing permissions and
-  limitations under the License.
- */
-
-#include <ts/ts.h>
-#include <openssl/ssl.h>
-#include <strings.h>
-#include <cstring>
-#include <unistd.h>
-#include <sys/types.h>
-#include <sys/stat.h>
-#include <fcntl.h>
-#include <string>
-#include <unordered_map>
-#include <vector>
-
-#define PN "ssl_secret_load_test"
-#define PCP "[" PN " Plugin] "
-
-// Map of secret name to last modified time
-std::unordered_map<std::string, time_t> secret_versions;
-
-void
-update_file_name(const std::string_view &path, std::string &newname)
-{
-  // insert the "ssl" directory into the path
-  auto offset = path.find_last_of("/");
-  if (offset == std::string::npos) {
-    newname = "ssl/";
-    newname.append(path);
-  } else {
-    newname = path.substr(0, offset + 1);
-    newname.append("ssl/");
-    newname.append(path.substr(offset + 1));
-  }
-}
-
-bool
-load_file(const std::string &newname, struct stat *statdata, std::string 
&data_item)
-{
-  if (stat(newname.c_str(), statdata) < 0) {
-    return false;
-  }
-
-  int fd = open(newname.c_str(), O_RDONLY);
-  if (fd < 0) {
-    TSDebug(PN, "Failed to load %s", newname.c_str());
-    return false;
-  }
-  size_t total_size = statdata->st_size;
-  data_item.resize(total_size);
-  size_t offset = 0;
-  char *data    = data_item.data();
-  while (offset < total_size) {
-    int num_read = read(fd, data + offset, total_size - offset);
-    if (num_read < 0) {
-      close(fd);
-      return false;
-    }
-    offset += num_read;
-  }
-  close(fd);
-  return true;
-}
-
-int
-CB_Load_Secret(TSCont cont, TSEvent event, void *edata)
-{
-  TSSecretID *id = reinterpret_cast<TSSecretID *>(edata);
-
-  TSDebug(PN, "Load secret for %*.s", static_cast<int>(id->cert_name_len), 
id->cert_name);
-
-  std::string newname;
-  std::string data_item;
-  struct stat statdata;
-
-  update_file_name(std::string_view{id->cert_name, id->cert_name_len}, 
newname);
-
-  TSDebug(PN, "Really load secret for %s", newname.c_str());
-
-  // Load the secret and add it to the map
-  if (!load_file(newname, &statdata, data_item)) {
-    return TS_ERROR;
-  }
-  secret_versions.insert(std::make_pair(std::string{id->cert_name, 
id->cert_name_len}, statdata.st_mtime));
-
-  TSSslSecretSet(id->cert_name, id->cert_name_len, data_item.data(), 
data_item.size());
-
-  if (id->key_name_len > 0) {
-    TSDebug(PN, "Load secret for %*.s", static_cast<int>(id->key_name_len), 
id->key_name);
-    update_file_name(std::string_view{id->key_name, id->key_name_len}, 
newname);
-
-    TSDebug(PN, "Really load secret for %s", newname.c_str());
-
-    // Load the secret and add it to the map
-    if (!load_file(newname, &statdata, data_item)) {
-      return TS_ERROR;
-    }
-    secret_versions.insert(std::make_pair(std::string{id->key_name, 
id->key_name_len}, statdata.st_mtime));
-
-    TSSslSecretSet(id->key_name, id->key_name_len, data_item.data(), 
data_item.size());
-  }
-
-  return TS_SUCCESS;
-}
-
-int
-CB_Update_Secret(TSCont cont, TSEvent event, void *edata)
-{
-  std::vector<std::string> updates;
-  for (auto iter = secret_versions.begin(); iter != secret_versions.end(); 
++iter) {
-    std::string newname;
-    std::string data_item;
-    struct stat statdata;
-
-    update_file_name(iter->first, newname);
-    TSDebug(PN, "check secret for %s, really %s", iter->first.c_str(), 
newname.c_str());
-
-    if (stat(newname.c_str(), &statdata) < 0) {
-      continue;
-    }
-
-    if (statdata.st_mtime > iter->second) {
-      TSDebug(PN, "check secret %s has been updated", newname.c_str());
-      if (!load_file(newname, &statdata, data_item)) {
-        continue;
-      }
-      TSSslSecretSet(iter->first.c_str(), iter->first.length(), 
data_item.data(), data_item.size());
-      updates.push_back(iter->first);
-      iter->second = statdata.st_mtime;
-    }
-  }
-  for (auto name : updates) {
-    TSDebug(PN, "update cert for secret %s", name.c_str());
-    TSSslSecretUpdate(name.c_str(), name.length());
-  }
-  TSContScheduleOnPool(cont, 3000, TS_THREAD_POOL_TASK);
-  return TS_SUCCESS;
-}
-
-// Called by ATS as our initialization point
-void
-TSPluginInit(int argc, const char *argv[])
-{
-  TSPluginRegistrationInfo info;
-  info.plugin_name   = const_cast<char *>("SSL secret load test");
-  info.vendor_name   = const_cast<char *>("apache");
-  info.support_email = const_cast<char *>("[email protected]");
-  if (TSPluginRegister(&info) != TS_SUCCESS) {
-    TSError("[%s] Plugin registration failed", PN);
-  }
-
-  TSCont cb = TSContCreate(&CB_Load_Secret, nullptr);
-  TSLifecycleHookAdd(TS_LIFECYCLE_SSL_SECRET_HOOK, cb);
-
-  // Scheduled a call back to trigger every 3 seconds to look for changes to 
the files
-  TSCont cb_update = TSContCreate(&CB_Update_Secret, TSMutexCreate());
-  TSContScheduleOnPool(cb_update, 3000, TS_THREAD_POOL_TASK);
-
-  return;
-}


Reply via email to