This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 2086cc46c88 [improve][pip] PIP-337: SSL Factory Plugin to customize
SSL Context and SSL Engine generation (#22016)
2086cc46c88 is described below
commit 2086cc46c882df7fb2855a3cdb2580e1bc3adc5b
Author: Apurva007 <[email protected]>
AuthorDate: Thu Jul 4 00:22:19 2024 -0700
[improve][pip] PIP-337: SSL Factory Plugin to customize SSL Context and SSL
Engine generation (#22016)
Co-authored-by: Apurva Telang <[email protected]>
---
pip/pip-337.md | 382 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 382 insertions(+)
diff --git a/pip/pip-337.md b/pip/pip-337.md
new file mode 100644
index 00000000000..283bb9710de
--- /dev/null
+++ b/pip/pip-337.md
@@ -0,0 +1,382 @@
+# PIP-337: SSL Factory Plugin to customize SSLContext/SSLEngine generation
+
+# Background knowledge
+Apache Pulsar supports TLS encrypted communication between the clients and
servers. The TLS encryption setup requires
+loading the TLS certificates and its respective passwords to generate the SSL
Context. Pulsar supports loading these
+certificates and passwords via the filesystem. It supports both Java based
Keystores/Truststores and TLS information in
+".crt", ".pem" & ".key" formats. This information is refreshed based on a
configurable interval.
+
+Apache Pulsar internally uses 3 different frameworks for connection management:
+
+- Netty: Connection management for Pulsar server and client that understands
Pulsar binary protocol.
+- Jetty: HTTP Server creation for Pulsar Admin and websocket. Jetty Client is
used by proxy for admin client calls.
+- AsyncHttpClient: HTTP Client creation for Admin client and HTTP Lookup
+
+Each of the above frameworks supports customizing the generation of the SSL
Context and SSL Engine. Currently, Pulsar
+uses these features to feed the SSL Context via its internal security tools
after loading the file based certificates.
+One of the issues of using these features is that pulsar tries to bootstrap
the SSL Context in multiple ways to suit
+each framework and file type.
+
+```mermaid
+flowchart TB
+ Proxy.DirectProxyHandler --> NettyClientSslContextRefresher
+ Proxy.DirectProxyHandler --> NettySSLContextAutoRefreshBuilder
+ Proxy.AdminProxyHandler --> KeyStoreSSLContext
+ Proxy.AdminProxyHandler --> SecurityUtility
+ Proxy.ServiceChannelInitializer --> NettySSLContextAutoRefreshBuilder
+ Proxy.ServiceChannelInitializer --> NettyServerSslContextBuilder
+ Broker.PulsarChannelInitializer --> NettyServerSslContextBuilder
+ Broker.PulsarChannelInitializer --> NettySSLContextAutoRefreshBuilder
+ Client.PulsarChannelInitializer --> NettySSLContextAutoRefreshBuilder
+ Client.PulsarChannelInitializer --> SecurityUtility
+ Broker.WebService --> JettySSlContextFactory
+ Proxy.WebServer --> JettySSlContextFactory
+ PulsarAdmin --> AsyncHttpConnector
+ AsyncHttpConnector --> KeyStoreSSLContext
+ AsyncHttpConnector --> SecurityUtility
+ JettySSlContextFactory --> NetSslContextBuilder
+ JettySSlContextFactory --> DefaultSslContextBuilder
+ NettyClientSslContextRefresher -.-> SslContextAutoRefreshBuilder
+ NettySSLContextAutoRefreshBuilder -.-> SslContextAutoRefreshBuilder
+ NettyServerSslContextBuilder -.-> SslContextAutoRefreshBuilder
+ NetSslContextBuilder -.-> SslContextAutoRefreshBuilder
+ DefaultSslContextBuilder -.-> SslContextAutoRefreshBuilder
+ Client.HttpLookup.HttpClient --> KeyStoreSSLContext
+ Client.HttpLookup.HttpClient --> SecurityUtility
+ SecurityUtility -.-> KeyManagerProxy
+ SecurityUtility -.-> TrustManagerProxy
+```
+The above diagram is an example of the complexity of the TLS encryption setup
within Pulsar. The above diagram only
+contains the basic components of Pulsar excluding Websockets, Functions, etc.
+
+Pulsar uses 2 base classes to load the TLS information.
+
+- `SecurityUtility`: It loads files of type ".crt", ".pem" and ".key" and
converts it into SSL Context. This SSL Context
+can be of type `io.netty.handler.ssl.SslContext` or `javax.net.ssl.SSLContext`
based on the caller. Security Utility
+can be used to create SSL Context that internally has KeyManager and
Trustmanager proxies that load cert changes
+dynamically.
+- `KeyStoreSSLContext`: It loads files of type Java Keystore/Truststore and
converts it into SSL Context. This SSL
+Context will be of type `javax.net.ssl.SSLContext`. This is always used to
create the SSL Engine.
+
+Each of the above classes are either directly used by Pulsar Clients or used
via implementations of the abstract class
+`SslContextAutoRefreshBuilder`.
+
+- `SslContextAutoRefreshBuilder` - This abstract class is used to refresh
certificates at a configurable interval. It
+internally provides a public API to return the SSL Context.
+
+There are several implementations of the above abstract class to suit the
needs of each of the framework and the
+respective TLS certificate files:
+
+- `NettyClientSslContextRefresher` - It internally creates the
`io.netty.handler.ssl.SslContext` using the ".crt",
+".pem" and ".key" files for the proxy client.
+- `NettySSLContextAutoRefreshBuilder` - It internally creates the
`KeyStoreSSLContext` using the Java Keystores.
+- `NettyServerSslContextBuilder` - It internally creates the
`io.netty.handler.ssl.SslContext` using the ".crt",
+ ".pem" and ".key" files for the server.
+- `NetSslContextBuilder` - It internally creates the
`javax.net.ssl.SSLContext` using the Java Keystores for the web
+server.
+- `DefaultSslContextBuilder` - It internally creates the
`javax.net.ssl.SSLContext` using the ".crt", ".pem" and ".key"
+files for the web server.
+
+# Motivation
+Apache Pulsar's TLS encryption configuration is not pluggable. It only
supports file-based certificates. This makes
+Pulsar difficult to adopt for organizations that require loading TLS
certificates by other mechanisms.
+
+# Goals
+The purpose of this PIP is to introduce the following:
+
+- Provide a mechanism to plugin a custom SSL Factory that can generate SSL
Context and SSL Engine.
+- Simplify the Pulsar code base to universally use `javax.net.ssl.SSLContext`
and reduce the amount of code required to
+build and configure the SSL context taking into consideration backwards
compatibility.
+
+## In Scope
+
+- Creation of a new interface `PulsarSslFactory` that can generate a SSL
Context, Client SSL Engine and Server SSL
+Engine.
+- Creation of a default implementation of `PulsarSslFactory` that supports
loading the SSL Context and SSL Engine via
+file-based certificates. Internally it will use the SecurityUtility and
KeyStoreSSLContext.
+- Creation of a new class called "PulsarSslConfiguration" to store the ssl
configuration parameters which will be passed
+to the SSL Factory.
+- Modify the Pulsar Components to support the `PulsarSslFactory` instead of
the SslContextAutoRefreshBuilder, SecurityUtility
+and KeyStoreSSLContext.
+- Remove the SslContextAutoRefreshBuilder and all its implementations.
+- SSL Context refresh will be moved out of the factory. The responsibility of
refreshing the ssl context will lie with
+the components using the factory.
+- The factory will not be thread safe. We are isolating responsibilities by
having a single thread perform all writes,
+while all channel initializer threads will perform only reads. SSL Context
reads can be eventually consistent.
+- Each component calling the factory will internally initialize it as part of
the constructor as well as create the
+ssl context at startup as a blocking call. If this creation/initialization
fails then it will cause the Pulsar
+Component to shutdown. This is true for all components except the Pulsar
client due to past contracts where
+authentication provider may not have started before the client.
+- Each component will re-use its scheduled executor provider to schedule the
refresh of the ssl context based on its
+component's certificate refresh configurations.
+
+# High Level Design
+```mermaid
+flowchart TB
+ Proxy.DirectProxyHandler --> PulsarSslFactory
+ Proxy.AdminProxyHandler --> PulsarSslFactory
+ Proxy.ServiceChannelInitializer --> PulsarSslFactory
+ Broker.PulsarChannelInitializer --> PulsarSslFactory
+ Client.PulsarChannelInitializer --> PulsarSslFactory
+ Broker.WebService --> JettySSlContextFactory
+ Proxy.WebServer --> JettySSlContextFactory
+ PulsarAdmin --> AsyncHttpConnector
+ AsyncHttpConnector --> PulsarSslFactory
+ JettySSlContextFactory --> PulsarSslFactory
+ Client.HttpLookup.HttpClient --> PulsarSslFactory
+ PulsarSslFactory -.-> DefaultPulsarSslFactory
+ PulsarSslFactory -.-> CustomPulsarSslFactory
+```
+
+# Detailed Design
+
+## Design and Implementation Details
+
+### Pulsar Common Changes
+
+A new interface called `PulsarSslFactory` that provides public methods to
create a SSL Context, Client SSL Engine and
+Server SSL Engine. The SSL Context class returned will be of type
`javax.net.ssl.SSLContext`.
+
+```java
+public interface PulsarSslFactory extends AutoCloseable {
+ /*
+ * Utilizes the configuration to perform initialization operations and may
store information in instance variables.
+ * @param config PulsarSslConfiguration required by the factory for SSL
parameters
+ */
+ void initialize(PulsarSslConfiguration config);
+
+ /*
+ * Creates a client ssl engine based on the ssl context stored in the
instance variable and the respective parameters.
+ * @param peerHost Name of the peer host
+ * @param peerPort Port number of the peer
+ * @return A SSlEngine created using the instance variable stored Ssl Context
+ */
+ SSLEngine createClientSslEngine(String peerHost, int peerPort);
+
+ /*
+ * Creates a server ssl engine based on the ssl context stored in the
instance variable and the respective parameters.
+ * @return A SSLEngine created using the instance variable stored ssl context
+ */
+ SSLEngine createServerSslEngine();
+
+ /*
+ * Returns A boolean stating if the ssl context needs to be updated
+ * @return Boolean value representing if ssl context needs to be updated
+ */
+ boolean needsUpdate();
+
+ /*
+ * Checks if the SSL Context needs to be updated. If true, then a new SSL
Context should be internally create and
+ * should atomically replace the old ssl context stored in the instance
variable.
+ * @throws Exception It can throw an exception if the
createInternalSslContext method fails
+ */
+ default void update() throws Exception {
+ if (this.needsUpdate()) {
+ this.createInternalSslContext();
+ }
+ }
+
+ /*
+ * Creates a new SSL Context and internally stores it atomically into an
instance variable
+ * @throws It can throw an exception if the internal ssl context creation
fails.
+ */
+ void createInternalSslContext() throws Exception;
+
+ /*
+ * Returns the internally stored ssl context
+ * @throws IllegalStateException If the SSL Context has not be created
before this call, then it wil throw this
+ * exception.
+ */
+ SSLContext getInternalSslContext();
+
+ /*
+ * Shutdown the factory and close any internal dependencies
+ * @throws Exception It can throw an exception if there are any issues
shutting down the factory.
+ */
+ void close() throws Exception;
+
+}
+```
+
+A default implementation of the above SSLFactory class called
`DefaultPulsarSslFactory` that will generate the SSL
+Context and SSL Engines using File-based Certificates. It will be able to
support both Java keystores and "pem/crt/key"
+files.
+
+```java
+public class DefaultPulsarSslFactory implements PulsarSslFactory {
+ public void initialize(PulsarSslConfiguration config);
+ public SSLEngine createClientSslEngine(String peerHost, int peerPort);
+ public SSLEngine createServerSslEngine();
+ public boolean needsUpdate();
+ public void createInternalSslContext() throws Exception;
+ public SSLContext getInternalSslContext();
+ public void close() throws Exception;
+}
+```
+
+### Pulsar Commmon Changes
+
+4 new configurations will need to be added into the Configurations like
`ServiceConfiguration`,
+`ClientConfigurationData`, `ProxyConfiguration`, etc. All of the below will be
optional. It will use the default values
+to match the current behavior of Pulsar.
+
+- `sslFactoryPlugin`: SSL Factory Plugin class to provide SSLEngine and
SSLContext objects.
+The default class used is `DefaultPulsarSslFactory`.
+- `sslFactoryPluginParams`: SSL Factory plugin configuration parameters. It
will be of type string. It can be parsed by
+the plugin at its discretion.
+
+The below configs will be applicable only to the Pulsar Server components like
Broker and Proxy:
+- `brokerClientSslFactoryPlugin`: SSL Factory Plugin class used by internal
client to provide SSLEngine and SSLContext
+objects. The default class used is `DefaultPulsarSslFactory`.
+- `brokerClientSslFactoryPluginParams`: SSL Factory plugin configuration
parameters used by internal client. It can be
+parsed by the plugin at its discretion.
+
+`JettySslContextFactory` class will need to be changed to internally use the
`PulsarSslFactory` class to generate the
+SslContext.
+
+### SslFactory Usage across Pulsar Netty based server components
+
+Example Changes in broker's `PulsarChannelInitializer` to initialize the
PulsarSslFactory:
+```java
+PulsarSslConfiguration pulsarSslConfig = buildSslConfiguration(serviceConfig);
+this.sslFactory = (PulsarSslFactory)
Class.forName(serviceConfig.getSslFactoryPlugin())
+ .getConstructor().newInstance();
+this.sslFactory.initialize(pulsarSslConfig);
+this.sslFactory.createInternalSslContext();
+this.pulsar.getExecutor().scheduleWithFixedDelay(this::refreshSslContext,
+
serviceConfig.getTlsCertRefreshCheckDurationSec(),
+
serviceConfig.getTlsCertRefreshCheckDurationSec(),
+ TimeUnit.SECONDS);
+```
+
+Example changes in `PulsarChannelInitializer` to `initChannel(SocketChannel
ch)`:
+```java
+ch.pipeline().addLast(TLS_HANDLER, new
SslHandler(this.sslFactory.createServerSslEngine()));
+```
+
+The above changes is similar in all the Pulsar Server components that
internally utilize Netty.
+
+### SslFactory Usage across Pulsar Netty based Client components
+
+Example Changes in Client's `PulsarChannelInitializer` to initialize the
SslFactory:
+```java
+this.pulsarSslFactory = (PulsarSslFactory)
Class.forName(conf.getSslFactoryPlugin())
+ .getConstructor().newInstance();
+PulsarSslConfiguration sslConfiguration = buildSslConfiguration(conf);
+this.pulsarSslFactory.initialize(sslConfiguration);
+this.pulsarSslFactory.createInternalSslContext();
+scheduledExecutorProvider.getExecutor())
+ .scheduleWithFixedDelay(() -> {
+ this.refreshSslContext(conf);
+ }, conf.getAutoCertRefreshSeconds(),
+ conf.getAutoCertRefreshSeconds(), TimeUnit.SECONDS);
+```
+
+Example changes in `PulsarChannelInitializer` to `initChannel(SocketChannel
ch)`:
+```java
+SslHandler handler = new SslHandler(sslFactory
+ .createClientSslEngine(sniHost.getHostName(),
sniHost.getPort()));
+ch.pipeline().addFirst(TLS_HANDLER, handler);
+```
+
+The above changes is similar in all the Pulsar client components that
internally utilize Netty.
+
+### SslFactory Usage across Pulsar Jetty Based Server Components
+
+The initialization of the PulsarSslFactory is similar to the [Netty Server
initialization.](#sslfactory-usage-across-pulsar-jetty-based-server-components)
+
+The usage of the PulsarSslFactory requires changes in the
`JettySslContextFactory`. It will internally accept
+`PulsarSslFactory` as an input and utilize it to create the SSL Context.
+```java
+public class JettySslContextFactory {
+ private static class Server extends SslContextFactory.Server {
+ private final PulsarSslFactory sslFactory;
+
+ // New
+ public Server(String sslProviderString, PulsarSslFactory sslFactory,
+ boolean requireTrustedClientCertOnConnect, Set<String>
ciphers, Set<String> protocols) {
+ this.sslFactory = sslFactory;
+ // Current implementation
+ }
+
+ @Override
+ public SSLContext getSslContext() {
+ return this.sslFactory.getInternalSslContext();
+ }
+ }
+}
+```
+
+The above `JettySslContextFactory` will be used to create the SSL Context
within the Jetty Server. This pattern will be
+common across all Web Server created using Jetty within Pulsar.
+
+### SslFactory Usage across Pulsar AsyncHttpClient based Client Components
+
+The initialization of the PulsarSslFactory is similar to the [Netty Server
initialization.](#sslfactory-usage-across-pulsar-jetty-based-server-components)
+
+The usage of the PulsarSslFactory requires changes in the
`AsyncHttpConnector`. It will internally initialize the
+`PulsarSslFactory` and pass it to a new custom
`PulsarHttpAsyncSslEngineFactory` that implements
`org.asynchttpclient.SSLEngineFactory`.
+This new custom class will incorporate the features of the existing
`WithSNISslEngineFactory` and `JsseSslEngineFactory`
+and replace it.
+
+```java
+public class PulsarHttpAsyncSslEngineFactory extends DefaultSslEngineFactory {
+
+ private final PulsarSslFactory sslFactory;
+ private final String host;
+
+ public PulsarHttpAsyncSslEngineFactory(PulsarSslFactory sslFactory, String
host) {
+ this.sslFactory = sslFactory;
+ this.host = host;
+ }
+
+ @Override
+ protected void configureSslEngine(SSLEngine sslEngine,
AsyncHttpClientConfig config) {
+ super.configureSslEngine(sslEngine, config);
+ if (StringUtils.isNotBlank(host)) {
+ SSLParameters parameters = sslEngine.getSSLParameters();
+ parameters.setServerNames(Collections.singletonList(new
SNIHostName(host)));
+ sslEngine.setSSLParameters(parameters);
+ }
+ }
+
+ @Override
+ public SSLEngine newSslEngine(AsyncHttpClientConfig config, String
peerHost, int peerPort) {
+ SSLContext sslContext = this.sslFactory.getInternalSslContext();
+ SSLEngine sslEngine =
config.isDisableHttpsEndpointIdentificationAlgorithm()
+ ? sslContext.createSSLEngine() :
+ sslContext.createSSLEngine(domain(peerHost), peerPort);
+ configureSslEngine(sslEngine, config);
+ return sslEngine;
+ }
+
+}
+```
+
+The above `PulsarHttpAsyncSslEngineFactory` will be passed to the
DefaultAsyncHttpClientConfig.Builder while creating
+the DefaultAsyncHttpClient. This pattern will be common across all HTTP
Clients using AsyncHttpClient within Pulsar.
+
+## Public-facing Changes
+
+### Configuration
+
+Same as [Broker Common Changes](#pulsar-commmon-changes)
+
+### CLI
+CLI tools like `PulsarClientTool` and `PulsarAdminTool` will need to be
modified to support the new configurations.
+
+# Backward & Forward Compatibility
+
+## Revert
+Rolling back to the previous version of Pulsar will revert to the previous
behavior.
+
+## Upgrade
+Upgrading to the version containing the `PulsarSslFactory` will not cause any
behavior change. The `PulsarSslFactory`
+for the server, client and brokerclient will default to using the
`DefaultPulsarSslFactory` which will
+read the TLS certificates via the file system.
+
+The Pulsar system will use the custom plugin behavior only if the
`sslFactoryPlugin` configuration is set.
+
+# Links
+
+POC Changes: https://github.com/Apurva007/pulsar/pull/4
\ No newline at end of file
