lhotari commented on code in PR #22016:
URL: https://github.com/apache/pulsar/pull/22016#discussion_r1486474263
##########
pip/pip-333.md:
##########
@@ -0,0 +1,387 @@
+# PIP-337: SSL Factory Plugin to customize SSLContext/SSLEngine generation
+
+# Background knowledge
+Apache Pulsar supports TLS encrypted communication between the clients and
servers. The TLS encryption setup requires
+loading the TLS certificates and its respective passwords to generate the SSL
Context. Pulsar supports loading these
+certificates and passwords via the filesystem. It supports both Java based
Keystores/Truststores and TLS information in
+".crt", ".pem" & ".key" formats. This information is refreshed based on a
configurable interval.
+
+Apache Pulsar internally uses 3 different frameworks for connection management:
+
+- Netty: Connection management for Pulsar server and client that understands
Pulsar binary protocol.
+- Jetty: HTTP Server creation for Pulsar Admin and websocket. Jetty Client is
used by proxy for admin client calls.
+- AsyncHttpClient: HTTP Client creation for Admin client and HTTP Lookup
+
+Each of the above frameworks supports customizing the generation of the SSL
Context and SSL Engine. Currently, Pulsar
+uses these features to feed the SSL Context via its internal security tools
after loading the file based certificates.
+One of the issues of using these features is that pulsar tries to bootstrap
the SSL Context in multiple ways to suit
+each framework and file type.
+
+```mermaid
+flowchart TB
+ Proxy.DirectProxyHandler --> NettyClientSslContextRefresher
+ Proxy.DirectProxyHandler --> NettySSLContextAutoRefreshBuilder
+ Proxy.AdminProxyHandler --> KeyStoreSSLContext
+ Proxy.AdminProxyHandler --> SecurityUtility
+ Proxy.ServiceChannelInitializer --> NettySSLContextAutoRefreshBuilder
+ Proxy.ServiceChannelInitializer --> NettyServerSslContextBuilder
+ Broker.PulsarChannelInitializer --> NettyServerSslContextBuilder
+ Broker.PulsarChannelInitializer --> NettySSLContextAutoRefreshBuilder
+ Client.PulsarChannelInitializer --> NettySSLContextAutoRefreshBuilder
+ Client.PulsarChannelInitializer --> SecurityUtility
+ Broker.WebService --> JettySSlContextFactory
+ Proxy.WebServer --> JettySSlContextFactory
+ PulsarAdmin --> AsyncHttpConnector
+ AsyncHttpConnector --> KeyStoreSSLContext
+ AsyncHttpConnector --> SecurityUtility
+ JettySSlContextFactory --> NetSslContextBuilder
+ JettySSlContextFactory --> DefaultSslContextBuilder
+ NettyClientSslContextRefresher -.-> SslContextAutoRefreshBuilder
+ NettySSLContextAutoRefreshBuilder -.-> SslContextAutoRefreshBuilder
+ NettyServerSslContextBuilder -.-> SslContextAutoRefreshBuilder
+ NetSslContextBuilder -.-> SslContextAutoRefreshBuilder
+ DefaultSslContextBuilder -.-> SslContextAutoRefreshBuilder
+ Client.HttpLookup.HttpClient --> KeyStoreSSLContext
+ Client.HttpLookup.HttpClient --> SecurityUtility
+ SecurityUtility -.-> KeyManagerProxy
+ SecurityUtility -.-> TrustManagerProxy
+```
+The above diagram is an example of the complexity of the TLS encryption setup
within Pulsar. The above diagram only
+contains the basic components of Pulsar excluding Websockets, Functions, etc.
+
+Pulsar uses 2 base classes to load the TLS information.
+
+- `SecurityUtility`: It loads files of type ".crt", ".pem" and ".key" and
converts it into SSL Context. This SSL Context
+can be of type `io.netty.handler.ssl.SslContext` or `javax.net.ssl.SSLContext`
based on the caller. Security Utility
+can be used to create SSL Context that internally has KeyManager and
Trustmanager proxies that load cert changes
+dynamically.
+- `KeyStoreSSLContext`: It loads files of type Java Keystore/Truststore and
converts it into SSL Context. This SSL
+Context will be of type `javax.net.ssl.SSLContext`. This is always used to
create the SSL Engine.
+
+Each of the above classes are either directly used by Pulsar Clients or used
via implementations of the abstract class
+`SslContextAutoRefreshBuilder`.
+
+- `SslContextAutoRefreshBuilder` - This abstract class is used to refresh
certificates at a configurable interval. It
+internally provides a public API to return the SSL Context.
+
+There are several implementations of the above abstract class to suit the
needs of each of the framework and the
+respective TLS certificate files:
+
+- `NettyClientSslContextRefresher` - It internally creates the
`io.netty.handler.ssl.SslContext` using the ".crt",
+".pem" and ".key" files for the proxy client.
+- `NettySSLContextAutoRefreshBuilder` - It internally creates the
`KeyStoreSSLContext` using the Java Keystores.
+- `NettyServerSslContextBuilder` - It internally creates the
`io.netty.handler.ssl.SslContext` using the ".crt",
+ ".pem" and ".key" files for the server.
+- `NetSslContextBuilder` - It internally creates the
`javax.net.ssl.SSLContext` using the Java Keystores for the web
+server.
+- `DefaultSslContextBuilder` - It internally creates the
`javax.net.ssl.SSLContext` using the ".crt", ".pem" and ".key"
+files for the web server.
+
+# Motivation
+Apache Pulsar's TLS encryption configuration is not pluggable. It only
supports file-based certificates. This makes
+Pulsar difficult to adopt for organizations that require loading TLS
certificates by other mechanisms.
+
+# Goals
+The purpose of this PIP is to introduce the following:
+
+- Provide a mechanism to plugin a custom SSL Factory that can generate SSL
Context and SSL Engine.
+- Simplify the Pulsar code base to universally use `javax.net.ssl.SSLContext`
and reduce the amount of code required to
+build and configure the SSL context taking into consideration backwards
compatibility.
+
+## In Scope
+
+- Creation of a new abstract class `SSLFactory` that can generate a SSL
Context, Client SSL Engine and Server SSL
+Engine. It should support refreshing the certificates at a configurable
interval.
+- Creation of a default implementation of `SSLFactory` that supports loading
the SSL Context and SSL Engine via
+file-based certificates. Internally it will use the SecurityUtility and
KeyStoreSSLContext.
+- Modify the Pulsar Components to support the `SSLFactory` instead of the
SslContextAutoRefreshBuilder, SecurityUtility
+and KeyStoreSSLContext.
+- Remove the SslContextAutoRefreshBuilder and all its implementations.
+
+# High Level Design
+```mermaid
+flowchart TB
+ Proxy.DirectProxyHandler --> Sslfactory
+ Proxy.AdminProxyHandler --> Sslfactory
+ Proxy.ServiceChannelInitializer --> Sslfactory
+ Broker.PulsarChannelInitializer --> Sslfactory
+ Client.PulsarChannelInitializer --> Sslfactory
+ Broker.WebService --> JettySSlContextFactory
+ Proxy.WebServer --> JettySSlContextFactory
+ PulsarAdmin --> AsyncHttpConnector
+ AsyncHttpConnector --> Sslfactory
+ JettySSlContextFactory --> Sslfactory
+ Client.HttpLookup.HttpClient --> Sslfactory
+ Sslfactory -.-> DefaultSslFactory
+ Sslfactory -.-> CustomSslFactory
+```
+
+# Detailed Design
+
+## Design and Implementation Details
+
+### Pulsar Common Changes
+
+A new abstract class called `SslFactory` that provides public methods to
create a SSL Context, Client SSL Engine and
+Server SSL Engine. The SSL Context class returned will be of type
`javax.net.ssl.SSLContext`. This class will refresh
+the SSL Context at the configured interval. This refresh behavior is the same
as the
+current `SslContextAutoRefreshBuilder`.
+
+```java
+public abstract class SslFactory {
Review Comment:
I guess we could make this a Java `interface` so that it would be easier to
evolve later. The name could be Pulsar specific so that it's clear that it's an
interface for Pulsar. (`PulsarSslFactory`?)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
