sijie closed pull request #2286: [website] cherry-pick "Rework TLS docs to
separate concerns" to new website
URL: https://github.com/apache/incubator-pulsar/pull/2286
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/site2/docs/admin-api-clusters.md b/site2/docs/admin-api-clusters.md
index 91e700e8f6..e5adbf268a 100644
--- a/site2/docs/admin-api-clusters.md
+++ b/site2/docs/admin-api-clusters.md
@@ -79,7 +79,7 @@ bin/pulsar initialize-cluster-metadata \
--broker-service-url-tls pulsar+ssl://pulsar.us-west.example.com:6651/
```
-You'll need to use `--*-tls` flags only if you're using [TLS
authentication](security-tls.md) in your instance.
+You'll need to use `--*-tls` flags only if you're using [TLS
authentication](security-tls-authentication.md) in your instance.
### Get configuration
diff --git a/site2/docs/admin-api-permissions.md
b/site2/docs/admin-api-permissions.md
index 8a7793e9b1..51a93f65ea 100644
--- a/site2/docs/admin-api-permissions.md
+++ b/site2/docs/admin-api-permissions.md
@@ -4,6 +4,111 @@ title: Managing permissions
sidebar_label: Persmissions
---
-## Permissions resources
+Permissions in Pulsar are managed at the
[namespace](reference-terminology.md#namespace) level
+(that is, within [tenants](reference-terminology.md#tenant) and
[clusters](reference-terminology.md#cluster)).
-{% include explanations/permissions.md %}
+## Grant permissions
+
+You can grant permissions to specific roles for lists of operations such as
`produce` and `consume`.
+
+### pulsar-admin
+
+Use the [`grant-permission`](reference-pulsar-admin.md#grant-permission)
subcommand and specify a namespace, actions using the `--actions` flag, and a
role using the `--role` flag:
+
+```shell
+$ pulsar-admin namespaces grant-permission test-tenant/ns1 \
+ --actions produce,consume \
+ --role admin10
+```
+
+Wildcard authorization can be performed when
`authorizationAllowWildcardsMatching` is set to `true` in `broker.conf`.
+
+e.g.
+```shell
+$ pulsar-admin namespaces grant-permission test-tenant/ns1 \
+ --actions produce,consume \
+ --role 'my.role.*'
+```
+
+Then, roles `my.role.1`, `my.role.2`, `my.role.foo`, `my.role.bar`, etc. can
produce and consume.
+
+```shell
+$ pulsar-admin namespaces grant-permission test-tenant/ns1 \
+ --actions produce,consume \
+ --role '*.role.my'
+```
+
+Then, roles `1.role.my`, `2.role.my`, `foo.role.my`, `bar.role.my`, etc. can
produce and consume.
+
+**Note**: A wildcard matching works at **the beginning or end of the role name
only**.
+
+e.g.
+```shell
+$ pulsar-admin namespaces grant-permission test-tenant/ns1 \
+ --actions produce,consume \
+ --role 'my.*.role'
+```
+
+In this case, only the role `my.*.role` has permissions.
+Roles `my.1.role`, `my.2.role`, `my.foo.role`, `my.bar.role`, etc. **cannot**
produce and consume.
+
+### REST API
+
+{@inject:
endpoint|POST|/admin/v2/namespaces/:tenant/:namespace/permissions/:role|operation/grantPermissionOnNamespace}
+
+### Java
+
+```java
+admin.namespaces().grantPermissionOnNamespace(namespace, role,
getAuthActions(actions));
+```
+
+## Get permissions
+
+You can see which permissions have been granted to which roles in a namespace.
+
+### pulsar-admin
+
+Use the [`permissions`](reference-pulsar-admin#permissions) subcommand and
specify a namespace:
+
+```shell
+$ pulsar-admin namespaces permissions test-tenant/ns1
+{
+ "admin10": [
+ "produce",
+ "consume"
+ ]
+}
+```
+
+### REST API
+
+{@inject:
endpoint|GET|/admin/v2/namespaces/:tenant/:namespace/permissions|operation/getPermissions}
+
+### Java
+
+```java
+admin.namespaces().getPermissions(namespace);
+```
+
+## Revoke permissions
+
+You can revoke permissions from specific roles, which means that those roles
will no longer have access to the specified namespace.
+
+### pulsar-admin
+
+Use the [`revoke-permission`](reference-pulsar-admin.md#revoke-permission)
subcommand and specify a namespace and a role using the `--role` flag:
+
+```shell
+$ pulsar-admin namespaces revoke-permission test-tenant/ns1 \
+ --role admin10
+```
+
+### REST API
+
+{@inject:
endpoint|DELETE|/admin/v2/namespaces/:tenant/:namespace/permissions/:role|operation/revokePermissionsOnNamespace}
+
+### Java
+
+```java
+admin.namespaces().revokePermissionsOnNamespace(namespace, role);
+```
diff --git a/site2/docs/client-libraries-go.md
b/site2/docs/client-libraries-go.md
index eeae027483..e7a901efed 100644
--- a/site2/docs/client-libraries-go.md
+++ b/site2/docs/client-libraries-go.md
@@ -48,7 +48,7 @@ A URL for a production Pulsar cluster may look something like
this:
pulsar://pulsar.us-west.example.com:6650
```
-If you're using [TLS](security-tls.md) authentication, the URL will look like
something like this:
+If you're using [TLS](security-tls-authentication.md) authentication, the URL
will look like something like this:
```http
pulsar+ssl://pulsar.us-west.example.com:6651
@@ -435,7 +435,7 @@ Parameter | Description
## TLS encryption and authentication
-In order to use [TLS encryption](security-tls.md), you'll need to configure
your client to do so:
+In order to use [TLS encryption](security-tls-transport.md), you'll need to
configure your client to do so:
* Use `pulsar+ssl` URL type
* Set `TLSTrustCertsFilePath` to the path to the TLS certs used by your
client and the Pulsar broker
diff --git a/site2/docs/client-libraries-java.md
b/site2/docs/client-libraries-java.md
index 137f05bf39..c342e886d9 100644
--- a/site2/docs/client-libraries-java.md
+++ b/site2/docs/client-libraries-java.md
@@ -63,7 +63,7 @@ A URL for a production Pulsar cluster may look something like
this:
pulsar://pulsar.us-west.example.com:6650
```
-If you're using [TLS](security-tls.md) authentication, the URL will look like
something like this:
+If you're using [TLS](security-tls-authentication.md) authentication, the URL
will look like something like this:
```http
pulsar+ssl://pulsar.us-west.example.com:6651
@@ -407,11 +407,11 @@ The following schema formats are currently available for
Java:
## Authentication
-Pulsar currently supports two authentication schemes: [TLS](security-tls.md)
and [Athenz](security-athenz.md). The Pulsar Java client can be used with both.
+Pulsar currently supports two authentication schemes:
[TLS](security-tls-authentication.md) and [Athenz](security-athenz.md). The
Pulsar Java client can be used with both.
### TLS Authentication
-To use [TLS](security-tls.md), you need to set TLS to `true` using the
`setUseTls` method, point your Pulsar client to a TLS cert path, and provide
paths to cert and key files.
+To use [TLS](security-tls-authentication.md), you need to set TLS to `true`
using the `setUseTls` method, point your Pulsar client to a TLS cert path, and
provide paths to cert and key files.
Here's an example configuration:
diff --git a/site2/docs/concepts-architecture-overview.md
b/site2/docs/concepts-architecture-overview.md
index a58c06feb0..fe19fb3baa 100644
--- a/site2/docs/concepts-architecture-overview.md
+++ b/site2/docs/concepts-architecture-overview.md
@@ -131,7 +131,7 @@ $ bin/pulsar proxy \
Some important things to know about the Pulsar proxy:
* Connecting clients don't need to provide *any* specific configuration to use
the Pulsar proxy. You won't need to update the client configuration for
existing applications beyond updating the IP used for the service URL (for
example if you're running a load balancer over the Pulsar proxy).
-* [TLS encryption and authentication](security-tls.md) is supported by the
Pulsar proxy
+* [TLS encryption](security-tls-transport.md) and
[authentication](security-tls-authentication.md) is supported by the Pulsar
proxy
## Service discovery
diff --git a/site2/docs/deploy-bare-metal-multi-cluster.md
b/site2/docs/deploy-bare-metal-multi-cluster.md
index 2b94aa79d8..270ebdea46 100644
--- a/site2/docs/deploy-bare-metal-multi-cluster.md
+++ b/site2/docs/deploy-bare-metal-multi-cluster.md
@@ -211,7 +211,7 @@ As you can see from the example above, the following needs
to be specified:
* The web service URL for the cluster
* A broker service URL enabling interaction with the
[brokers](reference-terminology.md#broker) in the cluster
-If you're using [TLS](security-tls.md), you'll also need to specify a TLS web
service URL for the cluster as well as a TLS broker service URL for the brokers
in the cluster.
+If you're using [TLS](security-tls-transport.md), you'll also need to specify
a TLS web service URL for the cluster as well as a TLS broker service URL for
the brokers in the cluster.
Make sure to run `initialize-cluster-metadata` for each cluster in your
instance.
diff --git a/site2/docs/deploy-bare-metal.md b/site2/docs/deploy-bare-metal.md
index 0c9236b5b7..14d4c66fd2 100644
--- a/site2/docs/deploy-bare-metal.md
+++ b/site2/docs/deploy-bare-metal.md
@@ -130,9 +130,9 @@ Flag | Description
`--zookeeper` | A "local" ZooKeeper connection string for the cluster. This
connection string only needs to include *one* machine in the ZooKeeper cluster.
`--configuration-store` | The configuration store connection string for the
entire instance. As with the `--zookeeper` flag, this connection string only
needs to include *one* machine in the ZooKeeper cluster.
`--web-service-url` | The web service URL for the cluster, plus a port. This
URL should be a standard DNS name. The default port is 8080 (we don't recommend
using a different port).
-`--web-service-url-tls` | If you're using [TLS](security-tls.md), you'll also
need to specify a TLS web service URL for the cluster. The default port is 8443
(we don't recommend using a different port).
+`--web-service-url-tls` | If you're using [TLS](security-tls-transport.md),
you'll also need to specify a TLS web service URL for the cluster. The default
port is 8443 (we don't recommend using a different port).
`--broker-service-url` | A broker service URL enabling interaction with the
brokers in the cluster. This URL should use the same DNS name as the web
service URL but should use the `pulsar` scheme instead. The default port is
6650 (we don't recommend using a different port).
-`--broker-service-url-tls` | If you're using [TLS](security-tls.md), you'll
also need to specify a TLS web service URL for the cluster as well as a TLS
broker service URL for the brokers in the cluster. The default port is 6651 (we
don't recommend using a different port).
+`--broker-service-url-tls` | If you're using [TLS](security-tls-transport.md),
you'll also need to specify a TLS web service URL for the cluster as well as a
TLS broker service URL for the brokers in the cluster. The default port is 6651
(we don't recommend using a different port).
## Deploying a BookKeeper cluster
diff --git a/site2/docs/security-authorization.md
b/site2/docs/security-authorization.md
index 433f35051f..cf5a1e025b 100644
--- a/site2/docs/security-authorization.md
+++ b/site2/docs/security-authorization.md
@@ -4,17 +4,54 @@ title: Authentication and authorization in Pulsar
sidebar_label: Authorization and ACLs
---
+
In Pulsar, the [authentication
provider](security-overview.md#authentication-providers) is charged with
properly identifying clients and
-associating them with [role tokens](security-overview.md#role-tokens).
*Authorization* is the process that determines *what* clients are able to do.
+associating them with [role tokens](security-overview.md#role-tokens). If only
authentication is enabled, an authenticated role token will have the ability to
access all resources in the cluster. *Authorization* is the process that
determines *what* clients are able to do.
+
+The role tokens with the most privileges are the *superusers*. The
*superusers* can create and destroy tenants, along with having full access to
all tenant resources.
+
+When a [tenant](reference-terminology.md#tenant) is created by a superuser,
that tenant is assigned an admin role. A client with the admin role token can
then create, modify and destroy namespaces, and grant and revoke permissions to
*other role tokens* on those namespaces.
+
+## Broker and Proxy Setup
+
+### Enabling Authorization and Assigning Superusers
+
+Authorization is enabled and superusers are assigned in the broker
([`conf/broker.conf`](reference-configuration.md#broker)) and proxy
([`conf/proxy.conf`](reference-configuration.md#proxy)) configuration files.
+
+```properties
+authorizationEnabled=true
+superUserRoles=my-super-user-1,my-super-user-2
+```
+
+> A full list of parameters available in the `conf/broker.conf` file,
+> as well as the default values for those parameters, can be found in [Broker
Configuration](reference-configuration.md#broker)
+
+Typically, superuser roles are used for administrators and clients but also
for broker-to-broker authorization. When using
[geo-replication](concepts-replication.md), every broker needs to be able to
publish to all the other clusters' topics.
+
+### Proxy Roles
+
+By default, the broker treats the connection between a proxy and the broker as
a normal user connection. The user is authenticated as the role configured in
```proxy.conf``` (see ["Enabling TLS Authentication on
Proxies"](security-tls-authentication#on-proxies)). However, this is rarely the
behaviour that the user desires when connecting to the cluster through a proxy.
The user expects to be able to interact with the cluster as the role for which
they have authenticated with the proxy.
+
+Pulsar uses *Proxy roles* to enable this. Proxy roles are specified in the
broker configuration file,
[`conf/broker.conf`](reference-configuration.md#broker). If a client that is
authenticated with a broker is one of its ```proxyRoles```, all requests from
that client must also carry information about the role of the client that is
authenticated with the proxy. If this information, which we call the *original
principal*, is missing, the client will not be able to access anything.
-Authorization in Pulsar is managed at the
[tenant](reference-terminology.md#tenant) level, which means that you can have
multiple authorization schemes active
-in a single Pulsar instance. You could, for example, create a `shopping`
tenant that has one set of [roles](security-overview.md#role-tokens)
-and applies to a shopping application used by your company, while an
`inventory` tenant would be used only by an inventory application.
+Both the *proxy role* and the *original principle* must be authorized to
access a resource for that resource to be accessible via the proxy.
Administrators can take two approaches to this.
-> When working with properties, you can specify which of your Pulsar clusters
your property is allowed to use.
-> This enables you to also have cluster-level authorization schemes.
+The more secure approach is to grant access to the proxy roles each time you
grant access to a resource. For example, if you have a proxy role ```proxy1```,
when a tenant is created by the superuser, ```proxy1``` should be specified as
one of the admin roles. When a role is granted permissions to produce or
consume from a namespace, if that client wants to produce or consume through a
proxy, ```proxy1``` should also be granted the same permissions.
-## Creating a new tenant
+Another approach is to make the proxy role a superuser. This will allow the
proxy to access all resources. The client will still need to authenticate with
the proxy, and all requests made through the proxy will have their role
downgraded to the *original principal* of the authenticated client. However, if
the proxy is compromised, a bad actor could get full access to your cluster.
+
+Roles can be specified as proxy roles in
[`conf/broker.conf`](reference-configuration.md#broker).
+
+```properties
+proxyRoles=my-proxy-role
+
+# if you want to allow superusers to use the proxy (see above)
+superUserRoles=my-super-user-1,my-super-user-2,my-proxy-role
+```
+
+## Administering Tenants
+
+### Creating a new tenant
A Pulsar [tenant](reference-terminology.md#tenant) is typically provisioned by
Pulsar [instance](reference-terminology.md#instance) administrators or by some
kind of self-service portal.
@@ -36,27 +73,11 @@ The structure of topic names in Pulsar reflects the
hierarchy between tenants, c
persistent://tenant/namespace/topic
```
-## Managing permissions
-
-{% include explanations/permissions.md %}
-
-## Superusers
-
-In Pulsar you can assign certain roles to be *superusers* of the system. A
superuser is allowed to perform all administrative tasks on all tenants and
namespaces, as well as to publish and subscribe to all topics.
-
-Superusers are configured in the broker configuration file in
[`conf/broker.conf`](reference-configuration.md#broker) configuration file,
using the [`superUserRoles`](reference-configuration.md#broker-superUserRoles)
parameter:
-
-```tenants
-superUserRoles=my-super-user-1,my-super-user-2
-```
-
-> A full listing of parameters available in the `conf/broker.conf` file, as
well as the default
-> values for those parameters, can be found in [Broker
Configuration](reference-configuration.md#broker).
+### Managing permissions
-Typically, superuser roles are used for administrators and clients but also
for broker-to-broker authorization. When using
[geo-replication](administration-geo.md), every broker
-needs to be able to publish to other clusters' topics.
+You can use [Pulsar Admin Tools](admin-api-permissions.md) for managing
permission in Pulsar.
-## Pulsar admin authentication
+### Pulsar admin authentication
```java
String authPluginClassName = "com.org.MyAuthPluginClass";
diff --git a/site2/docs/security-overview.md b/site2/docs/security-overview.md
index 97c1ffb3aa..abc25261b4 100644
--- a/site2/docs/security-overview.md
+++ b/site2/docs/security-overview.md
@@ -27,12 +27,12 @@ role token is then used for [Authorization and
ACLs](security-authorization.md)
Currently Pulsar supports two authentication providers:
-- [TLS Authentication](security-tls.md)
+- [TLS Authentication](security-tls-authentication.md)
- [Athenz](security-athenz.md)
## Contents
-- [Encryption and Authentication using TLS](security-tls.md)
+- [Encryption](security-tls-transport.md) and
[Authentication](security-tls-authentication.md) using TLS
- [Authentication using Athenz](security-athenz.md)
- [Authorization and ACLs](security-authorization.md)
- [End-to-End Encryption](security-encryption.md)
diff --git a/site2/docs/security-tls-authentication.md
b/site2/docs/security-tls-authentication.md
new file mode 100644
index 0000000000..d5eb7d7ebe
--- /dev/null
+++ b/site2/docs/security-tls-authentication.md
@@ -0,0 +1,136 @@
+---
+id: security-tls-authentication
+title: Authentication using TLS
+sidebar_label: Authentication using TLS
+---
+
+## TLS Authentication Overview
+
+TLS authentication is an extension of [TLS transport
encryption](security-tls-transport.md), but instead of only servers having keys
and certs which the client uses the verify the server's identity, clients also
have keys and certs which the server uses to verify the client's identity. You
must have TLS transport encryption configured on your cluster before you can
use TLS authentication. This guide assumes you already have TLS transport
encryption configured.
+
+### Creating client certificates
+
+Client certificates are generated using the same certificate authority as was
used to generate the server certificates.
+
+The biggest difference between client certs and server certs is that the
**common name** for the client certificate is the **role token** which that
client will be authenticated as.
+
+First generate the key.
+```bash
+$ openssl genrsa -out admin.key.pem 2048
+```
+
+Similar to the broker, the client expects the key to be in [PKCS
8](https://en.wikipedia.org/wiki/PKCS_8) format, so convert it.
+
+```bash
+$ openssl pkcs8 -topk8 -inform PEM -outform PEM \
+ -in admin.key.pem -out admin.key-pk8.pem -nocrypt
+```
+
+Generate the certificate request. When asked for a **common name**, enter the
**role token** which you want this key pair to authenticate a client as.
+
+```bash
+$ openssl req -config openssl.cnf \
+ -key admin.key.pem -new -sha256 -out admin.cert.pem
+```
+
+Sign with request with the certificate authority. Note that that client certs
uses the **usr_cert** extension, which allows the cert to be used for client
authentication.
+
+```bash
+$ openssl ca -config openssl.cnf -extensions usr_cert \
+ -days 1000 -notext -md sha256 \
+ -in admin.csr.pem -out admin.cert.pem
+```
+
+This will give you a cert, `admin.cert.pem`, and a key, `admin.key-pk8.pem`,
which, with `ca.cert.pem`, can be used by clients to authenticate themselves to
brokers and proxies as the role token ``admin``.
+
+## Enabling TLS Authentication ...
+
+### ... on Brokers
+
+To configure brokers to authenticate clients, put the following in
`broker.conf`, alongside [the configuration to enable tls
transport](security-tls-transport.md#broker-configuration):
+
+```properties
+# Configuration to enable authentication
+authenticationEnabled=true
+authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderTls
+```
+
+### ... on Proxies
+
+To configure proxies to authenticate clients, put the folling in `proxy.conf`,
alongside [the configuration to enable tls
transport](security-tls-transport.md#proxy-configuration):
+
+The proxy should have its own client key pair for connecting to brokers. The
role token for this key pair should be configured in the ``proxyRoles`` of the
brokers. See the [authorization guide](security-authorization.md) for more
details.
+
+```properties
+# For clients connecting to the proxy
+authenticationEnabled=true
+authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderTls
+
+# For the proxy to connect to brokers
+brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationTls
+brokerClientAuthenticationParameters=tlsCertFile:/path/to/proxy.cert.pem,tlsKeyFile:/path/to/proxy.key-pk8.pem
+```
+
+## Client configuration
+
+When TLS authentication, the client needs to connect via TLS transport, so you
need to configure the client to use ```https://``` and port 8443 for the web
service URL, and ```pulsar+ssl://``` and port 6651 for the broker service URL.
+
+### CLI tools
+
+[Command-line tools](reference-cli-tools.md) like
[`pulsar-admin`](reference-pulsar-admin.md),
[`pulsar-perf`](reference-cli-tools.md#pulsar-perf), and
[`pulsar-client`](reference-cli-tools.md#pulsar-client) use the
`conf/client.conf` config file in a Pulsar installation.
+
+You'll need to add the following parameters to that file to use TLS
authentication with Pulsar's CLI tools:
+
+```properties
+webServiceUrl=https://broker.example.com:8443/
+brokerServiceUrl=pulsar+ssl://broker.example.com:6651/
+useTls=true
+tlsAllowInsecureConnection=false
+tlsTrustCertsFilePath=/path/to/ca.cert.pem
+authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationTls
+authParams=tlsCertFile:/path/to/my-role.cert.pem,tlsKeyFile:/path/to/my-role.key-pk8.pem
+```
+
+### Java client
+
+```java
+import org.apache.pulsar.client.api.PulsarClient;
+
+PulsarClient client = PulsarClient.builder()
+ .serviceUrl("pulsar+ssl://broker.example.com:6651/")
+ .enableTls(true)
+ .tlsTrustCertsFilePath("/path/to/ca.cert.pem")
+ .authentication("org.apache.pulsar.client.impl.auth.AuthenticationTls",
+
"tlsCertFile:/path/to/my-role.cert.pem,tlsKeyFile:/path/to/my-role.key-pk8.pem")
+ .build();
+```
+
+### Python client
+
+```python
+from pulsar import Client, AuthenticationTLS
+
+auth = AuthenticationTLS("/path/to/my-role.cert.pem",
"/path/to/my-role.key-pk8.pem")
+client = Client("pulsar+ssl://broker.example.com:6651/",
+ tls_trust_certs_file_path="/path/to/ca.cert.pem",
+ tls_allow_insecure_connection=False,
+ authentication=auth)
+```
+
+### C++ client
+
+```c++
+#include <pulsar/Client.h>
+
+pulsar::ClientConfiguration config;
+config.setUseTls(true);
+config.setTlsTrustCertsFilePath("/path/to/ca.cert.pem");
+config.setTlsAllowInsecureConnection(false);
+
+pulsar::AuthenticationPtr auth =
pulsar::AuthTls::create("/path/to/my-role.cert.pem",
+
"/path/to/my-role.key-pk8.pem")
+config.setAuth(auth);
+
+pulsar::Client client("pulsar+ssl://broker.example.com:6651/", config);
+```
+
diff --git a/site2/docs/security-tls-transport.md
b/site2/docs/security-tls-transport.md
new file mode 100644
index 0000000000..70db61f0be
--- /dev/null
+++ b/site2/docs/security-tls-transport.md
@@ -0,0 +1,196 @@
+---
+id: security-tls-transport
+title: Transport Encryption using TLS
+sidebar_label: Transport Encryption using TLS
+---
+
+## TLS Overview
+
+By default, Apache Pulsar clients communicate with the Apache Pulsar service
in plain text, which means that all data is sent in the clear. TLS can be used
to encrypt this traffic so that it cannot be snooped by a man-in-the-middle
attacker.
+
+TLS can be configured for both encryption and authentication. You may
configure just TLS transport encryption, which is covered in this guide. TLS
authentication is covered [elsewhere](security-tls-authentication.md).
Alternatively, you can use [another authentication
mechanism](security-athenz.md) on top of TLS transport encryption.
+
+> Note that enabling TLS may have a performance impact due to encryption
overhead.
+
+## TLS concepts
+
+TLS is a form of [public key
cryptography](https://en.wikipedia.org/wiki/Public-key_cryptography).
Encryption is performed using key pairs consisting of a public key and a
private key. Messages are encrypted with the public key and can be decrypted
with the private key.
+
+To use TLS transport encryption, you need two kinds of key pairs, **server key
pairs** and a **certificate authority**.
+
+A third kind of key pair, **client key pairs**, are used for [client
authentication](security-tls-authentication.md).
+
+The **certificate authority** private key should be stored in a very secure
location (a fully encrypted, disconnected, air gapped computer). The
certificate authority public key, the **trust cert**, can be freely shared.
+
+For both client and server key pairs, the administrator first generates a
private key and a certificate request. Then the certificate authority private
key is used to sign the certificate request, generating a certificate. This
certificate is the public key for the server/client key pair.
+
+For TLS transport encryption, the clients can use the **trust cert** to verify
that the server they are talking to has a key pair that was signed by the
certificate authority. A man-in-the-middle attacker would not have access to
the certificate authority, so they couldn't create a server with such a key
pair.
+
+For TLS authentication, the server uses the **trust cert** to verify that the
client has a key pair that was signed by the certificate authority. The Common
Name of the **client cert** is then used as the client's role token (see
[Overview](security-overview.md)).
+
+## Creating TLS Certificates
+
+Creating TLS certificates for Pulsar involves creating a [certificate
authority](#certificate-authority) (CA), [server
certificate](#server-certificate), and [client
certificate](#client-certificate).
+
+The following guide is an abridged guide to setting up a certificate
authority. For a more detailed guide, there are plenty of resource on the
internet. We recommend the [this
guide](https://jamielinux.com/docs/openssl-certificate-authority/index.html).
+
+### Certificate authority
+
+The first step is to create the certificate for the CA. The CA will be used to
sign both the broker and client certificates, in order to ensure that each
party will trust the others. The CA should be stored in a very secure location
(ideally completely disconnected from networks, air gapped, and fully
encrypted).
+
+Create a directory for your CA, and place [this openssl configuration
file](https://github.com/apache/incubator-pulsar/tree/master/site2/website/static/examples/openssl.cnf)
in the directory. You may want to modify the default answers for company name
and department in the configuration file. Export the location of the CA
directory to the environment variable, CA_HOME. The configuration file uses
this environment variable to find the rest of the files and directories needed
for the CA.
+
+```bash
+$ mkdir my-ca
+$ cd my-ca
+$ wget
https://github.com/apache/incubator-pulsar/tree/master/site2/website/static/examples/openssl.cnf
+$ export CA_HOME=$(pwd)
+```
+
+Create the necessary directories, keys and certs.
+
+```bash
+$ mkdir certs crl newcerts private
+$ chmod 700 private/
+$ touch index.txt
+$ echo 1000 > serial
+$ openssl genrsa -aes256 -out private/ca.key.pem 4096
+$ chmod 400 private/ca.key.pem
+$ openssl req -config openssl.cnf -key private/ca.key.pem \
+ -new -x509 -days 7300 -sha256 -extensions v3_ca \
+ -out certs/ca.cert.pem
+$ chmod 444 certs/ca.cert.pem
+```
+
+After answering the question prompts, this will store CA-related files in the
`./my-ca` directory. Within that directory:
+
+* `certs/ca.cert.pem` is the public certificate. It is meant to be distributed
to all parties involved.
+* `private/ca.key.pem` is the private key. This is only needed when signing a
new certificate for either broker or clients and it must be safely guarded.
+
+### Server certificate
+
+Once a CA certificate has been created, you can create certificate requests
and sign them with the CA.
+
+The following commands will ask you a few questions and then create the
certificates. When asked for the common name, you should match the hostname of
the broker. You could also use a wildcard to match a group of broker hostnames,
for example `*.broker.usw.example.com`. This ensures that the same certificate
can be reused on multiple machines.
+
+> #### Tips
+>
+> Sometimes it is not possible or makes no sense to match the hostname,
+> such as when the brokers are created with random hostnames, or you
+> plan to connect to the hosts via their IP. In this case, the client
+> should be configured to disable TLS hostname verification.
+
+First generate the key.
+```bash
+$ openssl genrsa -out broker.key.pem 2048
+```
+
+The broker expects the key to be in [PKCS
8](https://en.wikipedia.org/wiki/PKCS_8) format, so convert it.
+
+```bash
+$ openssl pkcs8 -topk8 -inform PEM -outform PEM \
+ -in broker.key.pem -out broker.key-pk8.pem -nocrypt
+```
+
+Generate the certificate request...
+
+```bash
+$ openssl req -config openssl.cnf \
+ -key broker.key.pem -new -sha256 -out broker.cert.pem
+```
+
+... and sign it with the certificate authority.
+```bash
+$ openssl ca -config openssl.cnf -extensions server_cert \
+ -days 1000 -notext -md sha256 \
+ -in broker.csr.pem -out broker.cert.pem
+```
+
+At this point, you have a cert, `broker.cert.pem`, and a key,
`broker.key-pk8.pem`, which can be used along with `ca.cert.pem` to configure
TLS transport encryption for your broker and proxy nodes.
+
+## Broker Configuration
+
+To configure a Pulsar [broker](reference-terminology.md#broker) to use TLS
transport encryption, you'll need to make some changes to `broker.conf`, which
is located in the `conf` directory of your [Pulsar
installation](getting-started-standalone.md).
+
+Add these values to the configuration file (substituting the appropriate
certificate paths where necessary):
+
+```properties
+tlsEnabled=true
+tlsCertificateFilePath=/path/to/broker.cert.pem
+tlsKeyFilePath=/path/to/broker.key-pk8.pem
+tlsTrustCertsFilePath=/path/to/ca.cert.pem
+```
+
+> A full list of parameters available in the `conf/broker.conf` file,
+> as well as the default values for those parameters, can be found in [Broker
Configuration](reference-configuration.md#broker)
+
+## Proxy Configuration
+
+Proxies need to configure TLS in two directions, for clients connecting to the
proxy, and for the proxy to be able to connect to brokers.
+
+```properties
+# For clients connecting to the proxy
+tlsEnabledInProxy=true
+tlsCertificateFilePath=/path/to/broker.cert.pem
+tlsKeyFilePath=/path/to/broker.key-pk8.pem
+tlsTrustCertsFilePath=/path/to/ca.cert.pem
+
+# For the proxy to connect to brokers
+tlsEnabledWithBroker=true
+brokerClientTrustCertsFilePath=/path/to/ca.cert.pem
+```
+
+## Client configuration
+
+When TLS transport encryption is enabled, you need to configure the client to
use ```https://``` and port 8443 for the web service URL, and
```pulsar+ssl://``` and port 6651 for the broker service URL.
+
+As the server certificate you generated above doesn't belong to any of the
default trust chains, you also need to either specify the path the **trust
cert** (recommended), or tell the client to allow untrusted server certs.
+
+### CLI tools
+
+[Command-line tools](reference-cli-tools.md) like
[`pulsar-admin`](reference-cli-tools#pulsar-admin),
[`pulsar-perf`](reference-cli-tools#pulsar-perf), and
[`pulsar-client`](reference-cli-tools#pulsar-client) use the `conf/client.conf`
config file in a Pulsar installation.
+
+You'll need to add the following parameters to that file to use TLS transport
with Pulsar's CLI tools:
+
+```properties
+webServiceUrl=https://broker.example.com:8443/
+brokerServiceUrl=pulsar+ssl://broker.example.com:6651/
+useTls=true
+tlsAllowInsecureConnection=false
+tlsTrustCertsFilePath=/path/to/ca.cert.pem
+```
+
+### Java client
+
+```java
+import org.apache.pulsar.client.api.PulsarClient;
+
+PulsarClient client = PulsarClient.builder()
+ .serviceUrl("pulsar+ssl://broker.example.com:6651/")
+ .enableTls(true)
+ .tlsTrustCertsFilePath("/path/to/ca.cert.pem")
+ .build();
+```
+
+### Python client
+
+```python
+from pulsar import Client
+
+client = Client("pulsar+ssl://broker.example.com:6651/",
+ tls_trust_certs_file_path="/path/to/ca.cert.pem",
+ tls_allow_insecure_connection=False)
+```
+
+### C++ client
+
+```c++
+#include <pulsar/Client.h>
+
+pulsar::ClientConfiguration config;
+config.setUseTls(true);
+config.setTlsTrustCertsFilePath("/path/to/ca.cert.pem");
+config.setTlsAllowInsecureConnection(false);
+
+pulsar::Client client("pulsar+ssl://broker.example.com:6651/", config);
+```
diff --git a/site2/docs/security-tls.md b/site2/docs/security-tls.md
deleted file mode 100644
index 190b5c8f06..0000000000
--- a/site2/docs/security-tls.md
+++ /dev/null
@@ -1,142 +0,0 @@
----
-id: security-tls
-title: Encryption and Authentication using TLS
-sidebar_label: Encryption and Authentication using TLS
----
-
-With [TLS](https://en.wikipedia.org/wiki/Transport_Layer_Security)
authentication, the server authenticates the client (also called “2-way
authentication”).
-Since TLS authentication requires TLS encryption, this page shows you how to
configure both at the same time.
-
-By default, Apache Pulsar communicates in plain text service url, which means
that all data is sent in the clear.
-To encrypt communication, it is recommended to configure all the Apache Pulsar
components in your deployment to use TLS encryption.
-
-TLS can be configured for encryption or authentication. You may configure just
TLS encryption
-(by default TLS encryption includes certificate authentication of the server)
and independently choose a separate mechanism
-for client authentication, e.g. TLS, [Athenz](security-athenz.md), etc. Note
that TLS encryption, technically speaking, already enables
-1-way authentication in which the client authenticates the server certificate.
So when referring to TLS authentication, it is really
-referring to 2-way authentication in which the broker also authenticates the
client certificate.
-
-> Note that enabling TLS may have a performance impact due to encryption
overhead.
-
-## Creating TLS Certificates
-
-Creating TLS certificates for Pulsar involves creating a [certificate
authority](#certificate-authority) (CA), [broker
certificate](#broker-certificate), and [client
certificate](#client-certificate).
-
-### Certificate authority
-
-The first step is to create the certificate for the CA. The CA will be used to
sign both the broker and client certificates, in order to ensure that each
party will trust the others.
-
-#### Linux
-
-```bash
-$ CA.pl -newca
-```
-
-#### macOS
-
-```bash
-$ /System/Library/OpenSSL/misc/CA.pl -newca
-```
-
-After answering the question prompts, this will store CA-related files in the
`./demoCA` directory. Within that directory:
-
-* `demoCA/cacert.pem` is the public certificate. It is meant to be distributed
to all parties involved.
-* `demoCA/private/cakey.pem` is the private key. This is only needed when
signing a new certificate for either broker or clients and it must be safely
guarded.
-
-### Broker certificate
-
-Once a CA certificate has been created, you can create certificate requests
and sign them with the CA.
-
-The following commands will ask you a few questions and then create the
certificates. When asked for the common name, you need to match the hostname of
the broker. You could also use a wildcard to match a group of broker hostnames,
for example `*.broker.usw.example.com`. This ensures that the same certificate
can be reused on multiple machines.
-
-```shell
-$ openssl req \
- -newkey rsa:2048 \
- -sha256 \
- -nodes \
- -out broker-cert.csr \
- -outform PEM
-```
-
-Convert the key to [PKCS 8](https://en.wikipedia.org/wiki/PKCS_8) format:
-
-```shell
-$ openssl pkcs8 \
- -topk8 \
- -inform PEM \
- -outform PEM \
- -in privkey.pem \
- -out broker-key.pem \
- -nocrypt
-```
-
-This will create two broker certificate files named `broker-cert.csr` and
`broker-key.pem`. Now you can create the signed certificate:
-
-```shell
-$ openssl ca \
- -out broker-cert.pem \
- -infiles broker-cert.csr
-```
-
-At this point, you should have a `broker-cert.pem` and `broker-key.pem` file.
These will be needed for the broker.
-
-### Client certificate
-
-To create a client certificate, repeat the steps in the previous section, but
did create `client-cert.pem` and `client-key.pem` files instead.
-
-For the client common name, you need to use a string that you intend to use as
the [role token](security-overview.md#role-tokens) for this client, though it
doesn't need to match the client hostname.
-
-## Configure the broker for TLS
-
-To configure a Pulsar [broker](reference-terminology.md#broker) to use TLS
authentication, you'll need to make some changes to the `broker.conf`
configuration file, which is located in the `conf` directory of your [Pulsar
installation](getting-started-standalone.md).
-
-Add these values to the configuration file (substituting the appropriate
certificate paths where necessary):
-
-```properties
-# Enable TLS and point the broker to the right certs
-tlsEnabled=true
-tlsCertificateFilePath=/path/to/broker-cert.pem
-tlsKeyFilePath=/path/to/broker-key.pem
-tlsTrustCertsFilePath=/path/to/cacert.pem
-
-# Enable the TLS auth provider
-authenticationEnabled=true
-authorizationEnabled=true
-authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderTls
-```
-
-> A full listing of parameters available in the `conf/broker.conf` file, as
well as the default
-> values for those parameters, can be found in [Broker
Configuration](reference-configuration.md#broker).
-
-## Configure the discovery service
-
-The [discovery service](reference-terminology.md#service-discovery) used by
Pulsar brokers needs to redirect all HTTPS requests, which means that it needs
to be trusted by the client as well. Add this configuration in
`conf/discovery.conf` in your Pulsar installation:
-
-```properties
-tlsEnabled=true
-tlsCertificateFilePath=/path/to/broker-cert.pem
-tlsKeyFilePath=/path/to/broker-key.pem
-```
-
-## Configure clients
-
-For more information on Pulsar client authentication using TLS, see the
following language-specific docs:
-
-* [Java client](client-libraries-java.md)
-* [C++ client](client-libraries-cpp.md)
-
-## Configure CLI tools
-
-[Command-line tools](reference-cli-tools.md) like
[`pulsar-admin`](reference-pulsar-admin.md),
[`pulsar-perf`](reference-cli-tools.md#pulsar-perf), and
[`pulsar-client`](reference-cli-tools.md#pulsar-client) use the
`conf/client.conf` config file in a Pulsar installation.
-
-You'll need to add the following authentication parameters to that file to use
TLS with Pulsar's CLI tools:
-
-```properties
-serviceUrl=https://broker.example.com:8443/
-authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationTls
-authParams=tlsCertFile:/path/to/client-cert.pem,tlsKeyFile:/path/to/client-key.pem
-useTls=true
-tlsAllowInsecureConnection=false
-tlsTrustCertsFilePath=/path/to/cacert.pem
-```
-
diff --git a/site2/website/sidebars.json b/site2/website/sidebars.json
index 48a50d43da..c852fee2fa 100644
--- a/site2/website/sidebars.json
+++ b/site2/website/sidebars.json
@@ -48,7 +48,8 @@
],
"Security": [
"security-overview",
- "security-tls",
+ "security-tls-transport",
+ "security-tls-authentication",
"security-athenz",
"security-authorization",
"security-encryption",
diff --git a/site2/website/static/examples/openssl.cnf
b/site2/website/static/examples/openssl.cnf
new file mode 100644
index 0000000000..eee8a47298
--- /dev/null
+++ b/site2/website/static/examples/openssl.cnf
@@ -0,0 +1,152 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+#
+# Configuration file for testing certificate authority.
+# The environment variable, CA_HOME, must be set to point to the directory
+# containing this file before running any openssl commands.
+#
+[ ca ]
+# `man ca`
+default_ca = CA_default
+
+[ CA_default ]
+# Directory and file locations.
+dir = $ENV::CA_HOME
+certs = $dir/certs
+crl_dir = $dir/crl
+new_certs_dir = $dir/newcerts
+database = $dir/index.txt
+serial = $dir/serial
+RANDFILE = $dir/private/.rand
+
+# The root key and root certificate.
+private_key = $dir/private/ca.key.pem
+certificate = $dir/certs/ca.cert.pem
+
+# For certificate revocation lists.
+crlnumber = $dir/crlnumber
+crl = $dir/crl/ca.crl.pem
+crl_extensions = crl_ext
+default_crl_days = 30
+
+# SHA-1 is deprecated, so use SHA-2 instead.
+default_md = sha256
+
+name_opt = ca_default
+cert_opt = ca_default
+default_days = 375
+preserve = no
+policy = policy_strict
+
+[ policy_strict ]
+# The root CA should only sign intermediate certificates that match.
+# See the POLICY FORMAT section of `man ca`.
+countryName = match
+stateOrProvinceName = match
+organizationName = match
+organizationalUnitName = optional
+commonName = supplied
+emailAddress = optional
+
+[ policy_loose ]
+# Allow the intermediate CA to sign a more diverse range of certificates.
+# See the POLICY FORMAT section of the `ca` man page.
+countryName = optional
+stateOrProvinceName = optional
+localityName = optional
+organizationName = optional
+organizationalUnitName = optional
+commonName = supplied
+emailAddress = optional
+
+[ req ]
+# Options for the `req` tool (`man req`).
+default_bits = 2048
+distinguished_name = req_distinguished_name
+string_mask = utf8only
+
+# SHA-1 is deprecated, so use SHA-2 instead.
+default_md = sha256
+
+# Extension to add when the -x509 option is used.
+x509_extensions = v3_ca
+
+[ req_distinguished_name ]
+# See <https://en.wikipedia.org/wiki/Certificate_signing_request>.
+countryName = Country Name (2 letter code)
+stateOrProvinceName = State or Province Name
+localityName = Locality Name
+0.organizationName = Organization Name
+organizationalUnitName = Organizational Unit Name
+commonName = Common Name
+emailAddress = Email Address
+
+# Optionally, specify some defaults.
+countryName_default = US
+stateOrProvinceName_default = California
+localityName_default = Palo Alto
+0.organizationName_default = My company
+organizationalUnitName_default = IT
+emailAddress_default =
+
+[ v3_ca ]
+# Extensions for a typical CA (`man x509v3_config`).
+subjectKeyIdentifier = hash
+authorityKeyIdentifier = keyid:always,issuer
+basicConstraints = critical, CA:true
+keyUsage = critical, digitalSignature, cRLSign, keyCertSign
+
+[ v3_intermediate_ca ]
+# Extensions for a typical intermediate CA (`man x509v3_config`).
+subjectKeyIdentifier = hash
+authorityKeyIdentifier = keyid:always,issuer
+basicConstraints = critical, CA:true, pathlen:0
+keyUsage = critical, digitalSignature, cRLSign, keyCertSign
+
+[ usr_cert ]
+# Extensions for client certificates (`man x509v3_config`).
+basicConstraints = CA:FALSE
+nsCertType = client, email
+nsComment = "OpenSSL Generated Client Certificate"
+subjectKeyIdentifier = hash
+authorityKeyIdentifier = keyid,issuer
+keyUsage = critical, nonRepudiation, digitalSignature, keyEncipherment
+extendedKeyUsage = clientAuth, emailProtection
+
+[ server_cert ]
+# Extensions for server certificates (`man x509v3_config`).
+basicConstraints = CA:FALSE
+nsCertType = server
+nsComment = "OpenSSL Generated Server Certificate"
+subjectKeyIdentifier = hash
+authorityKeyIdentifier = keyid,issuer:always
+keyUsage = critical, digitalSignature, keyEncipherment
+extendedKeyUsage = serverAuth
+
+[ crl_ext ]
+# Extension for CRLs (`man x509v3_config`).
+authorityKeyIdentifier=keyid:always
+
+[ ocsp ]
+# Extension for OCSP signing certificates (`man ocsp`).
+basicConstraints = CA:FALSE
+subjectKeyIdentifier = hash
+authorityKeyIdentifier = keyid,issuer
+keyUsage = critical, digitalSignature
+extendedKeyUsage = critical, OCSPSigning
\ No newline at end of file
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
