ivankelly commented on a change in pull request #2220: Rework TLS docs to separate concerns URL: https://github.com/apache/incubator-pulsar/pull/2220#discussion_r204910002
########## File path: site/docs/latest/security/tls.md ########## @@ -23,139 +23,134 @@ title: Encryption and Authentication using TLS --> -## TLS Overview +## TLS Authentication Overview -With [TLS](https://en.wikipedia.org/wiki/Transport_Layer_Security) authentication, the server authenticates the client (also called “2-way authentication”). -Since TLS authentication requires TLS encryption, this page shows you how to configure both at the same time. +TLS authentication is an extension of [TLS transport encryption](../tls-transport), but instead of only servers having keys and certs which the client uses the verify the server's identity, clients also have keys and certs which the server uses to verify the client's identity. You must have TLS transport encryption configured on your cluster before you can use TLS authentication. This guide assumes you already have TLS transport encryption configured. -By default, Apache Pulsar communicates in plain text service url, which means that all data is sent in the clear. -To encrypt communication, it is recommended to configure all the Apache Pulsar components in your deployment to use TLS encryption. +### Creating client certificates -TLS can be configured for encryption or authentication. You may configure just TLS encryption -(by default TLS encryption includes certificate authentication of the server) and independently choose a separate mechanism -for client authentication, e.g. TLS, [Athenz](../athenz), etc. Note that TLS encryption, technically speaking, already enables -1-way authentication in which the client authenticates the server certificate. So when referring to TLS authentication, it is really -referring to 2-way authentication in which the broker also authenticates the client certificate. +Client certificates are generated using the same certificate authority as was used to generate the server certificates. -> Note that enabling TLS may have a performance impact due to encryption overhead. +The biggest difference between client certs and server certs is that the **common name** for the client certificate is the **role token** which that client will be authenticated as. -## Creating TLS Certificates - -Creating TLS certificates for Pulsar involves creating a [certificate authority](#certificate-authority) (CA), [broker certificate](#broker-certificate), and [client certificate](#client-certificate). +First generate the key. +```bash +$ openssl genrsa -out admin.key.pem 2048 +``` -### Certificate authority +Similar to the broker, the client expects the key to be in [PKCS 8](https://en.wikipedia.org/wiki/PKCS_8) format, so convert it. -The first step is to create the certificate for the CA. The CA will be used to sign both the broker and client certificates, in order to ensure that each party will trust the others. +```bash +$ openssl pkcs8 -topk8 -inform PEM -outform PEM \ + -in admin.key.pem -out admin.key-pk8.pem -nocrypt +``` -#### Linux +Generate the certificate request. When asked for a **common name**, enter the **role token** which you want this key pair to authenticate a client as. ```bash -$ CA.pl -newca +$ openssl req -config openssl.cnf \ + -key admin.key.pem -new -sha256 -out admin.cert.pem ``` -#### macOS +Sign with request with the certificate authority. Note that that client certs uses the **usr_cert** extension, which allows the cert to be used for client authentication. ```bash -$ /System/Library/OpenSSL/misc/CA.pl -newca +$ openssl ca -config openssl.cnf -extensions usr_cert \ + -days 1000 -notext -md sha256 \ + -in admin.csr.pem -out admin.cert.pem ``` -After answering the question prompts, this will store CA-related files in the `./demoCA` directory. Within that directory: +This will give you a cert, `admin.cert.pem`, and a key, `admin.key-pk8.pem`, which, with `ca.cert.pem`, can be used by clients to authenticate themselves to brokers and proxies as the role token ``admin``. -* `demoCA/cacert.pem` is the public certificate. It is meant to be distributed to all parties involved. -* `demoCA/private/cakey.pem` is the private key. This is only needed when signing a new certificate for either broker or clients and it must be safely guarded. +## Enabling TLS Authentication ... -### Broker certificate +### ... on Brokers -Once a CA certificate has been created, you can create certificate requests and sign them with the CA. +To configure brokers to authenticate clients, put the following in `broker.conf`, alongside [the configuration to enable tls transport](../tls-transport#broker-configuration): -The following commands will ask you a few questions and then create the certificates. When asked for the common name, you need to match the hostname of the broker. You could also use a wildcard to match a group of broker hostnames, for example `*.broker.usw.example.com`. This ensures that the same certificate can be reused on multiple machines. - -```shell -$ openssl req \ - -newkey rsa:2048 \ - -sha256 \ - -nodes \ - -out broker-cert.csr \ - -outform PEM +```properties +# Configuration to enable authentication +authenticationEnabled=true +authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderTls ``` -Convert the key to [PKCS 8](https://en.wikipedia.org/wiki/PKCS_8) format: +### ... on Proxies -```shell -$ openssl pkcs8 \ - -topk8 \ - -inform PEM \ - -outform PEM \ - -in privkey.pem \ - -out broker-key.pem \ - -nocrypt -``` - -This will create two broker certificate files named `broker-cert.csr` and `broker-key.pem`. Now you can create the signed certificate: +To configure proxies to authenticate clients, put the folling in `proxy.conf`, alongside [the configuration to enable tls transport](../tls-transport#proxy-configuration): -```shell -$ openssl ca \ - -out broker-cert.pem \ - -infiles broker-cert.csr -``` +The proxy should have its own client key pair for connecting to brokers. The role token for this key pair should be configured in the ``proxyRoles`` of the brokers. See the [authorization guide](../authorization) for more details. -At this point, you should have a `broker-cert.pem` and `broker-key.pem` file. These will be needed for the broker. +```properties +# For clients connecting to the proxy +authenticationEnabled=true +authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderTls -### Client certificate +# For the proxy to connect to brokers +brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationTls +brokerClientAuthenticationParameters=tlsCertFile:/path/to/proxy.cert.pem,tlsKeyFile:/path/to/proxy.key-pk8.pem +``` -To create a client certificate, repeat the steps in the previous section, but did create `client-cert.pem` and `client-key.pem` files instead. +## Client configuration -For the client common name, you need to use a string that you intend to use as the [role token](../overview#role-tokens) for this client, though it doesn't need to match the client hostname. +When TLS authentication, the client needs to connect via TLS transport, so you need to configure the client to use ```https://``` and port 8443 for the web service URL, and ```pulsar+ssl://``` and port 6651 for the broker service URL. -## Configure the broker for TLS +### CLI tools -To configure a Pulsar {% popover broker %} to use TLS authentication, you'll need to make some changes to the `broker.conf` configuration file, which is located in the `conf` directory of your [Pulsar installation](../../getting-started/LocalCluster). +[Command-line tools](../../reference/CliTools) like [`pulsar-admin`](../../reference/CliTools#pulsar-admin), [`pulsar-perf`](../../reference/CliTools#pulsar-perf), and [`pulsar-client`](../../reference/CliTools#pulsar-client) use the `conf/client.conf` config file in a Pulsar installation. -Add these values to the configuration file (substituting the appropriate certificate paths where necessary): +You'll need to add the following parameters to that file to use TLS authentication with Pulsar's CLI tools: ```properties -# Enable TLS and point the broker to the right certs -tlsEnabled=true -tlsCertificateFilePath=/path/to/broker-cert.pem -tlsKeyFilePath=/path/to/broker-key.pem -tlsTrustCertsFilePath=/path/to/cacert.pem - -# Enable the TLS auth provider -authenticationEnabled=true -authorizationEnabled=true -authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderTls +webServiceUrl=https://broker.example.com:8443/ +brokerServiceUrl=pulsar+ssl://broker.example.com:6651/ +useTls=true +tlsAllowInsecureConnection=false +tlsTrustCertsFilePath=/path/to/ca.cert.pem +authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationTls +authParams=tlsCertFile:/path/to/my-role.cert.pem,tlsKeyFile:/path/to/my-role.key-pk8.pem ``` -{% include message.html id="broker_conf_doc" %} +### Java client -## Configure the discovery service +```java +import org.apache.pulsar.client.api.PulsarClient; -The {% popover discovery %} service used by Pulsar brokers needs to redirect all HTTPS requests, which means that it needs to be trusted by the client as well. Add this configuration in `conf/discovery.conf` in your Pulsar installation: - -```properties -tlsEnabled=true -tlsCertificateFilePath=/path/to/broker-cert.pem -tlsKeyFilePath=/path/to/broker-key.pem +PulsarClient client = PulsarClient.builder() + .serviceUrl("pulsar+ssl://broker.example.com:6651/") + .enableTls(true) + .tlsTrustCertsFilePath("/path/to/ca.cert.pem") + .authentication("org.apache.pulsar.client.impl.auth.AuthenticationTls", + "tlsCertFile:/path/to/my-role.cert.pem,tlsKeyFile:/path/to/my-role.key-pk8.pem") + .build(); ``` -## Configure clients +### Python client -For more information on Pulsar client authentication using TLS, see the following language-specific docs: +```python +from pulsar import Client, Authentication -* [Java client](../../clients/Java) -* [C++ client](../../clients/Cpp) +auth = Authentication("tls", + "tlsCertFile:/path/to/my-role.cert.pem,tlsKeyFile:/path/to/my-role.key-pk8.pem") Review comment: This works too (since the fix for auth with functions). ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
