This is an automated email from the ASF dual-hosted git repository.
zike pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 23695a97b6b [cleanup][doc] Merge the broker and proxy authentication
configuration (#18343)
23695a97b6b is described below
commit 23695a97b6b0efd998d1b83e183875b70d138497
Author: Zixuan Liu <[email protected]>
AuthorDate: Wed Nov 16 09:04:54 2022 +0800
[cleanup][doc] Merge the broker and proxy authentication configuration
(#18343)
### Motivation
Authentication documentation has too many duplicate configurations and some
unnecessary configurations.
Signed-off-by: Zixuan Liu <[email protected]>
Co-authored-by: momo-jun <[email protected]>
---
site2/docs/security-athenz.md | 38 ++------------
site2/docs/security-basic-auth.md | 40 ++------------
site2/docs/security-jwt.md | 42 ++-------------
site2/docs/security-kerberos.md | 2 +-
site2/docs/security-oauth2.md | 19 ++++---
site2/docs/security-overview.md | 18 ++++---
site2/docs/security-tls-authentication.md | 87 +++++--------------------------
7 files changed, 48 insertions(+), 198 deletions(-)
diff --git a/site2/docs/security-athenz.md b/site2/docs/security-athenz.md
index 3bccffd70b7..5d10173dd09 100644
--- a/site2/docs/security-athenz.md
+++ b/site2/docs/security-athenz.md
@@ -36,42 +36,17 @@ Note that you can specify any action and resource in step 2
since they are not u
For more specific steps involving the Athenz UI, refer to [Example Service
Access Control
Setup](https://github.com/AthenZ/athenz/blob/master/docs/example_service_athenz_setup.md#server-provider-domain).
-## Enable Athenz authentication on brokers
+## Enable Athenz authentication on brokers/proxies
-:::note
-
-When you are using Athenz as an authentication provider, it's highly
recommended to use [TLS encryption](security-tls-transport.md) as it can
protect role tokens from being intercepted and reused. For more details
involving TLS encryption, see [Architecture - Data
Model](https://github.com/AthenZ/athenz/blob/master/docs/data_model).
-
-:::
-
-In the `conf/broker.conf` configuration file in your Pulsar installation, you
need to provide the class name of the Athenz authentication provider as well as
a comma-separated list of provider domain names.
-
-```properties
-# Add the Athenz auth provider
-authenticationEnabled=true
-authorizationEnabled=true
-authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderAthenz
-athenzDomainNames=pulsar
-
-# Authentication settings of the broker itself. Used when the broker connects
to other brokers, either in same or other clusters
-brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationAthenz
-brokerClientAuthenticationParameters={"tenantDomain":"shopping","tenantService":"some_app","providerDomain":"pulsar","privateKey":"file:///path/to/private.pem","keyId":"v1"}
-```
-
-> A full listing of parameters is available in the `conf/broker.conf` file,
you can also find the default
-> values for those parameters in [Broker
Configuration](reference-configuration.md#broker).
-
-## Enable Athenz authentication on proxies
-
-Configure the required parameters in the `conf/proxy.conf` file in your Pulsar
installation.
+To configure brokers/proxies to authenticate clients using Authenz, add the
following parameters to the `conf/broker.conf` and the `conf/proxy.conf` files
and provide the class name of the Athenz authentication provider as well as a
comma-separated list of provider domain names. If you use a standalone Pulsar,
you need to add these parameters to the `conf/standalone.conf` file.
```properties
# Add the Athenz auth provider
authenticationEnabled=true
-authorizationEnabled=true
authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderAthenz
athenzDomainNames=pulsar
+# Authentication settings of the broker itself. Used when the broker connects
to other brokers, or when the proxy connects to brokers, either in same or
other clusters
brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationAthenz
brokerClientAuthenticationParameters={"tenantDomain":"shopping","tenantService":"some_app","providerDomain":"pulsar","privateKey":"file:///path/to/private.pem","keyId":"v1"}
```
@@ -205,14 +180,9 @@ You need to add the following authentication parameters to
the `conf/client.conf
```properties
# URL for the broker
-serviceUrl=https://broker.example.com:8443/
+serviceUrl=http://broker.example.com:8080
# Set Athenz auth plugin and its parameters
authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationAthenz
authParams={"tenantDomain":"shopping","tenantService":"some_app","providerDomain":"pulsar","privateKey":"file:///path/to/private.pem","keyId":"v1"}
-
-# Enable TLS
-useTls=true
-tlsAllowInsecureConnection=false
-tlsTrustCertsFilePath=/path/to/cacert.pem
```
\ No newline at end of file
diff --git a/site2/docs/security-basic-auth.md
b/site2/docs/security-basic-auth.md
index 608a1281bef..b6aad41114f 100644
--- a/site2/docs/security-basic-auth.md
+++ b/site2/docs/security-basic-auth.md
@@ -55,9 +55,9 @@ cat path/to/.htpasswd
superuser:$apr1$GBIYZYFZ$MzLcPrvoUky16mLcK6UtX/
```
-## Enable basic authentication on brokers
+## Enable basic authentication on brokers/proxies
-To configure brokers to authenticate clients, add the following parameters to
the `conf/broker.conf` file. If you use a standalone Pulsar, you need to add
these parameters to the `conf/standalone.conf` file:
+To configure brokers/proxies to authenticate clients using basic, add the
following parameters to the `conf/broker.conf` and the `conf/proxy.conf` file.
If you use a standalone Pulsar, you need to add these parameters to the
`conf/standalone.conf` file:
```conf
# Configuration to enable Basic authentication
@@ -70,43 +70,9 @@ basicAuthConf=file:///path/to/.htpasswd
# basicAuthConf=data:;base64,YOUR-BASE64
# basicAuthConf=YOUR-BASE64
-# Authentication settings of the broker itself. Used when the broker connects
to other brokers, either in same or other clusters
+# Authentication settings of the broker itself. Used when the broker connects
to other brokers, or when the proxy connects to brokers, either in same or
other clusters
brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationBasic
brokerClientAuthenticationParameters={"userId":"superuser","password":"admin"}
-
-# If this flag is set then the broker authenticates the original Auth data
-# else it just accepts the originalPrincipal and authorizes it (if required).
-authenticateOriginalAuthData=true
-```
-
-:::note
-
-You can also set an environment variable named `PULSAR_EXTRA_OPTS` and the
value is `-Dpulsar.auth.basic.conf=/path/to/.htpasswd`. Pulsar reads this
environment variable to implement HTTP basic authentication.
-
-:::
-
-## Enable basic authentication on proxies
-
-To configure proxies to authenticate clients, add the following parameters to
the `conf/proxy.conf` file:
-
-```conf
-# For clients connecting to the proxy
-authenticationEnabled=true
-authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderBasic
-
-basicAuthConf=file:///path/to/.htpasswd
-# basicAuthConf=/path/to/.htpasswd
-# When use the base64 format, you need to encode the .htpaswd content to bas64
-# basicAuthConf=data:;base64,YOUR-BASE64
-# basicAuthConf=YOUR-BASE64
-
-# For the proxy to connect to brokers
-brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationBasic
-brokerClientAuthenticationParameters={"userId":"superuser","password":"admin"}
-
-# Whether client authorization credentials are forwarded to the broker for
re-authorization.
-# Authentication must be enabled via authenticationEnabled=true for this to
take effect.
-forwardAuthorizationCredentials=true
```
:::note
diff --git a/site2/docs/security-jwt.md b/site2/docs/security-jwt.md
index 88d0ac408d9..2ca2737db8d 100644
--- a/site2/docs/security-jwt.md
+++ b/site2/docs/security-jwt.md
@@ -91,27 +91,22 @@ The token itself does not have any permission associated.
You need to [enable au
:::
-## Enable JWT authentication on brokers
+## Enable JWT authentication on brokers/proxies
-To configure brokers to authenticate clients using JWT, add the following
parameters to the `conf/broker.conf` or `conf/standalone.conf` file.
+To configure brokers/proxies to authenticate clients using JWT, add the
following parameters to the `conf/broker.conf` and the `conf/proxy.conf` file.
If you use a standalone Pulsar, you need to add these parameters to the
`conf/standalone.conf` file:
```properties
# Configuration to enable authentication
authenticationEnabled=true
authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderToken
-# Authentication settings of the broker itself. Used when the broker connects
to other brokers, either in same or other clusters
+# Authentication settings of the broker itself. Used when the broker connects
to other brokers, or when the proxy connects to brokers, either in same or
other clusters
brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationToken
brokerClientAuthenticationParameters={"token":"eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0LXVzZXIifQ.9OHgE9ZUDeBTZs7nSMEFIuGNEX18FLR3qvy8mqxSxXw"}
# Either configure the token string or specify to read it from a file. The
following three available formats are all valid:
# brokerClientAuthenticationParameters={"token":"your-token-string"}
# brokerClientAuthenticationParameters=token:your-token-string
# brokerClientAuthenticationParameters=file:///path/to/token
-brokerClientTrustCertsFilePath=/path/my-ca/certs/ca.cert.pem
-
-# If this flag is set then the broker authenticates the original Auth data
-# else it just accepts the originalPrincipal and authorizes it (if required).
-authenticateOriginalAuthData=true
# If using secret key (Note: key files must be DER-encoded)
tokenSecretKey=file:///path/to/secret.key
@@ -122,37 +117,6 @@ tokenSecretKey=file:///path/to/secret.key
# tokenPublicKey=file:///path/to/public.key
```
-:::note
-
-Equivalent to `brokerClientAuthenticationParameters`, you need to configure
`authParams` in the `conf/client.conf` file.
-
-:::
-
-## Enable JWT authentication on proxies
-
-To configure proxies to authenticate clients using JWT, add the following
parameters to the `conf/proxy.conf` file.
-
-```properties
-# For clients connecting to the proxy
-authenticationEnabled=true
-authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderToken
-tokenSecretKey=file:///path/to/secret.key
-
-# For the proxy to connect to brokers
-brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationToken
-brokerClientAuthenticationParameters={"token":"eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0ZXN0LXVzZXIifQ.9OHgE9ZUDeBTZs7nSMEFIuGNEX18FLR3qvy8mqxSxXw"}
-# Either configure the token string or specify to read it from a file. The
following three available formats are all valid:
-# brokerClientAuthenticationParameters={"token":"your-token-string"}
-# brokerClientAuthenticationParameters=token:your-token-string
-# brokerClientAuthenticationParameters=file:///path/to/token
-```
-
-:::note
-
-The proxy uses its own token when connecting to brokers. You need to configure
the role token for this key pair in the `proxyRoles` of the brokers. For more
details, see [authorization](security-authorization.md).
-
-:::
-
## Configure JWT authentication in CLI Tools
[Command-line tools](reference-cli-tools.md) like
[`pulsar-admin`](/tools/pulsar-admin/),
[`pulsar-perf`](reference-cli-tools.md), and
[`pulsar-client`](reference-cli-tools.md) use the `conf/client.conf` config
file in a Pulsar installation.
diff --git a/site2/docs/security-kerberos.md b/site2/docs/security-kerberos.md
index 9d7b7ddf1aa..150173bebae 100644
--- a/site2/docs/security-kerberos.md
+++ b/site2/docs/security-kerberos.md
@@ -56,7 +56,7 @@
authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationPr
saslJaasClientAllowedIds=.*client.* ## regex for principals that are allowed
to connect to brokers
saslJaasServerSectionName=PulsarBroker ## corresponds to the section in the
JAAS configuration file for brokers
-## Authentication settings of the broker itself. Used when the broker connects
to other brokers
+# Authentication settings of the broker itself. Used when the broker connects
to other brokers, or when the proxy connects to brokers, either in same or
other clusters
brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationSasl
brokerClientAuthenticationParameters={"saslJaasClientSectionName":"PulsarClient",
"serverType":"broker"}
```
diff --git a/site2/docs/security-oauth2.md b/site2/docs/security-oauth2.md
index abedc3bba57..bd994e26419 100644
--- a/site2/docs/security-oauth2.md
+++ b/site2/docs/security-oauth2.md
@@ -15,18 +15,25 @@ After communicating with the OAuth 2.0 server, the Pulsar
client gets an access
## Enable OAuth2 authentication on brokers/proxies
-To configure brokers to authenticate clients using OAuth2， add the following
parameters to the `conf/broker.conf` and `conf/proxy.conf` file.
+To configure brokers/proxies to authenticate clients using OAuth2, add the
following parameters to the `conf/broker.conf` and the `conf/proxy.conf` file.
If you use a standalone Pulsar, you need to add these parameters to the
`conf/standalone.conf` file:
```properties
# Configuration to enable authentication
authenticationEnabled=true
authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderToken
-tokenPublicKey=/path/to/publicKey
-# Authentication settings of the broker itself. Used when the broker connects
to other brokers,
-# either in same or other clusters
+
+# Authentication settings of the broker itself. Used when the broker connects
to other brokers, or when the proxy connects to brokers, either in same or
other clusters
brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.oauth2.AuthenticationOAuth2
-brokerClientAuthenticationParameters={"privateKey":"/path/to/privateKey",\
-
"audience":"https://dev-kt-aa9ne.us.auth0.com/api/v2/","issuerUrl":"https://dev-kt-aa9ne.us.auth0.com"}
+brokerClientAuthenticationParameters={"privateKey":"file:///path/to/privateKey","audience":"https://dev-kt-aa9ne.us.auth0.com/api/v2/","issuerUrl":"https://dev-kt-aa9ne.us.auth0.com"}
+#
brokerClientAuthenticationParameters={"privateKey":"data:application/json;base64,privateKey-body-to-base64","audience":"https://dev-kt-aa9ne.us.auth0.com/api/v2/","issuerUrl":"https://dev-kt-aa9ne.us.auth0.com"}
+
+# If using secret key (Note: key files must be DER-encoded)
+tokenSecretKey=file:///path/to/secret.key
+# The key can also be passed inline:
+# tokenSecretKey=data:;base64,FLFyW0oLJ2Fi22KKCm21J18mbAdztfSHN/lAT5ucEKU=
+
+# If using public/private (Note: key files must be DER-encoded)
+# tokenPublicKey=file:///path/to/public.key
```
## Configure OAuth2 authentication in Pulsar clients
diff --git a/site2/docs/security-overview.md b/site2/docs/security-overview.md
index 917b7ac7af2..a5b88c21a38 100644
--- a/site2/docs/security-overview.md
+++ b/site2/docs/security-overview.md
@@ -28,7 +28,7 @@ Encryption ensures that if an attacker gets access to your
data, the attacker ca
Authentication is the process of verifying the identity of clients. In Pulsar,
the authentication provider is responsible for properly identifying clients and
associating them with role tokens. Note that if you only enable authentication,
an authenticated role token can access all resources in the cluster.
-**How it works in Pulsar**
+### How it works in Pulsar
Pulsar provides a pluggable authentication framework, and Pulsar
brokers/proxies use this mechanism to authenticate clients.
@@ -38,13 +38,17 @@ The way how each client passes its authentication data to
brokers varies dependi
- If a client supports authentication refreshing and the credential is
expired, brokers send the `CommandAuthChallenge` command to exchange the
authentication data with the client. If the next check finds that the previous
authentication exchange has not been returned, brokers disconnect the client.
- If a client does not support authentication refreshing and the credential
is expired, brokers disconnect the client.
-:::note
+### Authentication data limitations on the proxies
-When you use proxies between clients and brokers, brokers only authenticate
proxies (known as **self-authentication**) by default. To forward the
authentication data from clients to brokers for client authentication (known as
**original authentication**), you need to:
-1. Set `forwardAuthorizationCredentials` to `true` in the `conf/proxy.conf`
file.
-2. Set `authenticateOriginalAuthData` to `true` in the `conf/broker.conf`
file, which ensures that brokers recheck the client authentication.
+When you use proxies between clients and brokers, there are two authentication
data:
+* authentication data from proxies that brokers default to authenticate -
known as **self-authentication**.
+* authentication data from clients that proxies forward to brokers for
authenticating - known as **original authentication**.
-:::
+**Important:** If your authentication data contains an expiration time, or
your authorization provider depends on the authentication data, you must:
+
+1. Ensure your authentication data of proxies has no expiration time since
brokers don't support refreshing this authentication data.
+2. Set `forwardAuthorizationCredentials` to `true` in the `conf/proxy.conf`
file.
+3. Set `authenticateOriginalAuthData` to `true` in the `conf/broker.conf`
file, which ensures that brokers recheck the client authentication.
**What's next?**
@@ -59,7 +63,7 @@ When you use proxies between clients and brokers, brokers
only authenticate prox
:::note
-Starting from 2.11.0, [TLS authentication](security-tls-authentication.md)
includes [TLS encryption](security-tls-transport.md) by default. If you
configure TLS authentication first, then TLS encryption automatically applies;
if you configure TLS encryption first, you can select any one of the above
authentication providers.
+Starting from 2.11.0, you can configure [Mutual
TLS](security-tls-transport.md) with any one of the above authentication
providers.
:::
diff --git a/site2/docs/security-tls-authentication.md
b/site2/docs/security-tls-authentication.md
index 7b86ca8e906..f37d5db1fb3 100644
--- a/site2/docs/security-tls-authentication.md
+++ b/site2/docs/security-tls-authentication.md
@@ -13,91 +13,28 @@ import TabItem from '@theme/TabItem';
TLS authentication is an extension of [TLS transport
encryption](security-tls-transport.md). Not only servers have keys and certs
that the client uses to verify the identity of servers, clients also have keys
and certs that the server uses to verify the identity of clients. You must have
TLS transport encryption configured on your cluster before you can use TLS
authentication. This guide assumes you already have TLS transport encryption
configured.
-## Create client certificates
+## Enable TLS authentication on brokers/proxies
-Client certificates are generated using the certificate authority. Server
certificates are also generated with the same certificate authority.
-
-The biggest difference between client certs and server certs is that the
**common name** for the client certificate is the **role token** that the
client is authenticated as.
-
-To use client certificates, you need to set
`tlsRequireTrustedClientCertOnConnect=true` at the broker side. For details,
refer to [TLS broker
configuration](security-tls-transport.md#configure-brokers).
-
-First, you need to enter the following command to generate the key :
-
-```bash
-openssl genrsa -out admin.key.pem 2048
-```
-
-Similar to the broker, the client expects the key to be in [PKCS
8](https://en.wikipedia.org/wiki/PKCS_8) format, so you need to convert it by
entering the following command:
-
-```bash
-openssl pkcs8 -topk8 -inform PEM -outform PEM \
- -in admin.key.pem -out admin.key-pk8.pem -nocrypt
-```
-
-Next, enter the command below to generate the certificate request. When you
are asked for a **common name**, enter the **role token** that you want this
key pair to authenticate a client as.
-
-```bash
-openssl req -config openssl.cnf \
- -key admin.key.pem -new -sha256 -out admin.csr.pem
-```
-
-:::note
-
-If `openssl.cnf` is not specified, read [Certificate
authority](security-tls-transport.md#create-a-certificate-authority) to get
`openssl.cnf`.
-
-:::
-
-Then, enter the command below to sign with a request with the certificate
authority. Note that the client certs use the **usr_cert** extension, which
allows the cert to be used for client authentication.
-
-```bash
-openssl ca -config openssl.cnf -extensions usr_cert \
- -days 1000 -notext -md sha256 \
- -in admin.csr.pem -out admin.cert.pem
-```
-
-You can get a cert, `admin.cert.pem`, and a key, `admin.key-pk8.pem` from this
command. With `ca.cert.pem`, clients can use this cert and this key to
authenticate themselves to brokers and proxies as the role token ``admin``.
-
-:::note
-
-If the "unable to load CA private key" error occurs and the reason for this
error is "No such file or directory: /etc/pki/CA/private/cakey.pem" in this
step. Try the command below to generate `cakey.pem`.
-
-```bash
-cd /etc/pki/tls/misc/CA
-./CA -newca
-```
-
-:::
-
-## Enable TLS authentication on brokers
-
-To configure brokers to authenticate clients, add the following parameters to
`broker.conf`, alongside [the configuration to enable TLS
transport](security-tls-transport.md#configure-brokers):
+To configure brokers/proxies to authenticate clients using Mutual TLS, add the
following parameters to the `conf/broker.conf` and the `conf/proxy.conf` file.
If you use a standalone Pulsar, you need to add these parameters to the
`conf/standalone.conf` file:
```properties
# Configuration to enable authentication
authenticationEnabled=true
authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderTls
-tlsRequireTrustedClientCertOnConnect=true
brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationTls
-brokerClientAuthenticationParameters={"tlsCertFile":"/path/my-ca/admin.cert.pem","tlsKeyFile":"/path/my-ca/admin.key-pk8.pem"}
-brokerClientTrustCertsFilePath=/path/my-ca/certs/ca.cert.pem
-```
-
-## Enable TLS authentication on proxies
-
-To configure proxies to authenticate clients, add the following parameters to
`proxy.conf`, alongside [the configuration to enable TLS
transport](security-tls-transport.md#configure-proxies):
+brokerClientAuthenticationParameters={"tlsCertFile":"/path/to/admin.cert.pem","tlsKeyFile":"/path/to/admin.key-pk8.pem"}
+brokerClientTrustCertsFilePath=/path/to/ca.cert.pem
-The proxy should have its own client key pair for connecting to brokers. You
need to configure the role token for this key pair in the `proxyRoles` of the
brokers. See the [authorization guide](security-authorization.md) for more
details.
+tlsCertificateFilePath=/path/to/broker.cert.pem
+tlsKeyFilePath=/path/to/broker.key-pk8.pem
+tlsTrustCertsFilePath=/path/to/ca.cert.pem
-```properties
-# For clients connecting to the proxy
-authenticationEnabled=true
-authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderTls
tlsRequireTrustedClientCertOnConnect=true
+tlsAllowInsecureConnection=false
-# For the proxy to connect to brokers
-brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationTls
-brokerClientAuthenticationParameters=tlsCertFile:/path/to/proxy.cert.pem,tlsKeyFile:/path/to/proxy.key-pk8.pem
+# Tls cert refresh duration in seconds (set 0 to check on every new connection)
+tlsCertRefreshCheckDurationSec=300
```
## Configure TLS authentication in Pulsar clients
@@ -223,7 +160,6 @@
authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationPr
# Enable KeyStore type
tlsEnabledWithKeyStore=true
-tlsRequireTrustedClientCertOnConnect=true
# key store
tlsKeyStoreType=JKS
@@ -244,6 +180,9 @@ brokerClientTlsTrustStorePassword=clientpw
# internal auth config
brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationKeyStoreTls
brokerClientAuthenticationParameters={"keyStoreType":"JKS","keyStorePath":"/var/private/tls/client.keystore.jks","keyStorePassword":"clientpw"}
+
+tlsRequireTrustedClientCertOnConnect=true
+tlsAllowInsecureConnection=false
```
### Configure clients
