Anonymitaet commented on a change in pull request #7534:
URL: https://github.com/apache/pulsar/pull/7534#discussion_r454761678
##########
File path: site2/docs/functions-runtime.md
##########
@@ -138,20 +149,75 @@ roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: functions-worker
-subjects:
+subjectsKubernetesSec:
- kind: ServiceAccount
name: functions-worker
```
+If the service-account is not properly configured, you may see an error
message similar to this:
+```bash
+22:04:27.696 [Timer-0] ERROR
org.apache.pulsar.functions.runtime.KubernetesRuntimeFactory - Error while
trying to fetch configmap
example-pulsar-4qvmb5gur3c6fc9dih0x1xn8b-function-worker-config at namespace
pulsar
+io.kubernetes.client.ApiException: Forbidden
+ at io.kubernetes.client.ApiClient.handleResponse(ApiClient.java:882)
~[io.kubernetes-client-java-2.0.0.jar:?]
+ at io.kubernetes.client.ApiClient.execute(ApiClient.java:798)
~[io.kubernetes-client-java-2.0.0.jar:?]
+ at
io.kubernetes.client.apis.CoreV1Api.readNamespacedConfigMapWithHttpInfo(CoreV1Api.java:23673)
~[io.kubernetes-client-java-api-2.0.0.jar:?]
+ at
io.kubernetes.client.apis.CoreV1Api.readNamespacedConfigMap(CoreV1Api.java:23655)
~[io.kubernetes-client-java-api-2.0.0.jar:?]
+ at
org.apache.pulsar.functions.runtime.KubernetesRuntimeFactory.fetchConfigMap(KubernetesRuntimeFactory.java:284)
[org.apache.pulsar-pulsar-functions-runtime-2.4.0-42c3bf949.jar:2.4.0-42c3bf949]
+ at
org.apache.pulsar.functions.runtime.KubernetesRuntimeFactory$1.run(KubernetesRuntimeFactory.java:275)
[org.apache.pulsar-pulsar-functions-runtime-2.4.0-42c3bf949.jar:2.4.0-42c3bf949]
+ at java.util.TimerThread.mainLoop(Timer.java:555) [?:1.8.0_212]
+ at java.util.TimerThread.run(Timer.java:505) [?:1.8.0_212]
+```
+
+### Kubernetes Secrets integration
+
+In order to safely distribute secrets, Pulasr Functions can reference
Kubernetes secrets. To enable this, set the
`secretsProviderConfiguratorClassName` to
`org.apache.pulsar.functions.secretsproviderconfigurator.KubernetesSecretsProviderConfigurator`.
+
+Then, you can create a secret in the namespace where your functions will be
deployed. As an example, suppose we are deploying functions to the
`pulsar-func` Kubernetes namespace, and we have a secret named `database-creds`
with a field name `password`, which we want to mount in the pod as an
environment variable called `DATABASE_PASSWORD`.
+
+The following functions configuration would then allow us to reference that
secret and have the value be mounted as an environment variable in the pod.
+
+```Yaml
+tenant: "mytenant"
+namespace: "mynamespace"
+name: "myfunction"
+topicName: "persistent://mytenant/mynamespace/myfuncinput"
+className: "com.company.pulsar.myfunction"
+
+secrets:
+ # the secret will be mounted from the `password` field in the
`database-creds` secret as an env var called `DATABASE_PASSWORD`
+ DATABASE_PASSWORD:
+ path: "database-creds"
+ key: "password"
+
+```
+
+### Kubernetes Functions authentication
+
+
+When your Pulsar cluster uses authentication, the pod running your function
needs a mechanism to authenticate with the Broker.
Review comment:
```suggestion
When your Pulsar cluster uses authentication, the pod running your function
needs a mechanism to authenticate with the broker.
```
##########
File path: site2/docs/functions-runtime.md
##########
@@ -46,6 +46,17 @@ processContainerFactory:
## Configure Kubernetes runtime
+
+### How it works
+
+The Kubernetes runtime works by having the functions-worker generate and apply
kubernetes manifests. In the event that the functions worker is running on
Kubernetes already, it can use the `serviceAccount` that is associated with the
pod the functions worker is running in. Otherwise, it can be configured to
communicate with a Kubernetes cluster.
Review comment:
```suggestion
The Kubernetes runtime works by having the functions worker generate and
apply Kubernetes manifests. In the event that the functions worker is running
on Kubernetes already, it can use the `serviceAccount` that is associated with
the pod the functions worker is running in. Otherwise, it can be configured to
communicate with a Kubernetes cluster.
```
##########
File path: site2/docs/functions-runtime.md
##########
@@ -78,9 +89,23 @@ kubernetesContainerFactory:
percentMemoryPadding: 10
```
-If you have already run a Pulsar cluster on Kubernetes, you can keep the
settings unchanged at most of time.
+As stated earlier, If you already run your Functions Worker embedded in a
Broker on Kubernetes, you can keep many of these settings as their default.
Review comment:
```suggestion
As stated earlier, if you already run your functions worker embedded in a
broker on Kubernetes, you can keep many of these settings as default.
```
##########
File path: site2/docs/functions-runtime.md
##########
@@ -138,20 +149,75 @@ roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: functions-worker
-subjects:
+subjectsKubernetesSec:
- kind: ServiceAccount
name: functions-worker
```
+If the service-account is not properly configured, you may see an error
message similar to this:
+```bash
+22:04:27.696 [Timer-0] ERROR
org.apache.pulsar.functions.runtime.KubernetesRuntimeFactory - Error while
trying to fetch configmap
example-pulsar-4qvmb5gur3c6fc9dih0x1xn8b-function-worker-config at namespace
pulsar
+io.kubernetes.client.ApiException: Forbidden
+ at io.kubernetes.client.ApiClient.handleResponse(ApiClient.java:882)
~[io.kubernetes-client-java-2.0.0.jar:?]
+ at io.kubernetes.client.ApiClient.execute(ApiClient.java:798)
~[io.kubernetes-client-java-2.0.0.jar:?]
+ at
io.kubernetes.client.apis.CoreV1Api.readNamespacedConfigMapWithHttpInfo(CoreV1Api.java:23673)
~[io.kubernetes-client-java-api-2.0.0.jar:?]
+ at
io.kubernetes.client.apis.CoreV1Api.readNamespacedConfigMap(CoreV1Api.java:23655)
~[io.kubernetes-client-java-api-2.0.0.jar:?]
+ at
org.apache.pulsar.functions.runtime.KubernetesRuntimeFactory.fetchConfigMap(KubernetesRuntimeFactory.java:284)
[org.apache.pulsar-pulsar-functions-runtime-2.4.0-42c3bf949.jar:2.4.0-42c3bf949]
+ at
org.apache.pulsar.functions.runtime.KubernetesRuntimeFactory$1.run(KubernetesRuntimeFactory.java:275)
[org.apache.pulsar-pulsar-functions-runtime-2.4.0-42c3bf949.jar:2.4.0-42c3bf949]
+ at java.util.TimerThread.mainLoop(Timer.java:555) [?:1.8.0_212]
+ at java.util.TimerThread.run(Timer.java:505) [?:1.8.0_212]
+```
+
+### Kubernetes Secrets integration
+
+In order to safely distribute secrets, Pulasr Functions can reference
Kubernetes secrets. To enable this, set the
`secretsProviderConfiguratorClassName` to
`org.apache.pulsar.functions.secretsproviderconfigurator.KubernetesSecretsProviderConfigurator`.
+
+Then, you can create a secret in the namespace where your functions will be
deployed. As an example, suppose we are deploying functions to the
`pulsar-func` Kubernetes namespace, and we have a secret named `database-creds`
with a field name `password`, which we want to mount in the pod as an
environment variable called `DATABASE_PASSWORD`.
+
+The following functions configuration would then allow us to reference that
secret and have the value be mounted as an environment variable in the pod.
+
+```Yaml
+tenant: "mytenant"
+namespace: "mynamespace"
+name: "myfunction"
+topicName: "persistent://mytenant/mynamespace/myfuncinput"
+className: "com.company.pulsar.myfunction"
+
+secrets:
+ # the secret will be mounted from the `password` field in the
`database-creds` secret as an env var called `DATABASE_PASSWORD`
+ DATABASE_PASSWORD:
+ path: "database-creds"
+ key: "password"
+
+```
+
+### Kubernetes Functions authentication
+
+
+When your Pulsar cluster uses authentication, the pod running your function
needs a mechanism to authenticate with the Broker.
+
+An interface,
`org.apache.pulsar.functions.auth.KubernetesFunctionAuthProvider`, can be
extended to provide support for any authentication mechanism. The
`functionAuthProviderClassName` in `function-worker.yml` is used to specify
your path to this implementation.
+
+Pulsar includes an implementation of this interface that is suitable for token
auth, which should be configured like the following in the configuration:
+```Yaml
+functionAuthProviderClassName:
org.apache.pulsar.functions.auth.KubernetesSecretsTokenAuthProvider
+```
+
+For custom auth or TLS, the user will need to implement this interface (or use
an alternative mechanism to provide auth)
Review comment:
```suggestion
For custom authentication or TLS, you need to implement this interface (or
use an alternative mechanism to provide authentication)
```
##########
File path: site2/docs/functions-runtime.md
##########
@@ -138,20 +149,75 @@ roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: functions-worker
-subjects:
+subjectsKubernetesSec:
- kind: ServiceAccount
name: functions-worker
```
+If the service-account is not properly configured, you may see an error
message similar to this:
+```bash
+22:04:27.696 [Timer-0] ERROR
org.apache.pulsar.functions.runtime.KubernetesRuntimeFactory - Error while
trying to fetch configmap
example-pulsar-4qvmb5gur3c6fc9dih0x1xn8b-function-worker-config at namespace
pulsar
+io.kubernetes.client.ApiException: Forbidden
+ at io.kubernetes.client.ApiClient.handleResponse(ApiClient.java:882)
~[io.kubernetes-client-java-2.0.0.jar:?]
+ at io.kubernetes.client.ApiClient.execute(ApiClient.java:798)
~[io.kubernetes-client-java-2.0.0.jar:?]
+ at
io.kubernetes.client.apis.CoreV1Api.readNamespacedConfigMapWithHttpInfo(CoreV1Api.java:23673)
~[io.kubernetes-client-java-api-2.0.0.jar:?]
+ at
io.kubernetes.client.apis.CoreV1Api.readNamespacedConfigMap(CoreV1Api.java:23655)
~[io.kubernetes-client-java-api-2.0.0.jar:?]
+ at
org.apache.pulsar.functions.runtime.KubernetesRuntimeFactory.fetchConfigMap(KubernetesRuntimeFactory.java:284)
[org.apache.pulsar-pulsar-functions-runtime-2.4.0-42c3bf949.jar:2.4.0-42c3bf949]
+ at
org.apache.pulsar.functions.runtime.KubernetesRuntimeFactory$1.run(KubernetesRuntimeFactory.java:275)
[org.apache.pulsar-pulsar-functions-runtime-2.4.0-42c3bf949.jar:2.4.0-42c3bf949]
+ at java.util.TimerThread.mainLoop(Timer.java:555) [?:1.8.0_212]
+ at java.util.TimerThread.run(Timer.java:505) [?:1.8.0_212]
+```
+
+### Kubernetes Secrets integration
+
+In order to safely distribute secrets, Pulasr Functions can reference
Kubernetes secrets. To enable this, set the
`secretsProviderConfiguratorClassName` to
`org.apache.pulsar.functions.secretsproviderconfigurator.KubernetesSecretsProviderConfigurator`.
+
+Then, you can create a secret in the namespace where your functions will be
deployed. As an example, suppose we are deploying functions to the
`pulsar-func` Kubernetes namespace, and we have a secret named `database-creds`
with a field name `password`, which we want to mount in the pod as an
environment variable called `DATABASE_PASSWORD`.
+
+The following functions configuration would then allow us to reference that
secret and have the value be mounted as an environment variable in the pod.
+
+```Yaml
+tenant: "mytenant"
+namespace: "mynamespace"
+name: "myfunction"
+topicName: "persistent://mytenant/mynamespace/myfuncinput"
+className: "com.company.pulsar.myfunction"
+
+secrets:
+ # the secret will be mounted from the `password` field in the
`database-creds` secret as an env var called `DATABASE_PASSWORD`
+ DATABASE_PASSWORD:
+ path: "database-creds"
+ key: "password"
+
+```
+
+### Kubernetes Functions authentication
+
+
+When your Pulsar cluster uses authentication, the pod running your function
needs a mechanism to authenticate with the Broker.
+
+An interface,
`org.apache.pulsar.functions.auth.KubernetesFunctionAuthProvider`, can be
extended to provide support for any authentication mechanism. The
`functionAuthProviderClassName` in `function-worker.yml` is used to specify
your path to this implementation.
+
+Pulsar includes an implementation of this interface that is suitable for token
auth, which should be configured like the following in the configuration:
+```Yaml
+functionAuthProviderClassName:
org.apache.pulsar.functions.auth.KubernetesSecretsTokenAuthProvider
+```
+
+For custom auth or TLS, the user will need to implement this interface (or use
an alternative mechanism to provide auth)
+
+For token authentication, the way this works is that the functions worker
captures the token that was used to deploy (or update) the function. This token
is then saved as a secret and mounted into the pod.
+
+One thing to keep in mind is that if you use tokens that expire when deploying
functions, these tokens will expire.
+
+
### Kubernetes CustomRuntimeOptions
-The functions (and sinks/sources) API provides a flag, `customRuntimeOptions`
which can be used to pass options to the runtime to customize how the runtime
operates.
+The Kubernetes integration also includes the ability for the user to implement
a class which will customize how manifests will be generated. This is
configured by setting the `runtimeCustomizerClassName` in the
`functions-worker.yml` to the fully qualified class name. The interface the
user must implement is the
`org.apache.pulsar.functions.runtime.kubernetes.KubernetesManifestCustomizer`
interface.
+
+The functions (and sinks/sources) API provides a flag, `customRuntimeOptions`
which will be passed to this interface.
Review comment:
```suggestion
The functions (and sinks/sources) API provides a flag,
`customRuntimeOptions`, which is passed to this interface.
```
##########
File path: site2/docs/functions-runtime.md
##########
@@ -183,3 +249,68 @@ for the `runtimeCustomerClassName` property. This
implementation takes the follo
}
}
```
+
+## Other configuration considerations
+
+
+### In a cluster with geo-replication
+
+If you are running multiple clusters tied together with geo-replication, it is
important to use a different function namespace for each cluster! Otherwise,
the functions
+will share a namespace and potentially schedule across clusters.
+
+As an example, suppose we have clusters east-1 and west-1, we would configure
the Functions Worker in east-1 like:
+```Yaml
+pulsarFunctionsCluster: east-1
+pulsarFunctionsNamespace: public/functions-east-1
+```
+
+and the cluster in west-1 like:
+```Yaml
+pulsarFunctionsCluster: west-1
+pulsarFunctionsNamespace: public/functions-west-1
+```
+
+This ensures the two different Function Workers will use distinct sets of
topics for their internal co-ordination.
+
+### Configuring Standalone Functions Worker
+
+When configuring a standalone Functions Worker, you need to specify a few
properties in order for the Functions Worker to be able
+to communicate with the Broker. This requires many of the same properties to
be set that the Broker requires, especially when using TLS.
+
+The following properties are the baseline of what is required:
+
+```Yaml
+workerPort: 8080
+workerPortTls: 8443 # when using TLS
+tlsCertificateFilePath: /etc/pulsar/tls/tls.crt # when using TLS
+tlsKeyFilePath: /etc/pulsar/tls/tls.key # when using TLS
+tlsTrustCertsFilePath: /etc/pulsar/tls/ca.crt # when using TLS
+pulsarServiceUrl: pulsar://broker.pulsar:6650/ # or
pulsar+ssl://pulsar-prod-broker.pulsar:6651/ when using TLS
+pulsarWebServiceUrl: http://broker.pulsar:8080/ # or
https://pulsar-prod-broker.pulsar:8443/ when using TLS
+useTls: true # when using TLS, critical!
+
+```
+
+#### With Authentication
+
+When running a Functions Worker in a standalone process (i.e. not embedded in
the Broker) in a cluster with authentication, you must configure your Functions
Worker to both be able to interact with the Broker *and* also to authenticate
incoming requests as well. This requires many of the same properties to be set
that the Broker requires for authentication/authorization.
+
+As an example, assuming we want to use token authentication, here is an
example of properties we need to set in `function-worker.yml`
+
+```Yaml
+clientAuthenticationPlugin:
org.apache.pulsar.client.impl.auth.AuthenticationToken
+clientAuthenticationParameters: file:///etc/pulsar/token/admin-token.txt
+configurationStoreServers: zookeeper-cluster:2181 # auth requires a connection
to zookeeper
+authenticationProviders:
+ - "org.apache.pulsar.broker.authentication.AuthenticationProviderToken"
+authorizationEnabled: true
+authenticationEnabled: true
+superUserRoles:
+ - superuser
+ - proxy
+properties:
+ tokenSecretKey: file:///etc/pulsar/jwt/secret # if using a secret token
+ tokenPublicKey: file:///etc/pulsar/jwt/public.key # if using public/private
key tokens
+```
+
+Notice that we must configure both the Function Worker authz/authn for the
server to proper auth requests and configure the client to be authenticated to
communicate with the broker.
Review comment:
```suggestion
> #### Note
>
> You must configure both the Function Worker authorization or
authentication for the server to proper authentication requests and configure
the client to be authenticated to communicate with the broker.
```
##########
File path: site2/docs/functions-runtime.md
##########
@@ -46,6 +46,17 @@ processContainerFactory:
## Configure Kubernetes runtime
+
+### How it works
+
+The Kubernetes runtime works by having the functions-worker generate and apply
kubernetes manifests. In the event that the functions worker is running on
Kubernetes already, it can use the `serviceAccount` that is associated with the
pod the functions worker is running in. Otherwise, it can be configured to
communicate with a Kubernetes cluster.
+
+The manifests which the functions worker generates include a `StatefulSet`, a
`Service (which is used to communicate with the pods), and a `Secret` for auth
credentials (when applicable). The StatefulSet manifest (by default) has a
single pod, with the number of replicas determined by the "parallelism" of the
function. On pod boot, the pod will download the function payload (via the
functions worker REST API). The pod's container image is configurable, but must
have the functions runtime.
+
+The Kubernetes runtime also has supports for secrets, with the end user being
able to create a kubernetes secret and have it be exposed as an environment
variable in the pod (described below). Additionally, the Kubernetes runtime
fairly extensible, with the user being able to implement classes that customize
the way kubernetes manifests get generated, how auth data is passed to pods,
and how secrets can be integrated.
Review comment:
```suggestion
The Kubernetes runtime also supports secrets, with the end user being able
to create a Kubernetes secret and have it be exposed as an environment variable
in the pod (described below). Additionally, the Kubernetes runtime fairly
extensible, with the user being able to implement classes that customize the
way Kubernetes manifests get generated, how auth data is passed to pods, and
how secrets can be integrated.
```
##########
File path: site2/docs/functions-runtime.md
##########
@@ -78,9 +89,23 @@ kubernetesContainerFactory:
percentMemoryPadding: 10
```
-If you have already run a Pulsar cluster on Kubernetes, you can keep the
settings unchanged at most of time.
+As stated earlier, If you already run your Functions Worker embedded in a
Broker on Kubernetes, you can keep many of these settings as their default.
+
+### Standalone Functions Worker on K8S
+
+If you run your Functions Worker standalone (i.e. not embedded) on Kubernetes,
you will need to configure `pulsarSerivceUrl` to be the URL of the
+broker and `pulsarAdminUrl` as the URL to the Functions Worker.
+
+As an example, suppose both our Pulsar Brokers and Function Workers are run in
the `pulsar` K8S namespace. Additionally, assuming the Brokers have a service
called `brokers` and the Functions Worker has a service called `func-worker`,
then the settings would be:
Review comment:
```suggestion
As an example, suppose both our Pulsar brokers and Function Workers run in
the `pulsar` K8S namespace. Additionally, assuming the brokers have a service
called `brokers` and the Functions Worker has a service called `func-worker`,
then the settings would be:
```
##########
File path: site2/docs/functions-runtime.md
##########
@@ -46,6 +46,17 @@ processContainerFactory:
## Configure Kubernetes runtime
+
+### How it works
+
+The Kubernetes runtime works by having the functions-worker generate and apply
kubernetes manifests. In the event that the functions worker is running on
Kubernetes already, it can use the `serviceAccount` that is associated with the
pod the functions worker is running in. Otherwise, it can be configured to
communicate with a Kubernetes cluster.
+
+The manifests which the functions worker generates include a `StatefulSet`, a
`Service (which is used to communicate with the pods), and a `Secret` for auth
credentials (when applicable). The StatefulSet manifest (by default) has a
single pod, with the number of replicas determined by the "parallelism" of the
function. On pod boot, the pod will download the function payload (via the
functions worker REST API). The pod's container image is configurable, but must
have the functions runtime.
Review comment:
```suggestion
The manifests which the functions worker generates include a `StatefulSet`,
a `Service` (which is used to communicate with the pods), and a `Secret` for
auth credentials (when applicable). The `StatefulSet` manifest (by default) has
a single pod, with the number of replicas determined by the "parallelism" of
the function. On pod boot, the pod downloads the function payload (via the
functions worker REST API). The pod's container image is configurable, but must
have the functions runtime.
```
Use present tense if you are covering facts that were, are, and forever
shall be true.
Always use the present tense in technical writing by default. Only use past
or future tenses when you’re actually writing about something that happened in
the past or will happen in the future.
##########
File path: site2/docs/functions-runtime.md
##########
@@ -78,9 +89,23 @@ kubernetesContainerFactory:
percentMemoryPadding: 10
```
-If you have already run a Pulsar cluster on Kubernetes, you can keep the
settings unchanged at most of time.
+As stated earlier, If you already run your Functions Worker embedded in a
Broker on Kubernetes, you can keep many of these settings as their default.
+
+### Standalone Functions Worker on K8S
+
+If you run your Functions Worker standalone (i.e. not embedded) on Kubernetes,
you will need to configure `pulsarSerivceUrl` to be the URL of the
Review comment:
```suggestion
If you run your Functions Worker standalone (that is, not embedded) on
Kubernetes, you need to configure `pulsarSerivceUrl` to be the URL of the
```
IBM Writing Style Guide: Do not use Latin abbreviations, such as e.g., etc.,
and i.e.
##########
File path: site2/docs/functions-runtime.md
##########
@@ -138,20 +149,75 @@ roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: functions-worker
-subjects:
+subjectsKubernetesSec:
- kind: ServiceAccount
name: functions-worker
```
+If the service-account is not properly configured, you may see an error
message similar to this:
+```bash
+22:04:27.696 [Timer-0] ERROR
org.apache.pulsar.functions.runtime.KubernetesRuntimeFactory - Error while
trying to fetch configmap
example-pulsar-4qvmb5gur3c6fc9dih0x1xn8b-function-worker-config at namespace
pulsar
+io.kubernetes.client.ApiException: Forbidden
+ at io.kubernetes.client.ApiClient.handleResponse(ApiClient.java:882)
~[io.kubernetes-client-java-2.0.0.jar:?]
+ at io.kubernetes.client.ApiClient.execute(ApiClient.java:798)
~[io.kubernetes-client-java-2.0.0.jar:?]
+ at
io.kubernetes.client.apis.CoreV1Api.readNamespacedConfigMapWithHttpInfo(CoreV1Api.java:23673)
~[io.kubernetes-client-java-api-2.0.0.jar:?]
+ at
io.kubernetes.client.apis.CoreV1Api.readNamespacedConfigMap(CoreV1Api.java:23655)
~[io.kubernetes-client-java-api-2.0.0.jar:?]
+ at
org.apache.pulsar.functions.runtime.KubernetesRuntimeFactory.fetchConfigMap(KubernetesRuntimeFactory.java:284)
[org.apache.pulsar-pulsar-functions-runtime-2.4.0-42c3bf949.jar:2.4.0-42c3bf949]
+ at
org.apache.pulsar.functions.runtime.KubernetesRuntimeFactory$1.run(KubernetesRuntimeFactory.java:275)
[org.apache.pulsar-pulsar-functions-runtime-2.4.0-42c3bf949.jar:2.4.0-42c3bf949]
+ at java.util.TimerThread.mainLoop(Timer.java:555) [?:1.8.0_212]
+ at java.util.TimerThread.run(Timer.java:505) [?:1.8.0_212]
+```
+
+### Kubernetes Secrets integration
+
+In order to safely distribute secrets, Pulasr Functions can reference
Kubernetes secrets. To enable this, set the
`secretsProviderConfiguratorClassName` to
`org.apache.pulsar.functions.secretsproviderconfigurator.KubernetesSecretsProviderConfigurator`.
+
+Then, you can create a secret in the namespace where your functions will be
deployed. As an example, suppose we are deploying functions to the
`pulsar-func` Kubernetes namespace, and we have a secret named `database-creds`
with a field name `password`, which we want to mount in the pod as an
environment variable called `DATABASE_PASSWORD`.
+
+The following functions configuration would then allow us to reference that
secret and have the value be mounted as an environment variable in the pod.
+
+```Yaml
+tenant: "mytenant"
+namespace: "mynamespace"
+name: "myfunction"
+topicName: "persistent://mytenant/mynamespace/myfuncinput"
+className: "com.company.pulsar.myfunction"
+
+secrets:
+ # the secret will be mounted from the `password` field in the
`database-creds` secret as an env var called `DATABASE_PASSWORD`
+ DATABASE_PASSWORD:
+ path: "database-creds"
+ key: "password"
+
+```
+
+### Kubernetes Functions authentication
+
+
+When your Pulsar cluster uses authentication, the pod running your function
needs a mechanism to authenticate with the Broker.
+
+An interface,
`org.apache.pulsar.functions.auth.KubernetesFunctionAuthProvider`, can be
extended to provide support for any authentication mechanism. The
`functionAuthProviderClassName` in `function-worker.yml` is used to specify
your path to this implementation.
+
+Pulsar includes an implementation of this interface that is suitable for token
auth, which should be configured like the following in the configuration:
+```Yaml
+functionAuthProviderClassName:
org.apache.pulsar.functions.auth.KubernetesSecretsTokenAuthProvider
+```
+
+For custom auth or TLS, the user will need to implement this interface (or use
an alternative mechanism to provide auth)
+
+For token authentication, the way this works is that the functions worker
captures the token that was used to deploy (or update) the function. This token
is then saved as a secret and mounted into the pod.
+
+One thing to keep in mind is that if you use tokens that expire when deploying
functions, these tokens will expire.
+
+
### Kubernetes CustomRuntimeOptions
-The functions (and sinks/sources) API provides a flag, `customRuntimeOptions`
which can be used to pass options to the runtime to customize how the runtime
operates.
+The Kubernetes integration also includes the ability for the user to implement
a class which will customize how manifests will be generated. This is
configured by setting the `runtimeCustomizerClassName` in the
`functions-worker.yml` to the fully qualified class name. The interface the
user must implement is the
`org.apache.pulsar.functions.runtime.kubernetes.KubernetesManifestCustomizer`
interface.
Review comment:
```suggestion
The Kubernetes integration also includes the ability for the user to
implement a class which customizes how manifests is generated. This is
configured by setting the `runtimeCustomizerClassName` in the
`functions-worker.yml` to the fully qualified class name. The interface the you
must implement is the
`org.apache.pulsar.functions.runtime.kubernetes.KubernetesManifestCustomizer`
interface.
```
IBM Writing Style Guide: Use the second-person pronoun `you` as much as
possible. The subject of an imperative sentence is understood to be you.
##########
File path: site2/docs/functions-runtime.md
##########
@@ -138,20 +149,75 @@ roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: functions-worker
-subjects:
+subjectsKubernetesSec:
- kind: ServiceAccount
name: functions-worker
```
+If the service-account is not properly configured, you may see an error
message similar to this:
+```bash
+22:04:27.696 [Timer-0] ERROR
org.apache.pulsar.functions.runtime.KubernetesRuntimeFactory - Error while
trying to fetch configmap
example-pulsar-4qvmb5gur3c6fc9dih0x1xn8b-function-worker-config at namespace
pulsar
+io.kubernetes.client.ApiException: Forbidden
+ at io.kubernetes.client.ApiClient.handleResponse(ApiClient.java:882)
~[io.kubernetes-client-java-2.0.0.jar:?]
+ at io.kubernetes.client.ApiClient.execute(ApiClient.java:798)
~[io.kubernetes-client-java-2.0.0.jar:?]
+ at
io.kubernetes.client.apis.CoreV1Api.readNamespacedConfigMapWithHttpInfo(CoreV1Api.java:23673)
~[io.kubernetes-client-java-api-2.0.0.jar:?]
+ at
io.kubernetes.client.apis.CoreV1Api.readNamespacedConfigMap(CoreV1Api.java:23655)
~[io.kubernetes-client-java-api-2.0.0.jar:?]
+ at
org.apache.pulsar.functions.runtime.KubernetesRuntimeFactory.fetchConfigMap(KubernetesRuntimeFactory.java:284)
[org.apache.pulsar-pulsar-functions-runtime-2.4.0-42c3bf949.jar:2.4.0-42c3bf949]
+ at
org.apache.pulsar.functions.runtime.KubernetesRuntimeFactory$1.run(KubernetesRuntimeFactory.java:275)
[org.apache.pulsar-pulsar-functions-runtime-2.4.0-42c3bf949.jar:2.4.0-42c3bf949]
+ at java.util.TimerThread.mainLoop(Timer.java:555) [?:1.8.0_212]
+ at java.util.TimerThread.run(Timer.java:505) [?:1.8.0_212]
+```
+
+### Kubernetes Secrets integration
+
+In order to safely distribute secrets, Pulasr Functions can reference
Kubernetes secrets. To enable this, set the
`secretsProviderConfiguratorClassName` to
`org.apache.pulsar.functions.secretsproviderconfigurator.KubernetesSecretsProviderConfigurator`.
+
+Then, you can create a secret in the namespace where your functions will be
deployed. As an example, suppose we are deploying functions to the
`pulsar-func` Kubernetes namespace, and we have a secret named `database-creds`
with a field name `password`, which we want to mount in the pod as an
environment variable called `DATABASE_PASSWORD`.
Review comment:
```suggestion
Then, you can create a secret in the namespace where your functions are
deployed. As an example, suppose we are deploying functions to the
`pulsar-func` Kubernetes namespace, and we have a secret named `database-creds`
with a field name `password`, which we want to mount in the pod as an
environment variable called `DATABASE_PASSWORD`.
```
##########
File path: site2/docs/functions-runtime.md
##########
@@ -183,3 +249,68 @@ for the `runtimeCustomerClassName` property. This
implementation takes the follo
}
}
```
+
+## Other configuration considerations
+
+
+### In a cluster with geo-replication
+
+If you are running multiple clusters tied together with geo-replication, it is
important to use a different function namespace for each cluster! Otherwise,
the functions
+will share a namespace and potentially schedule across clusters.
+
+As an example, suppose we have clusters east-1 and west-1, we would configure
the Functions Worker in east-1 like:
+```Yaml
+pulsarFunctionsCluster: east-1
+pulsarFunctionsNamespace: public/functions-east-1
+```
+
+and the cluster in west-1 like:
+```Yaml
+pulsarFunctionsCluster: west-1
+pulsarFunctionsNamespace: public/functions-west-1
+```
+
+This ensures the two different Function Workers will use distinct sets of
topics for their internal co-ordination.
+
+### Configuring Standalone Functions Worker
+
+When configuring a standalone Functions Worker, you need to specify a few
properties in order for the Functions Worker to be able
+to communicate with the Broker. This requires many of the same properties to
be set that the Broker requires, especially when using TLS.
+
+The following properties are the baseline of what is required:
+
+```Yaml
+workerPort: 8080
+workerPortTls: 8443 # when using TLS
+tlsCertificateFilePath: /etc/pulsar/tls/tls.crt # when using TLS
+tlsKeyFilePath: /etc/pulsar/tls/tls.key # when using TLS
+tlsTrustCertsFilePath: /etc/pulsar/tls/ca.crt # when using TLS
+pulsarServiceUrl: pulsar://broker.pulsar:6650/ # or
pulsar+ssl://pulsar-prod-broker.pulsar:6651/ when using TLS
+pulsarWebServiceUrl: http://broker.pulsar:8080/ # or
https://pulsar-prod-broker.pulsar:8443/ when using TLS
+useTls: true # when using TLS, critical!
+
+```
+
+#### With Authentication
Review comment:
```suggestion
#### With authentication
```
##########
File path: site2/docs/functions-runtime.md
##########
@@ -183,3 +249,68 @@ for the `runtimeCustomerClassName` property. This
implementation takes the follo
}
}
```
+
+## Other configuration considerations
+
+
+### In a cluster with geo-replication
+
+If you are running multiple clusters tied together with geo-replication, it is
important to use a different function namespace for each cluster! Otherwise,
the functions
Review comment:
```suggestion
If you are running multiple clusters tied together with geo-replication, it
is important to use a different function namespace for each cluster. Otherwise,
the function
```
IBM Writing Style Guide:
Do not punctuate sentences with exclamation points because their tone can be
interpreted negatively,
for example, as aggressive, condescending, or overly informal.
Convey urgency or emphasis with the appropriate words, not with exclamation
points. To call
attention to important hints, tips, guidance, restrictions, or advice that
might be overlooked, consider
using a note that has a meaningful label.
##########
File path: site2/docs/functions-runtime.md
##########
@@ -183,3 +249,68 @@ for the `runtimeCustomerClassName` property. This
implementation takes the follo
}
}
```
+
+## Other configuration considerations
+
+
+### In a cluster with geo-replication
+
+If you are running multiple clusters tied together with geo-replication, it is
important to use a different function namespace for each cluster! Otherwise,
the functions
+will share a namespace and potentially schedule across clusters.
+
+As an example, suppose we have clusters east-1 and west-1, we would configure
the Functions Worker in east-1 like:
+```Yaml
+pulsarFunctionsCluster: east-1
+pulsarFunctionsNamespace: public/functions-east-1
+```
+
+and the cluster in west-1 like:
+```Yaml
+pulsarFunctionsCluster: west-1
+pulsarFunctionsNamespace: public/functions-west-1
+```
+
+This ensures the two different Function Workers will use distinct sets of
topics for their internal co-ordination.
+
+### Configuring Standalone Functions Worker
Review comment:
```suggestion
### Configure standalone Functions Worker
```
##########
File path: site2/docs/functions-runtime.md
##########
@@ -183,3 +249,68 @@ for the `runtimeCustomerClassName` property. This
implementation takes the follo
}
}
```
+
+## Other configuration considerations
+
+
+### In a cluster with geo-replication
+
+If you are running multiple clusters tied together with geo-replication, it is
important to use a different function namespace for each cluster! Otherwise,
the functions
+will share a namespace and potentially schedule across clusters.
Review comment:
```suggestion
shares a namespace and potentially schedule across clusters.
```
##########
File path: site2/docs/functions-runtime.md
##########
@@ -183,3 +249,68 @@ for the `runtimeCustomerClassName` property. This
implementation takes the follo
}
}
```
+
+## Other configuration considerations
+
+
+### In a cluster with geo-replication
+
+If you are running multiple clusters tied together with geo-replication, it is
important to use a different function namespace for each cluster! Otherwise,
the functions
+will share a namespace and potentially schedule across clusters.
+
+As an example, suppose we have clusters east-1 and west-1, we would configure
the Functions Worker in east-1 like:
+```Yaml
+pulsarFunctionsCluster: east-1
+pulsarFunctionsNamespace: public/functions-east-1
+```
+
+and the cluster in west-1 like:
+```Yaml
+pulsarFunctionsCluster: west-1
+pulsarFunctionsNamespace: public/functions-west-1
+```
+
+This ensures the two different Function Workers will use distinct sets of
topics for their internal co-ordination.
+
+### Configuring Standalone Functions Worker
+
+When configuring a standalone Functions Worker, you need to specify a few
properties in order for the Functions Worker to be able
+to communicate with the Broker. This requires many of the same properties to
be set that the Broker requires, especially when using TLS.
+
+The following properties are the baseline of what is required:
+
+```Yaml
+workerPort: 8080
+workerPortTls: 8443 # when using TLS
+tlsCertificateFilePath: /etc/pulsar/tls/tls.crt # when using TLS
+tlsKeyFilePath: /etc/pulsar/tls/tls.key # when using TLS
+tlsTrustCertsFilePath: /etc/pulsar/tls/ca.crt # when using TLS
+pulsarServiceUrl: pulsar://broker.pulsar:6650/ # or
pulsar+ssl://pulsar-prod-broker.pulsar:6651/ when using TLS
+pulsarWebServiceUrl: http://broker.pulsar:8080/ # or
https://pulsar-prod-broker.pulsar:8443/ when using TLS
+useTls: true # when using TLS, critical!
+
+```
+
+#### With Authentication
+
+When running a Functions Worker in a standalone process (i.e. not embedded in
the Broker) in a cluster with authentication, you must configure your Functions
Worker to both be able to interact with the Broker *and* also to authenticate
incoming requests as well. This requires many of the same properties to be set
that the Broker requires for authentication/authorization.
+
+As an example, assuming we want to use token authentication, here is an
example of properties we need to set in `function-worker.yml`
Review comment:
```suggestion
As an example, assuming you want to use token authentication. Here is an
example of properties you need to set in `function-worker.yml`
```
IBM Writing Style Guide:
• Avoid the first-person pronouns `I` and `we`, except in these situations:
° In the question portion of frequently asked questions (FAQs)
° In articles, white papers, or documents that have listed authors and in
which the authors describe their own actions or opinions
##########
File path: site2/docs/functions-runtime.md
##########
@@ -183,3 +249,68 @@ for the `runtimeCustomerClassName` property. This
implementation takes the follo
}
}
```
+
+## Other configuration considerations
+
+
+### In a cluster with geo-replication
+
+If you are running multiple clusters tied together with geo-replication, it is
important to use a different function namespace for each cluster! Otherwise,
the functions
+will share a namespace and potentially schedule across clusters.
+
+As an example, suppose we have clusters east-1 and west-1, we would configure
the Functions Worker in east-1 like:
+```Yaml
+pulsarFunctionsCluster: east-1
+pulsarFunctionsNamespace: public/functions-east-1
+```
+
+and the cluster in west-1 like:
+```Yaml
+pulsarFunctionsCluster: west-1
+pulsarFunctionsNamespace: public/functions-west-1
+```
+
+This ensures the two different Function Workers will use distinct sets of
topics for their internal co-ordination.
Review comment:
```suggestion
This ensures the two different Function Workers use distinct sets of topics
for their internal coordination.
```
##########
File path: site2/docs/functions-runtime.md
##########
@@ -183,3 +249,68 @@ for the `runtimeCustomerClassName` property. This
implementation takes the follo
}
}
```
+
+## Other configuration considerations
+
+
+### In a cluster with geo-replication
+
+If you are running multiple clusters tied together with geo-replication, it is
important to use a different function namespace for each cluster! Otherwise,
the functions
+will share a namespace and potentially schedule across clusters.
+
+As an example, suppose we have clusters east-1 and west-1, we would configure
the Functions Worker in east-1 like:
+```Yaml
+pulsarFunctionsCluster: east-1
+pulsarFunctionsNamespace: public/functions-east-1
+```
+
+and the cluster in west-1 like:
+```Yaml
+pulsarFunctionsCluster: west-1
+pulsarFunctionsNamespace: public/functions-west-1
+```
+
+This ensures the two different Function Workers will use distinct sets of
topics for their internal co-ordination.
+
+### Configuring Standalone Functions Worker
+
+When configuring a standalone Functions Worker, you need to specify a few
properties in order for the Functions Worker to be able
+to communicate with the Broker. This requires many of the same properties to
be set that the Broker requires, especially when using TLS.
Review comment:
```suggestion
to communicate with the broker. This requires many of the same properties to
be set that the broker requires, especially when using TLS.
```
##########
File path: site2/docs/functions-runtime.md
##########
@@ -183,3 +249,68 @@ for the `runtimeCustomerClassName` property. This
implementation takes the follo
}
}
```
+
+## Other configuration considerations
+
+
+### In a cluster with geo-replication
+
+If you are running multiple clusters tied together with geo-replication, it is
important to use a different function namespace for each cluster! Otherwise,
the functions
+will share a namespace and potentially schedule across clusters.
+
+As an example, suppose we have clusters east-1 and west-1, we would configure
the Functions Worker in east-1 like:
+```Yaml
+pulsarFunctionsCluster: east-1
+pulsarFunctionsNamespace: public/functions-east-1
+```
+
+and the cluster in west-1 like:
+```Yaml
+pulsarFunctionsCluster: west-1
+pulsarFunctionsNamespace: public/functions-west-1
+```
+
+This ensures the two different Function Workers will use distinct sets of
topics for their internal co-ordination.
+
+### Configuring Standalone Functions Worker
+
+When configuring a standalone Functions Worker, you need to specify a few
properties in order for the Functions Worker to be able
+to communicate with the Broker. This requires many of the same properties to
be set that the Broker requires, especially when using TLS.
+
+The following properties are the baseline of what is required:
+
+```Yaml
+workerPort: 8080
+workerPortTls: 8443 # when using TLS
+tlsCertificateFilePath: /etc/pulsar/tls/tls.crt # when using TLS
+tlsKeyFilePath: /etc/pulsar/tls/tls.key # when using TLS
+tlsTrustCertsFilePath: /etc/pulsar/tls/ca.crt # when using TLS
+pulsarServiceUrl: pulsar://broker.pulsar:6650/ # or
pulsar+ssl://pulsar-prod-broker.pulsar:6651/ when using TLS
+pulsarWebServiceUrl: http://broker.pulsar:8080/ # or
https://pulsar-prod-broker.pulsar:8443/ when using TLS
+useTls: true # when using TLS, critical!
+
+```
+
+#### With Authentication
+
+When running a Functions Worker in a standalone process (i.e. not embedded in
the Broker) in a cluster with authentication, you must configure your Functions
Worker to both be able to interact with the Broker *and* also to authenticate
incoming requests as well. This requires many of the same properties to be set
that the Broker requires for authentication/authorization.
Review comment:
```suggestion
When running a Functions Worker in a standalone process (that is, not
embedded in the broker) in a cluster with authentication, you must configure
your Functions Worker to both be able to interact with the broker *and* also to
authenticate incoming requests as well. This requires many of the same
properties to be set that the broker requires for authentication or
authorization.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
