jdbeck opened a new pull request #8452:
URL: https://github.com/apache/pulsar/pull/8452
Fixes #8012
### Motivation
Currently (as of 2.6.x), the Pulsar KubernetesRuntime class hardcodes the
jobName (the name assigned to the StatefulSet used to create the function pods)
to the format "pf-[tenant]-[namespace]-[function][-optional 8 char hash]."
While the intent of this name format was no doubt both to provide a human
readable name for the k8s objects and ensure uniqueness within k8s, we've found
it -- when combined with the 55 character size restriction imposed by
KubernetesRuntime -- to be unnecessarily limiting. In our environment, we
ensure that Pulsar functions under a particular Pulsar tenant deploy into a
kubernetes namespace dedicated to that tenant; hence, for us the [tenant]
portion of the function name is redundant. Further, the "pf-" prefix is
unnecessary, as we're able to distinguish the function pods from other pods
based on the function name alone. These issues may seem minor, but they consume
precious characters against the 55 character max!
### Modifications
Updated Kubernetes runtime and factory classes under pulsar/pulsar-functions
to add "jobName" to customRuntimeOptions just like jobNamespace for setting the
k8s namespace.
### Verifying this change
- [X ] Make sure that the change passes the CI checks.
This change added tests and can be verified as follows:
- Updated/extended
src/test/java/org/apache/pulsar/functions/runtime/kubernetes/KubernetesRuntimeTest.java
to contain news tests for overridden job Name (jobName).
- Updated other tests to account for new parameters for checks but does not
change any functionality in the tests.
### Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API: (no)
- The schema: (no)
- The default values of configurations: (no)
- The wire protocol: (no)
- The rest endpoints: (yes)
It does not change the endpoint for creating functions but it does change
the available string (json) that can be submitted for 'customRuntimeOptions:
string'. Like 'jobNamespace: <name>' you may add 'jobName: <name>' to the json
string for changing the k8s pod name for the function.
Example:
curl -v --form "functionConfig={\"tenant\": \"public\", \"namespace\":
\"default\", \"name\": \"exclamation\", \"className\":
\"org.apache.pulsar.functions.api.examples.ExclamationFunction\", \"inputs\":
[\"persistent://public/default/in-content\"], \"output\":
\"persistent://public/default/out-content\", \"customRuntimeOptions\":
\"{\\\"jobNamespace\\\": \\\"pulsar\\\", **\\\"jobName\\\":
\\\"test123\\\**"}\", \"jar\": \"api-examples.jar\"};type=application/json"
--form data=@/pulsar/examples/api-examples.jar
http://pulsar-broker.pulsar.svc.cluster.local:8080/admin/v3/functions/public/default/exclamation
- The admin cli options: (yes)
Same as above. Adds an available option for creating functions
'customRuntimeOptions'.
Example:
./bin/pulsar-admin functions create --custom-runtime-options
'{"jobNamespace": "pulsar", **"jobName": "test123"**}' --auto-ack false
--classname org.apache.pulsar.functions.api.examples.ExclamationFunction -i
persistent://public/default/in-content -o
persistent://public/default/out-content --jar /pulsar/examples/api-examples.jar
--name exclamation
- Anything that affects deployment: (no)
### Documentation
- Does this pull request introduce a new feature? (yes)
- If yes, how is the feature documented? (not documented)
Probably should be added to functions-runtime.md for 2.7.0 ... hopefully
(like 'jobNamespace' under 'customRuntimeOptions')
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
