asdf2014 commented on code in PR #16600:
URL: https://github.com/apache/druid/pull/16600#discussion_r1639120147
##########
docs/development/extensions-contrib/k8s-jobs.md:
##########
@@ -217,59 +487,98 @@ data:
druid.peon.mode=remote
druid.indexer.task.encapsulatedTask=true
```
-#### Dynamic Pod Template Selection Config
-The Dynamic Pod Template Selection feature enhances the K8s extension by
enabling more flexible and dynamic selection of pod templates based on task
properties. This process is governed by the `PodTemplateSelectStrategy`. Below
are the two strategies implemented:
+</details>
-|Property|Description|Default|
-|--------|-----------|-------|
-|`TaskTypePodTemplateSelectStrategy`| This strategy selects pod templates
based on task type for execution purposes, implementing the behavior that maps
templates to specific task types. | true |
-|`SelectorBasedPodTemplateSelectStrategy`| This strategy evaluates a series of
selectors, known as `selectors`, which are aligned with potential task
properties. | false |
+#### Pod Template Selection
+
+The Pod Template Adapapter can select which pod template should be used for a
Task using the [Task Runner Execution Config](#dynamic-config)
-`SelectorBasedPodTemplateSelectStrategy`, the strategy implementing this new
feature, is based on conditional `selectors` that match against top-level keys
from the task payload. Currently, it supports matching based on task context
tags, task type, and dataSource. These selectors are ordered in the dynamic
configuration, with the first selector given the highest priority during the
evaluation process. This means that the selection process uses these ordered
conditions to determine a task’s Pod template. The first matching condition
immediately determines the Pod template, thereby prioritizing certain
configurations over others. If no selector matches, it will fall back to an
optional `defaultKey` if configured; if there is still no match, it will use
the `base` template.
+##### TaskTypePodTemplateSelectStrategy
+This strategy selects pod templates based on task type for execution purposes,
implementing the behavior that maps templates to specific task types.
+This is the default pod template selection strategy.
-Example Configuration:
+Task specific pod templates can be specified as the runtime property
`druid.indexer.runner.k8s.podTemplate.{taskType}:
/path/to/taskSpecificPodSpec.yaml` where {taskType} is the name of the task
type i.e `index_parallel`.
-We define two template keys in the configuration—`low-throughput` and
`medium-throughput`—each associated with specific task conditions and arranged
in a priority order.
+If you are trying to use the default image's environment variable parsing
feature to set runtime properties, you need to add a extra escape underscore
when specifying pod templates.
+e.g. set the environment variable
`druid_indexer_runner_k8s_podTemplate_index__kafka` when setting
`druid.indxer.runner.k8s.podTemplate.index_kafka`
-- Low Throughput Template: This is the first template evaluated and has the
highest priority. Tasks that have a context tag
`billingCategory=streaming_ingestion` and a datasource of `wikipedia` will be
classified under the `low-throughput` template. This classification directs
such tasks to utilize a predefined pod template optimized for low throughput
requirements.
-- Medium Throughput Template: If a task does not meet the low-throughput
criteria, the system will then evaluate it against the next selector in order.
In this example, if the task type is index_kafka, it will fall into the
`medium-throughput` template.
+An example config would look like
+```
Review Comment:
```suggestion
```properties
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
