georgew5656 commented on code in PR #14674:
URL: https://github.com/apache/druid/pull/14674#discussion_r1279366020
##########
docs/development/extensions-contrib/k8s-jobs.md:
##########
@@ -115,33 +75,197 @@ You can keep your Dockerfile the same but you must have a
sidecar spec like so:
- arg2
```
-The following roles must also be accessible. An example spec could be:
+For both of these adapters, you can add optional labels to your K8s jobs /
pods if you need them by using the following configuration:
+`druid.indexer.runner.labels: '{"key":"value"}'`
+Annotations are the same with:
+`druid.indexer.runner.annotations: '{"key":"value"}'`
+
+All other configurations you had for the middle manager tasks must be moved
under the overlord with one caveat, you must specify javaOpts as an array:
+`druid.indexer.runner.javaOptsArray`, `druid.indexer.runner.javaOpts` is no
longer supported.
+
+If you are running without a middle manager you need to also use
`druid.processing.intermediaryData.storage.type=deepstore`
+
+### Custom Template Pod Adapter
+The custom template pod adapter allows you to specify a pod template file per
task type for more flexibility on how to define your pods. This adapter expects
a [Pod
Template](https://kubernetes.io/docs/concepts/workloads/pods/#pod-templates) to
be available on the overlord's file system. This pod template is used as the
base of the pod spec for the Kubernetes Job. You can override things like
labels, environment variables, resources, annotation, or even the base image
with this template. To enable this pod adapter you can specify the runtime
property `druid.indexer.runner.k8s.adapter.type: customTemplateAdapter`
+
+The base pod template must be specified as the runtime property
`druid.indexer.runner.k8s.podTemplate.base: /path/to/basePodSpec.yaml`
+
+Task specific pod templates must be specified as the runtime property
`druid.indexer.runner.k8s.podTemplate.{taskType}:
/path/to/taskSpecificPodSpec.yaml` where {taskType} is the name of the task
type i.e `index_parallel`
+
+The following is an example Pod Template that uses the regular druid docker
image.
+```
+apiVersion: "v1"
+kind: "PodTemplate"
+template:
+ metadata:
+ annotations:
+ custom-annotation: "hello"
+ labels:
+ custom-label: "hello"
+ spec:
+ affinity: {}
+ containers:
+ - command:
+ - sh
+ - -c
+ - |
+ /peon.sh /druid/data 1
+ env:
+ - name: CUSTOM_ENV_VARIABLE
+ value: "hello"
+ image: apache/druid:26.0.0
+ name: main
+ ports:
+ - containerPort: 8091
+ name: druid-tls-port
+ protocol: TCP
+ - containerPort: 8100
+ name: druid-port
+ protocol: TCP
+ resources:
+ limits:
+ cpu: "1"
+ memory: 2400M
+ requests:
+ cpu: "1"
+ memory: 2400M
Review Comment:
there was also 1G of direct mem so this is a pretty reasonable request i
think
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
