suneet-s commented on code in PR #14674:
URL: https://github.com/apache/druid/pull/14674#discussion_r1280215435
##########
docs/development/extensions-contrib/k8s-jobs.md:
##########
@@ -115,33 +75,197 @@ You can keep your Dockerfile the same but you must have a
sidecar spec like so:
- arg2
```
-The following roles must also be accessible. An example spec could be:
+For both of these adapters, you can add optional labels to your K8s jobs /
pods if you need them by using the following configuration:
+`druid.indexer.runner.labels: '{"key":"value"}'`
+Annotations are the same with:
+`druid.indexer.runner.annotations: '{"key":"value"}'`
+
+All other configurations you had for the middle manager tasks must be moved
under the overlord with one caveat, you must specify javaOpts as an array:
+`druid.indexer.runner.javaOptsArray`, `druid.indexer.runner.javaOpts` is no
longer supported.
+
+If you are running without a middle manager you need to also use
`druid.processing.intermediaryData.storage.type=deepstore`
+
+### Custom Template Pod Adapter
+The custom template pod adapter allows you to specify a pod template file per
task type for more flexibility on how to define your pods. This adapter expects
a [Pod
Template](https://kubernetes.io/docs/concepts/workloads/pods/#pod-templates) to
be available on the overlord's file system. This pod template is used as the
base of the pod spec for the Kubernetes Job. You can override things like
labels, environment variables, resources, annotation, or even the base image
with this template. To enable this pod adapter you can specify the runtime
property `druid.indexer.runner.k8s.adapter.type: customTemplateAdapter`
+
+The base pod template must be specified as the runtime property
`druid.indexer.runner.k8s.podTemplate.base: /path/to/basePodSpec.yaml`
+
+Task specific pod templates must be specified as the runtime property
`druid.indexer.runner.k8s.podTemplate.{taskType}:
/path/to/taskSpecificPodSpec.yaml` where {taskType} is the name of the task
type i.e `index_parallel`
+
+The following is an example Pod Template that uses the regular druid docker
image.
+```
+apiVersion: "v1"
+kind: "PodTemplate"
+template:
+ metadata:
+ annotations:
+ sidecar.istio.io/proxyCPU: "512m" # to handle a injected istio sidecar
+ labels:
+ app.kubernetes.io/name: "druid-realtime-backend"
+ spec:
+ affinity: {}
+ containers:
+ - command:
+ - sh
+ - -c
+ - |
+ /peon.sh /druid/data 1
+ env:
+ - name: CUSTOM_ENV_VARIABLE
+ value: "hello"
+ image: apache/druid:26.0.0
+ name: main
+ ports:
+ - containerPort: 8091
+ name: druid-tls-port
+ protocol: TCP
+ - containerPort: 8100
+ name: druid-port
+ protocol: TCP
+ resources:
+ limits:
+ cpu: "1"
+ memory: 2400M
+ requests:
+ cpu: "1"
+ memory: 2400M
+ volumeMounts:
+ - mountPath: /opt/druid/conf/druid/cluster/master/coordinator-overlord #
runtime props are still mounted in this location because that's where peon.sh
looks for configs
+ name: nodetype-config-volume
+ readOnly: true
+ - mountPath: /druid/data
+ name: data-volume
+ - mountPath: /druid/deepstorage
+ name: deepstorage-volume
+ restartPolicy: "Never"
+ securityContext:
+ fsGroup: 1000
+ runAsGroup: 1000
+ runAsUser: 1000
+ tolerations:
+ - effect: NoExecute
+ key: node.kubernetes.io/not-ready
+ operator: Exists
+ tolerationSeconds: 300
+ - effect: NoExecute
+ key: node.kubernetes.io/unreachable
+ operator: Exists
+ tolerationSeconds: 300
+ volumes:
+ - configMap:
+ defaultMode: 420
+ name: druid-tiny-cluster-peons-config
+ name: nodetype-config-volume
+ - emptyDir: {}
+ name: data-volume
+ - emptyDir: {}
+ name: deepstorage-volume
+```
+
+The below runtime properties need to be passed to the Job's peon process.
```
-apiVersion: rbac.authorization.k8s.io/v1
+druid.port=8100 (what port the peon should run on)
+druid.peon.mode=remote
+druid.service=druid/peon (for metrics reporting)
+druid.indexer.task.baseTaskDir=/druid/data (this should match the argument to
the ./peon.sh run command in the PodTemplate)
+druid.indexer.runner.type=k8s
+druid.indexer.task.encapsulatedTask=true
+```
+
+Any runtime property or JVM config used by the peon process can also be
passed. E.G. below is a example of a ConfigMap that can be used to generate the
`nodetype-config-volume` mount in the above template.
+```
+kind: ConfigMap
+metadata:
+ name: druid-tiny-cluster-peons-config
+ namespace: default
+apiVersion: v1
+data:
+ jvm.config: |-
+ -server
+ -XX:MaxDirectMemorySize=1000M
+ -Duser.timezone=UTC
+ -Dfile.encoding=UTF-8
+ -Dlog4j.debug
+ -Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager
+ -Djava.io.tmpdir=/druid/data
+ -Xmx1024M
+ -Xms1024M
+ log4j2.xml: |-
+ <?xml version="1.0" encoding="UTF-8" ?>
+ <Configuration status="WARN">
+ <Appenders>
+ <Console name="Console" target="SYSTEM_OUT">
+ <PatternLayout pattern="%d{ISO8601} %p [%t] %c - %m%n"/>
+ </Console>
+ </Appenders>
+ <Loggers>
+ <Root level="info">
+ <AppenderRef ref="Console"/>
+ </Root>
+ </Loggers>
+ </Configuration>
+ runtime.properties: |
+ druid.port=8100
+ druid.service=druid/peon
+ druid.server.http.numThreads=5
+ druid.indexer.task.baseTaskDir=/druid/data
+ druid.indexer.runner.type=k8s
+ druid.peon.mode=remote
+ druid.indexer.task.encapsulatedTask=true
+```
+
+
+Additional Configuration
Review Comment:
```suggestion
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
