FrankChen021 commented on code in PR #17738: URL: https://github.com/apache/druid/pull/17738#discussion_r1961551377
########## docs/development/extensions-core/k8s-jobs.md: ########## @@ -33,13 +33,32 @@ The K8s extension builds a pod spec for each task using the specified pod adapte ## Configuration -To use this extension please make sure to [include](../../configuration/extensions.md#loading-extensions)`druid-kubernetes-overlord-extensions` in the extensions load list for your overlord process. +To use this extension please make sure to [include](../../configuration/extensions.md#loading-extensions) `druid-kubernetes-overlord-extensions` in the extensions load list for your overlord process. The extension uses `druid.indexer.runner.capacity` to limit the number of k8s jobs in flight. A good initial value for this would be the sum of the total task slots of all the middle managers you were running before switching to K8s based ingestion. The K8s task runner uses one thread per Job that is created, so setting this number too large can cause memory issues on the overlord. Additionally set the variable `druid.indexer.runner.namespace` to the namespace in which you are running druid. Other configurations required are: `druid.indexer.runner.type: k8s` and `druid.indexer.task.encapsulatedTask: true` +### Running Task Pods in Another Namespace + +It is possible to run task pods in a different namespace from the rest of your Druid cluster. + +If you are running multiple Druid clusters and would like to have a dedicated namespace for all your task pods, you can make the following changes to the runtime properties for your Overlord deployment: + +- `druid.indexer.runner.namespace`: The namespace where the task pods will run. +- `druid.indexer.runner.overlordNamespace`: The namespace where the Overlord resides. Review Comment: ```suggestion - `druid.indexer.runner.overlordNamespace`: The namespace where the Overlord resides. This must be defined when tasks are scheduled in different namespace. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
