mik-laj commented on a change in pull request #10916: URL: https://github.com/apache/airflow/pull/10916#discussion_r487811569
########## File path: docs/executor/kubernetes.rst ########## @@ -41,6 +41,61 @@ The volumes are optional and depend on your configuration. There are two volumes To troubleshoot issue with KubernetesExecutor, you can use ``airflow kubernetes generate-dag-yaml`` command. This command generates the pods as they will be launched in Kubernetes and dumps them into yaml files for you to inspect. +.. _concepts:pod_template_file: + +pod_template_file +################# + +As of Airflow 1.10.12, users can now use a pod_template_file setting in their airflow.cfg to form the basis of +their KubernetesExecutor pods. This process is faster and easier to modify. + +We include multiple examples of working pod operators below, but we would also like to explain a few necessary components +if you want to customize your pod_template_files. As long as you have these components, every other element +in the template is +customizable. + +1. Airflow will overwrite the base container image and the pod name + +There are two points where airflow potentially overwrites the base image: in the airflow.cfg +or the ``pod_override`` (discussed below) setting. This value is overwitten to ensure that users do +not need to update multiple template files every time they upgrade their docker image. The other field +that Airflow overwrites is the ``pod.metadata.name`` field. This field has to be unique across all pods, +so we generate these names dynamically before launch. + +It's important to note while Airflow overwrites these fields, they **can not be left blank**. +If these fields do not exist, kubernetes can not load the yaml into a Kuberentes V1Pod + +2. Each airflow ``pod_template_file`` must have a container named ``base`` at the ``pod.spec.containers[0]`` position + +Airflow uses the ``pod_template_file`` by making certain assumptions about the structure of the template. +When airflow creates the worker pod's command, it assumes that the airflow worker container part exists +at the beginning of the container array. It then assumes that the container is named "base" +when it merges this pod with internal configs. Users are more than welcome to create Review comment: ```suggestion when it merges this pod with internal configs. You are more than welcome to create ``` https://developers.google.com/style/person ########## File path: docs/executor/kubernetes.rst ########## @@ -41,6 +41,61 @@ The volumes are optional and depend on your configuration. There are two volumes To troubleshoot issue with KubernetesExecutor, you can use ``airflow kubernetes generate-dag-yaml`` command. This command generates the pods as they will be launched in Kubernetes and dumps them into yaml files for you to inspect. +.. _concepts:pod_template_file: + +pod_template_file +################# + +As of Airflow 1.10.12, users can now use a pod_template_file setting in their airflow.cfg to form the basis of +their KubernetesExecutor pods. This process is faster and easier to modify. + +We include multiple examples of working pod operators below, but we would also like to explain a few necessary components +if you want to customize your pod_template_files. As long as you have these components, every other element +in the template is +customizable. + +1. Airflow will overwrite the base container image and the pod name + +There are two points where airflow potentially overwrites the base image: in the airflow.cfg +or the ``pod_override`` (discussed below) setting. This value is overwitten to ensure that users do +not need to update multiple template files every time they upgrade their docker image. The other field +that Airflow overwrites is the ``pod.metadata.name`` field. This field has to be unique across all pods, +so we generate these names dynamically before launch. + +It's important to note while Airflow overwrites these fields, they **can not be left blank**. +If these fields do not exist, kubernetes can not load the yaml into a Kuberentes V1Pod + +2. Each airflow ``pod_template_file`` must have a container named ``base`` at the ``pod.spec.containers[0]`` position + +Airflow uses the ``pod_template_file`` by making certain assumptions about the structure of the template. +When airflow creates the worker pod's command, it assumes that the airflow worker container part exists +at the beginning of the container array. It then assumes that the container is named "base" +when it merges this pod with internal configs. Users are more than welcome to create Review comment: ```suggestion when it merges this pod with internal configs. You are more than welcome to create ``` See: https://developers.google.com/style/person ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org