kaxil commented on issue #10027: URL: https://github.com/apache/airflow/issues/10027#issuecomment-666005385
You are using it wrongly. Let me try to explain. "airflow.cfg" and "airflow_local_settings.py" file needs to exist on the Scheduler (Whether your Scheduler is on a VM or a POD isn't relevant here). We have also added documentation on where to out this file: https://airflow.apache.org/docs/stable/concepts.html#where-to-put-airflow-local-settings-py The `pod_mutation_hook` now is used whenever you use `KubernetesExecutor` or `KubernetePodOperator`. The PODs launched by KubernetesExecutor or KubernetePodOperator will use this mutation hook. Now, back to the configmap. The case when you are using `KubernetesExecutor` and have a task that use KuberneretPodOperator, you need both `airflow.cfg` and `airflow_local_settings.py` file to exist on the worker pods launched by the KubernetesExecutor. The KubernetesExecutor launches a Worker Pod for this task. Scheduler Pod ---> Worker Pod (Pod_1 -- launched by KubernetesExecuetor) --> (Pod_2 -- launched by Pod_1 the task using KubernetePodOperator) Now the entire **[kubernetes]** section in airflow.cfg (https://github.com/apache/airflow/blob/master/airflow/config_templates/default_airflow.cfg#L870-L1028) is used only for KubernetesExecutor and affects what is mounted on the **Worker Pods** launched by KubernetesExecutor. If you don't specify `airflow_local_settings` configmap, the airflow_local_settings file would not be mounted to the worker pod (Pod_1 in the above example) and only airflow.cfg file is mounted. So now for Pod_2 (launched by Pod_1) -- (special case when you use KubernetesPodOperator with KubernetesExecutor), since Pod_1 (the worker POD) doesn't have airflow_local_settings.py file even though the Scheduler has it, Pod_2 would not be mutated since the file doesn't exist over there. Consider it same as airflow.cfg -- why do you mount airflow.cfg file both to the Scheduler POD and the worker POD. Similarly for this edge case you need airflow_local_settings.py file at both the places. https://github.com/apache/airflow/blob/ba2d6408e64f219e8f53a20a5a149e3d8109db31/airflow/kubernetes/worker_configuration.py#L279-L305 --> This code is used to decide what is mounted on the Worker Pod (**REF_1**) https://github.com/apache/airflow/blob/ba2d6408e64f219e8f53a20a5a149e3d8109db31/airflow/executors/kubernetes_executor.py#L462-L481 --> Pod created for each task run by KubernetesExecutor (**REF_2**) -- The mutation is applied to this POD as this is launched by the Scheduler and it has airflow_local_settings.py file https://github.com/apache/airflow/blob/ba2d6408e64f219e8f53a20a5a149e3d8109db31/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py#L383 --> This code is used to create a new POD when using KubernetesPod Operator (REF_3) -- Since airflow_local_settings.py wasn't mounted on POD generated in **REF_2** the mutations weren't applied to this POD. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
