mik-laj edited a comment on pull request #8150: URL: https://github.com/apache/airflow/pull/8150#issuecomment-663685993
@xEviL I confirm that pod_template_file option does not affect kubernetesPodOperaator. It's only for KubernetesExecutor.. pod_mutation_hook is global and is used by both the operator and the executor. Wouldn't you like to write some documentation that explains how you set up Kubernetes executor? I know our documentation for an executor is incomplete, but even writing one section is a step forward. I will try to motivate other people to complete documentation for other sections. > Is it a fixed/hard-coded manifest file? If so, I'd think its use cases are very limited. @fengsi In most cases, a static configuration is sufficient. If you want more dynamic configuration, you can extend this with [the pod mutation hook](https://airflow.readthedocs.io/en/latest/kubernetes.html#pod-mutation-hook) or [executor_config](https://github.com/apache/airflow/blob/master/airflow/example_dags/example_kubernetes_executor_config.py). If you want to base your logic based on task attributes, you can modify executor_config with [the cluster policy](https://airflow.readthedocs.io/en/latest/concepts.html#cluster-policy). > If we take a look at Airflow in K8s, its manifests are usually handled by other tools (e.g., Helm). Now both KubernetesExecutor and KubernetesPodOperator do similar things but rely on Airflow (K8s Python client), causing inconsistency between ecosystems. Yes. I realize this is not natural, but I think the YAML file brings us closer to the expected result. In the future, we might think about creating a CRD to make setup easier, but the CRD still contained a YAML file with a Pod definition. If we don't have to maintain a lot of configuration options and the code is less complex, I suspect it will happen soon. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
