F D created AIRFLOW-5700:
----------------------------
Summary: KubeExecutor - Specify YAML template
Key: AIRFLOW-5700
URL: https://issues.apache.org/jira/browse/AIRFLOW-5700
Project: Apache Airflow
Issue Type: New Feature
Components: executor-kubernetes
Affects Versions: 1.10.5
Reporter: F D
Assignee: Daniel Imberman
As a developer - I find myself struggling with the Airflow implementation of
launching pods within the KubeExecutor. Things are implemented in ways that
don't work with my kubernetes implementation, are unnecessarily difficult
because certain features of the Kubernetes Spec aren't implemented in a way I
can configure, or features are forced to be implemented in a certain way
because of historical design decisions by the airflow team.
Dask has really good Kubernetes support. They provide a hook to define a pod
template via YAML, OR to use defaults.
You can see an example here:
https://kubernetes.dask.org/en/latest/#quickstart
If Airflow implemented support for creating pods via a YAML template, new
feature requests and maintenance for the KubernetesExecutor would drop to very
little, and airflow users would have a lot of power and flexibility with what
pods look like in their system.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
