raman created AIRFLOW-3273:
------------------------------
Summary: Support for Passing Custom Env variables while launching
k8 Pod
Key: AIRFLOW-3273
URL: https://issues.apache.org/jira/browse/AIRFLOW-3273
Project: Apache Airflow
Issue Type: Improvement
Components: kubernetes
Reporter: raman
Is there a way to provide env variables while launching K8 pod through K8
executor. we need to pass some env variable which are referred inside our
Airflow Operator. so can we provide custom env variable to docker run command
while launching task pod. Currently it seems that it supports predefined env
variable.
worker_configuration.py
def _get_environment(self): """Defines any necessary environment variables for
the pod executor""" env = \{ 'AIRFLOW__CORE__DAGS_FOLDER': '/tmp/dags',
'AIRFLOW__CORE__EXECUTOR': 'LocalExecutor' } if
self.kube_config.airflow_configmap: env['AIRFLOW__CORE__AIRFLOW_HOME'] =
self.worker_airflow_home return env
Possible solution
At the moment there is not a way to configure environmental variables on a
per-task basis, but it shouldn't be too hard to add that functionality. Extra
config options can be passed through the `executor_config` on any operator:
[https://github.com/apache/incubator-airflow/blob/master/airflow/models.py#L2423-L2437]
Which are eventually used here to construct the kubernetes pod for the task:
[https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/kubernetes/worker_configuration.py#L186]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
