[
https://issues.apache.org/jira/browse/AIRFLOW-2755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16546394#comment-16546394
]
Ravi Kotecha commented on AIRFLOW-2755:
---------------------------------------
PR: https://github.com/apache/incubator-airflow/pull/3612
> k8s workers think DAGs are always in `/tmp/dags`
> ------------------------------------------------
>
> Key: AIRFLOW-2755
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2755
> Project: Apache Airflow
> Issue Type: Bug
> Components: configuration, worker
> Reporter: Aldo
> Priority: Minor
>
> We have Airflow configured to use the `KubernetesExecutor` and run tasks in
> newly created pods.
> I tried to use the `PythonOperator` to import the python callable from a
> python module located in the DAGs directory as [that should be
> possible|https://github.com/apache/incubator-airflow/blob/c7a472ed6b0d8a4720f57ba1140c8cf665757167/airflow/__init__.py#L42].
> Airflow complained that the module was not found.
> After a fair amount of digging we found that the issue was that the workers
> have the `AIRFLOW__CORE__DAGS_FOLDER` environment variable set to `/tmp/dags`
> as [you can see from the
> code|https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/kubernetes/worker_configuration.py#L84].
> Unset that environment variable from within the task's pod and running the
> task manually worked as expected. I think that this path should be
> configurable (I'll give it a try to add a `kubernetes.worker_dags_folder`
> configuration).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
