Aneesh Joseph created AIRFLOW-4986:
--------------------------------------
Summary: PATH variable isn't passed down when the worker while
using run_as_user in the DAG
Key: AIRFLOW-4986
URL: https://issues.apache.org/jira/browse/AIRFLOW-4986
Project: Apache Airflow
Issue Type: Bug
Components: worker
Affects Versions: 1.10.3
Reporter: Aneesh Joseph
I have installed airflow into a virtual environment and I start up the airflow
worker using a systemd script with
{code:java}
EnvironmentFile=/mnt/airflow/airflow.envExecStart=/bin/bash -a -c 'export
AIRFLOW_HOME && export PATH && exec /mnt/airflow/venv/bin/airflow worker
--queues queue1,queue2'{code}
/mnt/airflow/airflow.env looks like this
{code:java}
AIRFLOW_HOME=/mnt/airflow/
PATH=/mnt/airflow/venv/bin/:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin{code}
The service starts up ok and the worker shows up on the celery flower UI. The
worker correctly picks up the DAGs scheduled on queue1 or queue2, but the tasks
with a *run_as_user* setting fail without showing much info on the Airflow UI
logs, but looking at the worker log for that specific task instance shows this.
{noformat}
[2019-07-17 17:09:54,644] {__init__.py:1374} INFO - Executing
<Task(BashOperator): sample_task> on 2019-07-17T17:09:48.710579+00:00
[2019-07-17 17:09:54,644] {base_task_runner.py:119} INFO - Running: [u'sudo',
u'-E', u'-H', u'-u', 'myuser', u'airflow', u'run',
'sample-team-sample-pipeline', 'sample_task',
'2019-07-17T17:09:48.710579+00:00', u'--job_id', '471', u'--raw', u'-sd',
u'DAGS_FOLDER/sample_team/sample-dag.py', u'--cfg_path', '/tmp/tmpZe8CVI']
[2019-07-17 17:09:54,662] {base_task_runner.py:101} INFO - Job 471: Subtask
sample_task sudo: airflow: command not found {noformat}
Tasks which doesn't have the run_as_user setting runs alright. Workers where
airflow is globally installed(and not in a virtual env) works alright which
probably suggests me that the PATH variable is not being passed down when the
worker triggers the actual task bit with sudo.
Is there a way to get around this?
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
