Ram created AIRFLOW-1468: ---------------------------- Summary: Airflow job in queue state when worker is different from master Key: AIRFLOW-1468 URL: https://issues.apache.org/jira/browse/AIRFLOW-1468 Project: Apache Airflow Issue Type: Bug Reporter: Ram
Hi I setup the master/worker infrastructure. When i schedule my job with master as worker the job ran successfully. But when worker is different machine then i am seeing different behavior. In logs folder on remote work log is created and able see the output of bash operator and log showing the message like below [2017-07-26 15:51:14,892] {base_task_runner.py:95} INFO - Subtask: [2017-07-26 15:51:14,891] {bash_operator.py:94} INFO - Wed Jul 26 15:51:14 PDT 2017 [2017-07-26 15:51:14,892] {base_task_runner.py:95} INFO - Subtask: [2017-07-26 15:51:14,892] {bash_operator.py:97} INFO - Command exited with return code 0 [2017-07-26 15:51:19,056] {jobs.py:2083} INFO - Task exited with return code 0 [2017-07-26 15:51:26,516] {models.py:167} INFO - Filling up the DagBag from /home/skathane/airflow/dags/test.py [2017-07-26 15:51:26,605] {base_task_runner.py:112} INFO - Running: ['bash', '-c', u'airflow run test t2 2017-07-26T15:51:10.186620 --job_id 92 --raw -sd DAGS_FOLDER/test.py'] [2017-07-26 15:51:27,077] {base_task_runner.py:95} INFO - Subtask: [2017-07-26 15:51:27,077] {__init__.py:57} INFO - Using executor CeleryExecutor It seems task executed successfully but it not terminated and in Queue state only. In log further i am seeing below error. [2017-07-26 15:51:27,396] {base_task_runner.py:95} INFO - Subtask: [2017-07-26 15:51:27,396] {models.py:1120} I{color:red}*NFO - Dependencies not met for <TaskInstance: test.t2 2017-07-26 15:51:10.186620 [success]>, dependency 'Task Instance State' FAILED: Task is in the 'success' state which is not a valid state for execution. The task must be cleared in order to be run. *{color} -- This message was sent by Atlassian JIRA (v6.4.14#64029)