[
https://issues.apache.org/jira/browse/AIRFLOW-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16938003#comment-16938003
]
Aaron Dossett commented on AIRFLOW-4216:
----------------------------------------
We have observed something extremely similar to this on Airflow 1.10.3. In our
scenario the DAG's root tasks (i.e. no dependencies) are all sensors in
`reschedule` mode. We observe this pattern:
1) When the DAG starts the sensor tries to run the first time and we see this
in the scheduler log:
{noformat}
[2019-09-25 00:32:24,087] {jobs.py:1815} INFO - Creating / updating
<TaskInstance: etsy_dag_foo.sensor_buyer_first_last_mart 2019-09-24
00:30:00+00:00 [scheduled]> in ORM
<TaskInstance: etsy_dag_foo.sensor_buyer_first_last_mart 2019-09-24
00:30:00+00:00 [scheduled]>"
<TaskInstance: etsy_dag_foo.sensor_buyer_first_last_mart 2019-09-24
00:30:00+00:00 [queued]>"
[2019-09-25 00:32:28,070] {cli.py:517} INFO - Running <TaskInstance:
etsy_dag_foo.sensor_buyer_first_last_mart 2019-09-24T00:30:00+00:00 [queued]>
on host 9c74fbaa8a8a"
{noformat}
We do next expect this sensor to succeed the first time it runs.
2) A few minutes later we see:
{noformat}
[2019-09-25 00:36:06,304] {jobs.py:1815} INFO - Creating / updating
<TaskInstance: etsy_dag_foo.sensor_buyer_first_last_mart 2019-09-24
00:30:00+00:00 [scheduled]> in ORM
<TaskInstance: etsy_dag_foo.sensor_buyer_first_last_mart 2019-09-24
00:30:00+00:00 [scheduled]>
<TaskInstance: etsy_dag_foo.sensor_buyer_first_last_mart 2019-09-24
00:30:00+00:00 [queued]>
[2019-09-25 00:36:08,083] {jobs.py:1469} INFO - Executor reports execution of
etsy_dag_foo.sensor_buyer_first_last_mart execution_date=2019-09-24
00:30:00+00:00 exited with status success for try_number 1
[2019-09-25 00:36:08,090] {jobs.py:1485} ERROR - Executor reports task instance
<TaskInstance: etsy_dag_foo.sensor_buyer_first_last_mart 2019-09-24
00:30:00+00:00 [queued]> finished (success) although the task says its queued.
Was the task killed externally?
{noformat}
The tasks never succeed and never run a second time, they always hit that error.
DAGs with these sensors at the root that succeed the first time run just fine.
> Sensors not rescheduled when root task with 'reschedule' mode
> -------------------------------------------------------------
>
> Key: AIRFLOW-4216
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4216
> Project: Apache Airflow
> Issue Type: Bug
> Reporter: Jannik Franz
> Priority: Major
>
> When using Sensors and having them as root tasks (such as no prior dependency
> task) and using mode 'reschedule' then the DAG state will be set to 'failed'
> and tasks won't be rescheduled.
> This is when the state of a DAG is updated it doesn't check for
> 'up_for_reschedule' in tasks properly and sets the state to failed.
> The expected behaviour would be to reschedule the task without failing the
> DAG.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
