[
https://issues.apache.org/jira/browse/AIRFLOW-6163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17421862#comment-17421862
]
Younes Abouelnagah commented on AIRFLOW-6163:
---------------------------------------------
I agree the reasoning in https://issues.apache.org/jira/browse/AIRFLOW-1786 is
completely flawed, because if you want to soft_fail and succeed then you need
to handle the failure. If you soft_fail you can't just skip everything else,
and you can't just change the behavior of something like this. Maybe add
another mode! This breaks DAGs after upgrading to 1.10 (which is kinda overdue,
I admit). Must fix, please!
> BaseSensorOperator with soft_fail=True does not take into account downstream
> trigger_rule
> -----------------------------------------------------------------------------------------
>
> Key: AIRFLOW-6163
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6163
> Project: Apache Airflow
> Issue Type: Bug
> Components: operators, scheduler
> Affects Versions: 1.10.6
> Reporter: Damian Momot
> Priority: Major
>
> It looks like BaseSensorOperator skips all downstream tasks regardless of
> their trigger_rule setting. Using following example I would expect that
> "default_task" will be skipped, but "none_failed_task" should be run.
> Unfortunately both tasks are skipped
>
> {code:java}
> from datetime import datetime, timedelta
> from airflow.operators.dummy_operator import DummyOperator
> from airflow.utils.trigger_rule import TriggerRule
> from airflow import DAG
> from airflow.contrib.sensors.python_sensor import PythonSensor
> dag = DAG(
> dag_id="trigger_rule_test",
> schedule_interval=timedelta(minutes=1),
> start_date=(datetime.today() - timedelta(minutes=2))
> )
> soft_fail_sensor = PythonSensor(
> task_id="soft_fail_sensor",
> soft_fail=True,
> timeout=5,
> poke_interval=1,
> retries=0,
> python_callable=lambda *args, **kwargs: False,
> dag=dag
> )
> default_task = DummyOperator(
> task_id="default_task",
> dag=dag
> )
> none_failed_task = DummyOperator(
> task_id="none_failed_task",
> trigger_rule=TriggerRule.NONE_FAILED,
> dag=dag
> )
> default_task.set_upstream(soft_fail_sensor)
> none_failed_task.set_upstream(soft_fail_sensor)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
