[
https://issues.apache.org/jira/browse/AIRFLOW-5540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adan Christian Rosales Ornelas reassigned AIRFLOW-5540:
-------------------------------------------------------
Assignee: Adan Christian Rosales Ornelas
> task with SparkSubmitOperator does not fail if the spark job it executes
> fails.
> ----------------------------------------------------------------------------------
>
> Key: AIRFLOW-5540
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5540
> Project: Apache Airflow
> Issue Type: Wish
> Components: operators
> Affects Versions: 1.10.3
> Environment: RHEL 7.3 with default airflow installation.
> Reporter: Juan M George
> Assignee: Adan Christian Rosales Ornelas
> Priority: Major
> Attachments: airfow_issues
>
>
> In my test Dag, I have a task that uses SparkSubmitOperator operator to
> execute a spark job that reads from a table in a database and does some
> processing and write into a file. In a scenario, where source table from
> where I read data does not exist, the spark job fails but the task is
> still be shown as executed successfully. I don't see any other way to
> handle this business logic failure from the operator side.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
