[
https://issues.apache.org/jira/browse/AIRFLOW-3297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16678240#comment-16678240
]
ASF GitHub Bot commented on AIRFLOW-3297:
-----------------------------------------
danassa opened a new pull request #4152: AIRFLOW-3297 added possible states to
the EMR step sensor
URL: https://github.com/apache/incubator-airflow/pull/4152
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses
[AIRFLOW-3297](https://issues.apache.org/jira/browse/AIRFLOW-3297)
### Description
- [ ] This PR implements correct behaviour to EMR step states. Previously,
the states 'PENDING_CANCELLED' and 'INTERRUPTED' resulted in the Airflow task
being marked as successful. This PR fixed this by mentioning those states as
NON_TERMINAL_STATES and as FAILED_STATE, accordingly.
### Tests
- [ ] My PR adds the following unit tests - test_step_interrupted
### Code Quality
- [ ] Passes `flake8`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> EmrStepSensor marks cancelled step as successful
> -------------------------------------------------
>
> Key: AIRFLOW-3297
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3297
> Project: Apache Airflow
> Issue Type: Bug
> Reporter: Dana Assa
> Assignee: Dana Assa
> Priority: Minor
>
> When a step is manually cancelled or when the cluster is terminated
> unexpectedly, an EMR step can reach 'CANCEL_PENDING' state or 'INTERRUPTED'
> state.
> These states are not mentioned as FAILED_STATES nor as NON_TERMINAL_STATES in
> the "EmrStepSensor" class, thus the sensor returns True and the Airflow stage
> is marked as success instead of failed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
