ssulca opened a new issue, #30626:
URL: https://github.com/apache/airflow/issues/30626
### Apache Airflow Provider(s)
amazon
### Versions of Apache Airflow Providers
`apache-airflow-providers-amazon==7.4.1`
### Apache Airflow version
2.3.2
### Operating System
ubuntu
### Deployment
Docker-Compose
### Deployment details
* Docker version 23.0.3
* Docker Compose version 2.17.2
* Docker image: apache/airflow:2.3.2-python3.9
### What happened
In `wait_for_state` function there is a wrong condition
```python
while status in self.NON_TERMINAL_STATES and status != target_state:
self.log.info("Current status is %s", status)
if status in self.FAILED_STATES:
info = self.get_error_info(aws_account_id, data_set_id, ingestion_id)
raise AirflowException(f"The Amazon QuickSight Ingestion failed.
Error info: {info}")
if status == "CANCELLED":
raise AirflowException("The Amazon QuickSight SPICE ingestion
cancelled!")
# wait and try again
time.sleep(check_interval)
sec += check_interval
status = self.get_status(aws_account_id, data_set_id, ingestion_id)
```
in case `status` return an state like `FAILED` or `CANCELLED` , the
condition `status in self.NON_TERMINAL_STATES` will be false, so `while`
condition will finish and wont raise any `AirflowException`
### What you think should happen instead
the `status` needs to be got before `if` statements
```python
while status in self.NON_TERMINAL_STATES and status != target_state:
time.sleep(check_interval)
self.log.info("Current status is %s", status)
status = self.get_status(aws_account_id, data_set_id, ingestion_id)
if status in self.FAILED_STATES:
info = self.get_error_info(aws_account_id, data_set_id, ingestion_id)
raise AirflowException(f"The Amazon QuickSight Ingestion failed.
Error info: {info}")
if status == "CANCELLED":
raise AirflowException("The Amazon QuickSight SPICE ingestion
cancelled!")
# wait and try again
sec += check_interval
```
### How to reproduce
Create an ingestion, on a dataset where the `FULL_REFRESH` ingestion time is
less than `check_interval`, as:
```python
from uuid import uuid1
QuickSightCreateIngestionOperator(
task_id="quicksight_ingestion_update",
data_set_id="{{ var.value.dataset_id }}",
ingestion_type="FULL_REFRESH",
ingestion_id=str(uuid1()),
wait_for_completion=True,
aws_conn_id="aws_conn_default",
)
```
And where ingestion finish as `FAILED` ingestion
### Anything else
logs where start and then finish with `status = FAILED`
```sh
[2023-04-11, 13:05:36 -03] {quicksight.py:143} INFO - Current status is
RUNNING
[2023-04-11, 13:06:07 -03] {quicksight.py:152} INFO - QuickSight Ingestion
completed
```
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
