kushagra391 opened a new issue #17124:
URL: https://github.com/apache/airflow/issues/17124
**Apache Airflow version**:
2.1.1
**Kubernetes version (if you are using kubernetes)** (use `kubectl version`):
1.19
**Environment**:
- **Cloud provider or hardware configuration**: AWS
- **OS** (e.g. from /etc/os-release): ubuntu
- **Kernel** (e.g. `uname -a`): Linux x86_64 GNU/Linux
- **Install tools**:
- **Others**:
**What happened**:
Successful AwsBatchOperator are exiting with return code 1 instead of 0
```
[2021-07-21 07:37:20,074] {operators.py:203} INFO - AWS Batch job
(f0e1c462-239e-4e83-b88e-e85acfcdc3d8) succeeded
[2021-07-21 07:37:20,134] {taskinstance.py:1204} INFO - Marking task as
SUCCESS. dag_id=poc_aws_batch, task_id=poc_aws_batch_pass,
execution_date=20210720T000000, start_date=20210721T073707,
end_date=20210721T073720
[2021-07-21 07:37:20,191] {local_task_job.py:151} INFO - Task exited with
return code 1
```
This is making callbacks like `on_success_callback()` not getting triggered.
**What you expected to happen**:
They should exit with return code 0 instead
<!-- What do you think went wrong? -->
Not sure
**How to reproduce it**:
Can be reproduced with a simple dag
```
with poc_emr_dag as dag:
start = DummyOperator(task_id="start", retries=3)
end = DummyOperator(task_id="end", retries=3)
poc_aws_batch_pass = AwsBatchOperator(
**build_job_params(task_id="poc_aws_batch_pass", fail_job=False)
)
start >> poc_aws_batch_pass >> end
```
Logs are included in details section
**Anything else we need to know**:
This started with v2.x. Same code was working fine on v.1.10.12
How often does this problem occur? Once? Every time etc?
Everytime
Any relevant logs to include? Put them here in side a detail tag:
<details><summary>Click here for code snippet + logs</summary>
Tried with a simple DAG
```python
default_args = dict(
depends_on_past=False,
start_date=DAG_START_DATE,
end_date=DAG_END_DATE,
email_on_failure=False,
email_on_retry=False,
retries=0,
retry_delay=timedelta(minutes=5),
schedule_interval="15 6 * * *",
)
poc_emr_dag = DAG(
dag_id=DAG_NAME,
default_args=default_args,
description="Test DAG for trying out AwsBatchOperator",
schedule_interval=timedelta(days=1),
tags=["example"],
)
with poc_emr_dag as dag:
start = DummyOperator(task_id="start")
end = DummyOperator(task_id="end")
poc_aws_batch_pass = AwsBatchOperator(
**build_job_params(task_id="poc_aws_batch_pass", fail_job=True)
)
start >> poc_aws_batch_pass >> end
```
Logs show
```
[2021-07-21 07:37:16,346] {batch_client.py:349} INFO - AWS Batch job
(f0e1c462-239e-4e83-b88e-e85acfcdc3d8) check status (SUCCEEDED) in ['RUNNING',
'SUCCEEDED', 'FAILED']
[2021-07-21 07:37:19,970] {batch_client.py:349} INFO - AWS Batch job
(f0e1c462-239e-4e83-b88e-e85acfcdc3d8) check status (SUCCEEDED) in
['SUCCEEDED', 'FAILED']
[2021-07-21 07:37:19,973] {batch_client.py:283} INFO - AWS Batch job
(f0e1c462-239e-4e83-b88e-e85acfcdc3d8) has completed
[2021-07-21 07:37:20,073] {batch_client.py:257} INFO - AWS batch job
(f0e1c462-239e-4e83-b88e-e85acfcdc3d8) succeeded: ....
[2021-07-21 07:37:20,074] {operators.py:203} INFO - AWS Batch job
(f0e1c462-239e-4e83-b88e-e85acfcdc3d8) succeeded
[2021-07-21 07:37:20,134] {taskinstance.py:1204} INFO - Marking task as
SUCCESS. dag_id=poc_aws_batch, task_id=poc_aws_batch_pass,
execution_date=20210720T000000, start_date=20210721T073707,
end_date=20210721T073720
[2021-07-21 07:37:20,191] {local_task_job.py:151} INFO - Task exited with
return code 1
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
