Taragolis commented on issue #37243:
URL: https://github.com/apache/airflow/issues/37243#issuecomment-1933877377
Could you provide minimum reproducible example, as I could see this one
works fine in 2.7.3 as well as into the `main` branch
```python
from datetime import datetime, timezone
from airflow import DAG
from airflow.decorators import task
from airflow.utils.task_group import TaskGroup
def callback_1(context):
print("callback_1 called")
def callback_2(context):
print("callback_2 called")
def callback_3(context):
print("callback_3 called")
START_DATE = datetime(2024, 2, 1, tzinfo=timezone.utc)
DEFAULT_ARGS_DAG = {"on_failure_callback": [callback_1, callback_2],
"retires": 0}
DEFAULT_ARGS_TG = {"on_failure_callback": [callback_3, callback_2]}
with DAG(
"pr_37243",
schedule=None,
start_date=START_DATE,
default_args=DEFAULT_ARGS_DAG,
tags=["37243", "on_failure_callback", "default_args"]
):
@task(trigger_rule="all_done")
def fail_task():
1 / 0
with TaskGroup("group", default_args=DEFAULT_ARGS_TG) as tg:
fail_task()
fail_task() >> tg >> fail_task.override(on_failure_callback=[callback_3,
callback_1])()
```
```console
[2024-02-08, 11:22:37 UTC] {taskinstance.py:1159} INFO - Dependencies all
met for dep_context=non-requeueable deps ti=<TaskInstance: pr_37243.fail_task
manual__2024-02-08T11:22:35.523637+00:00 [queued]>
[2024-02-08, 11:22:37 UTC] {taskinstance.py:1159} INFO - Dependencies all
met for dep_context=requeueable deps ti=<TaskInstance: pr_37243.fail_task
manual__2024-02-08T11:22:35.523637+00:00 [queued]>
[2024-02-08, 11:22:37 UTC] {taskinstance.py:1361} INFO - Starting attempt 1
of 1
[2024-02-08, 11:22:37 UTC] {taskinstance.py:1382} INFO - Executing
<Task(_PythonDecoratedOperator): fail_task> on 2024-02-08 11:22:35.523637+00:00
[2024-02-08, 11:22:37 UTC] {standard_task_runner.py:57} INFO - Started
process 59 to run task
[2024-02-08, 11:22:37 UTC] {standard_task_runner.py:84} INFO - Running:
['***', 'tasks', 'run', 'pr_37243', 'fail_task',
'manual__2024-02-08T11:22:35.523637+00:00', '--job-id', '5', '--raw',
'--subdir', 'DAGS_FOLDER/issue_37243.py', '--cfg-path', '/tmp/tmpk65t7ms3']
[2024-02-08, 11:22:37 UTC] {standard_task_runner.py:85} INFO - Job 5:
Subtask fail_task
[2024-02-08, 11:22:38 UTC] {task_command.py:416} INFO - Running
<TaskInstance: pr_37243.fail_task manual__2024-02-08T11:22:35.523637+00:00
[running]> on host 0f46ca170204
[2024-02-08, 11:22:38 UTC] {taskinstance.py:1662} INFO - Exporting env vars:
AIRFLOW_CTX_DAG_OWNER='***' AIRFLOW_CTX_DAG_ID='pr_37243'
AIRFLOW_CTX_TASK_ID='fail_task'
AIRFLOW_CTX_EXECUTION_DATE='2024-02-08T11:22:35.523637+00:00'
AIRFLOW_CTX_TRY_NUMBER='1'
AIRFLOW_CTX_DAG_RUN_ID='manual__2024-02-08T11:22:35.523637+00:00'
[2024-02-08, 11:22:38 UTC] {taskinstance.py:1937} ERROR - Task failed with
exception
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.9/site-packages/airflow/decorators/base.py",
line 221, in execute
return_value = super().execute(context)
File
"/home/airflow/.local/lib/python3.9/site-packages/airflow/operators/python.py",
line 192, in execute
return_value = self.execute_callable()
File
"/home/airflow/.local/lib/python3.9/site-packages/airflow/operators/python.py",
line 209, in execute_callable
return self.python_callable(*self.op_args, **self.op_kwargs)
File "/opt/airflow/dags/issue_37243.py", line 31, in fail_task
1 / 0
ZeroDivisionError: division by zero
[2024-02-08, 11:22:38 UTC] {taskinstance.py:1400} INFO - Marking task as
FAILED. dag_id=pr_37243, task_id=fail_task, execution_date=20240208T112235,
start_date=20240208T112237, end_date=20240208T112238
[2024-02-08, 11:22:38 UTC] {logging_mixin.py:154} INFO - callback_1 called
[2024-02-08, 11:22:38 UTC] {logging_mixin.py:154} INFO - callback_2 called
[2024-02-08, 11:22:38 UTC] {standard_task_runner.py:104} ERROR - Failed to
execute job 5 for task fail_task (division by zero; 59)
[2024-02-08, 11:22:38 UTC] {local_task_job_runner.py:228} INFO - Task exited
with return code 1
[2024-02-08, 11:22:38 UTC] {taskinstance.py:2778} INFO - 1 downstream tasks
scheduled from follow-on schedule check
```
```console
[2024-02-08, 11:22:39 UTC] {taskinstance.py:1159} INFO - Dependencies all
met for dep_context=non-requeueable deps ti=<TaskInstance:
pr_37243.group.fail_task manual__2024-02-08T11:22:35.523637+00:00 [queued]>
[2024-02-08, 11:22:39 UTC] {taskinstance.py:1159} INFO - Dependencies all
met for dep_context=requeueable deps ti=<TaskInstance: pr_37243.group.fail_task
manual__2024-02-08T11:22:35.523637+00:00 [queued]>
[2024-02-08, 11:22:39 UTC] {taskinstance.py:1361} INFO - Starting attempt 1
of 1
[2024-02-08, 11:22:39 UTC] {taskinstance.py:1382} INFO - Executing
<Task(_PythonDecoratedOperator): group.fail_task> on 2024-02-08
11:22:35.523637+00:00
[2024-02-08, 11:22:39 UTC] {standard_task_runner.py:57} INFO - Started
process 62 to run task
[2024-02-08, 11:22:39 UTC] {standard_task_runner.py:84} INFO - Running:
['***', 'tasks', 'run', 'pr_37243', 'group.fail_task',
'manual__2024-02-08T11:22:35.523637+00:00', '--job-id', '6', '--raw',
'--subdir', 'DAGS_FOLDER/issue_37243.py', '--cfg-path', '/tmp/tmpakwhoxcs']
[2024-02-08, 11:22:39 UTC] {standard_task_runner.py:85} INFO - Job 6:
Subtask group.fail_task
[2024-02-08, 11:22:39 UTC] {task_command.py:416} INFO - Running
<TaskInstance: pr_37243.group.fail_task
manual__2024-02-08T11:22:35.523637+00:00 [running]> on host 0f46ca170204
[2024-02-08, 11:22:40 UTC] {taskinstance.py:1662} INFO - Exporting env vars:
AIRFLOW_CTX_DAG_OWNER='***' AIRFLOW_CTX_DAG_ID='pr_37243'
AIRFLOW_CTX_TASK_ID='group.fail_task'
AIRFLOW_CTX_EXECUTION_DATE='2024-02-08T11:22:35.523637+00:00'
AIRFLOW_CTX_TRY_NUMBER='1'
AIRFLOW_CTX_DAG_RUN_ID='manual__2024-02-08T11:22:35.523637+00:00'
[2024-02-08, 11:22:40 UTC] {taskinstance.py:1937} ERROR - Task failed with
exception
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.9/site-packages/airflow/decorators/base.py",
line 221, in execute
return_value = super().execute(context)
File
"/home/airflow/.local/lib/python3.9/site-packages/airflow/operators/python.py",
line 192, in execute
return_value = self.execute_callable()
File
"/home/airflow/.local/lib/python3.9/site-packages/airflow/operators/python.py",
line 209, in execute_callable
return self.python_callable(*self.op_args, **self.op_kwargs)
File "/opt/airflow/dags/issue_37243.py", line 31, in fail_task
1 / 0
ZeroDivisionError: division by zero
[2024-02-08, 11:22:40 UTC] {taskinstance.py:1400} INFO - Marking task as
FAILED. dag_id=pr_37243, task_id=group.fail_task,
execution_date=20240208T112235, start_date=20240208T112239,
end_date=20240208T112240
[2024-02-08, 11:22:40 UTC] {logging_mixin.py:154} INFO - callback_3 called
[2024-02-08, 11:22:40 UTC] {logging_mixin.py:154} INFO - callback_2 called
[2024-02-08, 11:22:40 UTC] {standard_task_runner.py:104} ERROR - Failed to
execute job 6 for task group.fail_task (division by zero; 62)
[2024-02-08, 11:22:40 UTC] {local_task_job_runner.py:228} INFO - Task exited
with return code 1
[2024-02-08, 11:22:40 UTC] {taskinstance.py:2778} INFO - 1 downstream tasks
scheduled from follow-on schedule check
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
