karenbraganz commented on issue #51301: URL: https://github.com/apache/airflow/issues/51301#issuecomment-2960586034
@KarthikeyanDevendhiran I have created [discussion #51597](https://github.com/apache/airflow/discussions/51597) to discuss the increased incidence of stuck in queued tasks. Please provide the requested information on that discussion thread. As for the issue of stuck in queued tasks not triggering failure callbacks- the solution I proposed in [this comment](https://github.com/apache/airflow/issues/51301#issuecomment-2932821660) works for Airflow 2 but not Airflow 3. I implemented and tested my proposed changes on Astro Runtime 3.0-2 (Airflow 3.0.1) but saw the below error when the callback was executed: ``` [2025-06-10T18:14:13.466+0000] {manager.py:147} INFO - DAG bundles loaded: main --- Supervised process Last chance exception handler --- Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/airflow/sdk/execution_time/supervisor.py", line 369, in _fork_main target() File "/usr/local/lib/python3.12/site-packages/airflow/dag_processing/processor.py", line 90, in _parse_file_entrypoint result = _parse_file(msg, log) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/airflow/dag_processing/processor.py", line 106, in _parse_file _execute_callbacks(bag, msg.callback_requests, log) File "/usr/local/lib/python3.12/site-packages/airflow/dag_processing/processor.py", line 146, in _execute_callbacks raise NotImplementedError( NotImplementedError: Haven't coded Task callback yet - https://github.com/apache/airflow/issues/44354! ``` I implemented and tested the same changes in Astro Runtime 13.0.0 (Airflow 2.11.0), and saw that the failure callback executed successfully. Prior to my changes, the callback was not executed when tasks got stuck in queued and failed. I believe this discrepancy between the Runtime versions might be due to breaking changes introduced in Airflow 3. Therefore, we might have to implement different solutions for Airflow 2 vs 3. I will keep looking into this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
