jscheffl opened a new issue, #49226: URL: https://github.com/apache/airflow/issues/49226
### Apache Airflow version main (development) ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? I tested Airflow 3 / main with the "integration test" DAG from Edge Executor (providers/edge/src/airflow/providers/edge/example_dags/integration_test.py) and when using latest main == cb295c351a016c0a10cab07f2a628b865cff3ca3 the task `connection` fails in line 109. Code is expecting an `AirflowNotFoundException` as the connection is not existing but when using CeleryExecutor a `ValueError` is raised with the following stack: ``` [2025-04-14, 13:49:27] ERROR - Task failed with exception: source="task" ValueError: I/O operation on closed file. File "/opt/airflow/task-sdk/src/airflow/sdk/execution_time/task_runner.py", line 809 in run File "/opt/airflow/task-sdk/src/airflow/sdk/execution_time/task_runner.py", line 1072 in _execute_task File "/opt/airflow/task-sdk/src/airflow/sdk/bases/operator.py", line 408 in wrapper File "/opt/airflow/task-sdk/src/airflow/sdk/bases/decorator.py", line 251 in execute File "/opt/airflow/task-sdk/src/airflow/sdk/bases/operator.py", line 408 in wrapper File "/opt/airflow/providers/standard/src/airflow/providers/standard/operators/python.py", line 212 in execute File "/opt/airflow/providers/standard/src/airflow/providers/standard/operators/python.py", line 235 in execute_callable File "/opt/airflow/task-sdk/src/airflow/sdk/execution_time/callback_runner.py", line 81 in run File "/files/dags/integration_test.py", line 109 in connection File "/opt/airflow/airflow-core/src/airflow/hooks/base.py", line 64 in get_connection File "/opt/airflow/airflow-core/src/airflow/models/connection.py", line 472 in get_connection_from_secrets File "/opt/airflow/task-sdk/src/airflow/sdk/definitions/connection.py", line 142 in get File "/opt/airflow/task-sdk/src/airflow/sdk/execution_time/context.py", line 127 in _get_connection File "/opt/airflow/task-sdk/src/airflow/sdk/execution_time/supervisor.py", line 1182 in ensure_secrets_backend_loaded File "/usr/local/lib/python3.12/site-packages/structlog/_native.py", line 144 in meth File "/usr/local/lib/python3.12/site-packages/structlog/_base.py", line 223 in _proxy_to_logger File "/usr/local/lib/python3.12/site-packages/structlog/_output.py", line 217 in msg ``` Note: Same is working as expected with LocalExecutor and EdgeExecutor. Made a retry and problem seems to be persistent. ### What you think should happen instead? CeleryExecutor should behave like LocalExecutor ### How to reproduce Copy `providers/edge/src/airflow/providers/edge/example_dags/integration_test.py` to `files/dags/` and use `breeze start-airflow --python 3.12 --load-example-dags --backend postgres --executor CeleryExecutor --answer y`. THen start the "Integration Test" DAG and see connection task fail. If you start with `LocalExecutor` you see the correct exception being catched. ### Operating System Ubuntu 24.04 ### Versions of Apache Airflow Providers n/a ### Deployment Other ### Deployment details Via breeze ### Anything else? Setting 3.0.0rc label as I assume it also happens on RC. ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
