ewewukek opened a new issue, #53147: URL: https://github.com/apache/airflow/issues/53147
### Apache Airflow version 3.0.2 ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? happens when the worker process is terminated due to timeout. manager closes it's log file but when reading remaining data from logging pipe tries to write to the log which causes `ValueError: write to closed file`. log file opened: [airflow/dag_processing/manager.py#L871](https://github.com/apache/airflow/blob/3fa68e670f608973133aeabba2c99988110b39d0/airflow-core/src/airflow/dag_processing/manager.py#L871) log file closed: [airflow/dag_processing/manager.py#L1043](https://github.com/apache/airflow/blob/3fa68e670f608973133aeabba2c99988110b39d0/airflow-core/src/airflow/dag_processing/manager.py#L1043) write attempt: [airflow/dag_processing/manager.py#L398](https://github.com/apache/airflow/blob/3fa68e670f608973133aeabba2c99988110b39d0/airflow-core/src/airflow/dag_processing/manager.py#L398) [airflow/sdk/execution_time/supervisor.py#L1583](https://github.com/apache/airflow/blob/3fa68e670f608973133aeabba2c99988110b39d0/task-sdk/src/airflow/sdk/execution_time/supervisor.py#L1583) ### What you think should happen instead? simplest solution probably would be not closing the log file, letting reader write last bits of data to the log file. it will get closed anyway when the pipe closes. (unless i misunderstand how the selector works) ### How to reproduce have a dag killed by a timeout. it's a race condition so it may not reproduce every time ### Operating System Debian GNU/Linux 12 (bookworm) ### Versions of Apache Airflow Providers _No response_ ### Deployment Virtualenv installation ### Deployment details _No response_ ### Anything else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
