ashb opened a new pull request #11278: URL: https://github.com/apache/airflow/pull/11278
Since these processes are spawned from SchedulerJob after it has registered it's signals, if any of them got signaled they would have the behaviour of killing the ProcessorAgent process group! (MP has a default spawn of fork on Linux, so they inherit all previous state -- signals, and access to the `_process.pid` inside the ProcessorAgent instance) This behaviour is not what we want for these multiprocessing.Pool tasks. --- Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more information. In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
