Yingbo Wang created AIRFLOW-6874:
------------------------------------
Summary: There are risks that subprocesses not killed when a task
failed
Key: AIRFLOW-6874
URL: https://issues.apache.org/jira/browse/AIRFLOW-6874
Project: Apache Airflow
Issue Type: Improvement
Components: worker
Affects Versions: 1.10.4
Reporter: Yingbo Wang
Assignee: Yingbo Wang
Many airflow tasks create subprocesses and these subprocesses may create more
subprocesses. In our experience, there is a risk that although a task failed
and tried to reap the process group, there are still left over processes
running and cause issues with both resources and correctness.
Propose to improve the cgroup task runner to reap all processes for current
node on node termination.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
