GURUPRASAD SRIDHARAN created AIRFLOW-6355:
---------------------------------------------
Summary: Tasks on dags spawned with user impersonation don't get
killed on marking them as FAILED
Key: AIRFLOW-6355
URL: https://issues.apache.org/jira/browse/AIRFLOW-6355
Project: Apache Airflow
Issue Type: Bug
Components: core
Affects Versions: 1.10.6
Environment: Linux Ubuntu 16.04
Reporter: GURUPRASAD SRIDHARAN
Assignee: GURUPRASAD SRIDHARAN
When running airflow with python version 3.7, we encountered this issue wherein
tasks marked FAILED from the webserver UI don't get killed even though SIGTERM
is passed.
This issue arises in dags run with user impersonation. The current behaviour in
helpers.py is to try and kill all processes with a particular group id. If
there is a permission error, then in except block an attempt is made to kill
the group of processes as sudo user.
{code:java}
if err.errno == errno.EPERM:
subprocess.check_call( ["sudo", "-n", "kill", "-" + str(sig)] +
map(children, lambda p: str(p.pid)) )
{code}
The command run via the subprocess call doesn't work with python 3.7 (I am not
sure about other versions). In order to make this command compatible with both
python2 and 3, the code should be changed as follows
{code:java}
if err.errno == errno.EPERM:
subprocess.check_call( ["sudo", "-n", "kill", "-" + str(int(sig))] +
map(children, lambda p: str(p.pid)) )
{code}
Along with this change, I am proposing some minor changes to log the exceptions
so that permission errors become more readily apparent in the task run logs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
