sodafountain opened a new issue #16573:
URL: https://github.com/apache/airflow/issues/16573
**Apache Airflow version**: 2.0.1
**Kubernetes version (if you are using kubernetes)** (use `kubectl
version`): 1.18.14
Environment:
Cloud provider or hardware configuration: Azure
OS (e.g. from /etc/os-release):
Kernel (e.g. uname -a):
Install tools:
Others:
**What happened**:
An occasional airflow tasks fails with the following error
```
[2021-06-21 05:39:48,424] {local_task_job.py:184} WARNING - State of this
instance has been externally set to up_for_retry. Terminating instance.
[2021-06-21 05:39:48,425] {process_utils.py:100} INFO - Sending
Signals.SIGTERM to GPID 259
[2021-06-21 05:39:48,426] {taskinstance.py:1238} ERROR - Received SIGTERM.
Terminating subprocesses.
[2021-06-21 05:39:48,426] {bash.py:185} INFO - Sending SIGTERM signal to
bash process group
[2021-06-21 05:39:49,133] {process_utils.py:66} INFO - Process
psutil.Process(pid=329, status='terminated', started='04:32:14') (329)
terminated with exit code None
[2021-06-21 05:39:50,278] {taskinstance.py:1454} ERROR - Task received
SIGTERM signal
Traceback (most recent call last):
File
"/usr/local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line
1112, in _run_raw_task
self._prepare_and_execute_task_with_callbacks(context, task)
File
"/usr/local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line
1284, in _prepare_and_execute_task_with_callbacks
result = self._execute_task(context, task_copy)
File
"/usr/local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line
1309, in _execute_task
result = task_copy.execute(context=context)
File "/usr/local/lib/python3.7/site-packages/airflow/operators/bash.py",
line 171, in execute
for raw_line in iter(self.sub_process.stdout.readline, b''):
File
"/usr/local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line
1240, in signal_handler
raise AirflowException("Task received SIGTERM signal")
airflow.exceptions.AirflowException: Task received SIGTERM signal
```
There is no indication as to what caused this error. The worker instance is
healthy and task did not hit the task timeout.
**What you expected to happen**:
Task to complete successfully. If a task fad to fail for unavoidable reason
(like timeout), it would be helpful to provide the reason for the failure.
**How to reproduce it**:
I'm not able to reproduce it consistently. It happens every now and then
with the same error as provided above.
I'm also wish to know how to debug these failures
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
