tnyz opened a new issue #22318:
URL: https://github.com/apache/airflow/issues/22318
### Apache Airflow version
2.2.4 (latest released)
### What happened
When the main container errors and failed to write a return.json file, the
xcom sidecar hangs and doesn't exit properly with an empty return.json.
This is a problem because we want to suppress the following error, as the
reason the pod failed should not be that xcom failed.
```
[2022-03-16, 17:08:07 UTC] {pod_manager.py:342} INFO - Running command...
cat /airflow/xcom/return.json
[2022-03-16, 17:08:07 UTC] {pod_manager.py:349} INFO - stderr from command:
cat: can't open '/airflow/xcom/return.json': No such file or directory
[2022-03-16, 17:08:07 UTC] {pod_manager.py:342} INFO - Running command...
kill -s SIGINT 1
[2022-03-16, 17:08:08 UTC] {kubernetes_pod.py:417} INFO - Deleting pod:
test.20882a4c607d418d94e87231214d34c0
[2022-03-16, 17:08:08 UTC] {taskinstance.py:1718} ERROR - Task failed with
exception
Traceback (most recent call last):
File
"/usr/local/lib/python3.8/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py",
line 385, in execute
result = self.extract_xcom(pod=self.pod)
File
"/usr/local/lib/python3.8/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py",
line 360, in extract_xcom
result = self.pod_manager.extract_xcom(pod)
File
"/usr/local/lib/python3.8/site-packages/airflow/providers/cncf/kubernetes/utils/pod_manager.py",
line 337, in extract_xcom
raise AirflowException(f'Failed to extract xcom from pod:
{pod.metadata.name}')
airflow.exceptions.AirflowException: Failed to extract xcom from pod:
test.20882a4c607d418d94e87231214d34c0
```
and have the KubernetesPodOperator exit gracefully
### What you think should happen instead
sidecar should exit with an empty xcom return value
### How to reproduce
KubernetesPodOperator with command `mkdir -p /airflow/xcom;touch
/airflow/xcom/return.json; cat a >> /airflow/xcom/return.json`
### Operating System
-
### Versions of Apache Airflow Providers
_No response_
### Deployment
Other Docker-based deployment
### Deployment details
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
