iljau opened a new issue #19168:
URL: https://github.com/apache/airflow/issues/19168
### Description
Currently if container launched via KubernetesPodOperator returns with
non-zero exit code, whole pod configuration is printed to logs and this
multiple times due to exception re-raising. This makes logs unreadable.
There should be some method to avoid showing pod config json in case of
non-zero exit code.
```
[2021-10-22, 17:48:23 UTC] {{taskinstance.py:1686}} ERROR - Task failed with
exception
Traceback (most recent call last):
File
"/usr/local/lib/python3.8/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py",
line 369, in execute
raise AirflowException(f'Pod {self.pod.metadata.name} returned a
failure: {remote_pod}')
airflow.exceptions.AirflowException: Pod ... returned a failure: [ .. LONG
JSON HERE .. ]
```
Sample DAG:
```
import secrets
from datetime import datetime
from airflow import DAG
from airflow.configuration import conf
from airflow.providers.cncf.kubernetes.operators.kubernetes_pod import (
KubernetesPodOperator,
)
from kubernetes.client import models as k8s
default_args = {
"owner": "airflow",
"start_date": datetime(2021, 9, 1),
}
with DAG(
dag_id="test.pod_logs",
default_args=default_args,
schedule_interval=None,
max_active_runs=1,
) as dag:
namespace = conf.get("kubernetes", "namespace")
pod_name = "pod-test-pod-logs-%s" % secrets.token_urlsafe(8)
container_debug = k8s.V1Container(
name="ubuntu-container",
image="ubuntu:focal-20211006",
command=["/bin/bash", "-c"],
args=["""
set -x;
echo 1;
echo 2;
echo 3;
exit 1;
"""],
resources=k8s.V1ResourceRequirements(
limits={
"cpu": "1000m",
"memory": "1024Mi",
},
requests={
"cpu": "1000m",
"memory": "1024Mi",
},
)
)
pod = k8s.V1Pod(
api_version="v1",
kind="Pod",
spec=k8s.V1PodSpec(
image_pull_secrets=[k8s.V1LocalObjectReference('dockerhub-data')],
containers=[
container_debug,
],
),
metadata=k8s.V1ObjectMeta(
name=pod_name,
),
)
task_run_pod = KubernetesPodOperator(
task_id="task-pod-run",
name=pod_name,
namespace=namespace,
# is_delete_operator_pod=True,
is_delete_operator_pod=False,
in_cluster=True,
full_pod_spec=pod,
# image_pull_policy="Always",
image_pull_policy="IfNotPresent",
)
```
### Use case/motivation
_No response_
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
