[
https://issues.apache.org/jira/browse/AIRFLOW-6782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maksym Shalak updated AIRFLOW-6782:
-----------------------------------
Description:
{color:#172b4d}I launch tasks using KubernetesPodOperator with
do_xcom_push=True{color}
{color:#172b4d}This creates my container with an airflow-xcom-sidecar container
inside a pod. {color}
{color:#172b4d}In 90% of cases that works fine, but sometimes my main container
completes, but airflow-xcom-sidecar is still running and never completes, which
prevents my pod from completion. {color}
{color:#172b4d}I see nothing in airflow-xcom-sidecar co{color}ntainer logs. The
data which I want to write is inside /airflow/xcom/return.json, so no bug there.
{color:#172b4d}Looks like this issue is common:
[https://stackoverflow.com/questions/54388441/kubernetes-pod-created-through-airflow-remains-in-running-state]
{color}
{color:#172b4d}Let me know if any additional information needed.{color}
Result of *kubectl describe pod*:
{code:java}
Containers:
base:
Container ID:
docker://e50ffc8e47ca6b75176732a4eb5dee6bd9d7decbf7c272acbd2dadc94034a807
...
State: Terminated
Reason: Completed
Exit Code: 0
Started: Wed, 12 Feb 2020 14:27:23 +0200
Finished: Wed, 12 Feb 2020 14:28:19 +0200
Ready: False
Restart Count: 0
Requests:
cpu: 300m
memory: 500Mi
Environment: <none>
Mounts:
/airflow/xcom from xcom (rw)
airflow-xcom-sidecar:
Container ID:
docker://f5a1a4d14bf1820387f5ed21541b4de3b5f89d067e4f91d18909d9a266c26696
Image: alpine
Image ID:
docker-pullable://alpine@sha256:ddba4d27a7ffc3f86dd6c2f92041af252a1f23a8e742c90e6e1297bfa1bc0c45
Port: <none>
Host Port: <none>
Command:
sh
-c
trap "exit 0" INT; while true; do sleep 30; done;
State: Running
Started: Wed, 12 Feb 2020 14:27:24 +0200
Ready: True
Restart Count: 0
Requests:
cpu: 1m
Environment: <none>
Mounts:
/airflow/xcom from xcom (rw)
{code}
was:
{color:#172b4d}I launch tasks using KubernetesPodOperator with
do_xcom_push=True{color}
{color:#172b4d}This creates my container with an airflow-xcom-sidecar container
inside a pod. {color}
{color:#172b4d}In 90% of cases that works fine, but sometimes my main container
completes, but airflow-xcom-sidecar is still running and never completes, which
prevents my pod from completion. {color}
{color:#172b4d}I see nothing in airflow-xcom-sidecar co{color}ntainer logs. The
data which I want to write is inside /airflow/xcom/return.json, so no bug there.
{color:#172b4d}Looks like this issue is common:
[https://stackoverflow.com/questions/54388441/kubernetes-pod-created-through-airflow-remains-in-running-state]
{color}
{color:#172b4d}Let me know if any additional information needed.{color}
Result of kubectl describe pod:
{code:java}
Containers:
base:
Container ID:
docker://e50ffc8e47ca6b75176732a4eb5dee6bd9d7decbf7c272acbd2dadc94034a807
...
State: Terminated
Reason: Completed
Exit Code: 0
Started: Wed, 12 Feb 2020 14:27:23 +0200
Finished: Wed, 12 Feb 2020 14:28:19 +0200
Ready: False
Restart Count: 0
Requests:
cpu: 300m
memory: 500Mi
Environment: <none>
Mounts:
/airflow/xcom from xcom (rw)
airflow-xcom-sidecar:
Container ID:
docker://f5a1a4d14bf1820387f5ed21541b4de3b5f89d067e4f91d18909d9a266c26696
Image: alpine
Image ID:
docker-pullable://alpine@sha256:ddba4d27a7ffc3f86dd6c2f92041af252a1f23a8e742c90e6e1297bfa1bc0c45
Port: <none>
Host Port: <none>
Command:
sh
-c
trap "exit 0" INT; while true; do sleep 30; done;
State: Running
Started: Wed, 12 Feb 2020 14:27:24 +0200
Ready: True
Restart Count: 0
Requests:
cpu: 1m
Environment: <none>
Mounts:
/airflow/xcom from xcom (rw)
{code}
> airflow-xcom-sidecar container prevents pods from completion
> ------------------------------------------------------------
>
> Key: AIRFLOW-6782
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6782
> Project: Apache Airflow
> Issue Type: Bug
> Components: xcom
> Affects Versions: 1.10.7
> Reporter: Maksym Shalak
> Priority: Major
> Attachments: Screenshot from 2020-02-12 16-19-49.png
>
>
> {color:#172b4d}I launch tasks using KubernetesPodOperator with
> do_xcom_push=True{color}
> {color:#172b4d}This creates my container with an airflow-xcom-sidecar
> container inside a pod. {color}
> {color:#172b4d}In 90% of cases that works fine, but sometimes my main
> container completes, but airflow-xcom-sidecar is still running and never
> completes, which prevents my pod from completion. {color}
> {color:#172b4d}I see nothing in airflow-xcom-sidecar co{color}ntainer logs.
> The data which I want to write is inside /airflow/xcom/return.json, so no bug
> there.
> {color:#172b4d}Looks like this issue is common:
> [https://stackoverflow.com/questions/54388441/kubernetes-pod-created-through-airflow-remains-in-running-state]
> {color}
> {color:#172b4d}Let me know if any additional information needed.{color}
>
> Result of *kubectl describe pod*:
> {code:java}
> Containers:
> base:
> Container ID:
> docker://e50ffc8e47ca6b75176732a4eb5dee6bd9d7decbf7c272acbd2dadc94034a807
> ...
> State: Terminated
> Reason: Completed
> Exit Code: 0
> Started: Wed, 12 Feb 2020 14:27:23 +0200
> Finished: Wed, 12 Feb 2020 14:28:19 +0200
> Ready: False
> Restart Count: 0
> Requests:
> cpu: 300m
> memory: 500Mi
> Environment: <none>
> Mounts:
> /airflow/xcom from xcom (rw)
> airflow-xcom-sidecar:
> Container ID:
> docker://f5a1a4d14bf1820387f5ed21541b4de3b5f89d067e4f91d18909d9a266c26696
> Image: alpine
> Image ID:
> docker-pullable://alpine@sha256:ddba4d27a7ffc3f86dd6c2f92041af252a1f23a8e742c90e6e1297bfa1bc0c45
> Port: <none>
> Host Port: <none>
> Command:
> sh
> -c
> trap "exit 0" INT; while true; do sleep 30; done;
> State: Running
> Started: Wed, 12 Feb 2020 14:27:24 +0200
> Ready: True
> Restart Count: 0
> Requests:
> cpu: 1m
> Environment: <none>
> Mounts:
> /airflow/xcom from xcom (rw)
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
