[
https://issues.apache.org/jira/browse/AIRFLOW-6166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cabell Maddux updated AIRFLOW-6166:
-----------------------------------
Description:
Hi,
I'm relatively new to airflow and am testing out Airflow running with the
Kubernetes executor locally on Minikube based off of the CI configs in the repo
([https://github.com/apache/airflow/tree/master/scripts/ci/kubernetes/app]).
Was able to get everything up and running smoothly, however thought default
behavior for xcom when no `do_xcom_push` argument is provided to a
`KubernetesPodOperator` was a bit confusing.
If a user doesn't explicitly create a result.json in the pod default
`XCOM_MOUNT_PATH` as part of the task pod, obviously the task will fail
([https://github.com/apache/airflow/blob/a36cfe049a2c5948b24fde7a878fe19cabede5f7/airflow/kubernetes/pod_launcher.py#L214)]
```
cat: can't open '/airflow/xcom/return.json': No such file or directory
```
Rather than the user having to explicitly create result.json as part of their
task, maybe more aligned with other operators to me to create
`\{XCOM_MOUNT_PATH}/result.json` when creating the xcom sidecar -
https://github.com/apache/airflow/blob/a36cfe049a2c5948b24fde7a878fe19cabede5f7/airflow/kubernetes/pod_generator.py#L36.
Otherwise, maybe would be nice to document that user of KubernetesPodOperator
should be careful to explicitly deal with `do_xcom_push` to avoid a bit of
confusion.
was:
Hi,
I'm relatively new to airflow and am testing out Airflow running with the
Kubernetes executor locally on Minikube based off of the CI configs in the repo
([https://github.com/apache/airflow/tree/master/scripts/ci/kubernetes/app]).
Was able to get everything up and running smoothly, however the default
behavior for xcom when no `do_xcom_push` argument is provided to a
`KubernetesPodOperator` is a bit confusing.
If a user doesn't explicitly create a result.json in the pod default
`XCOM_MOUNT_PATH` as part of the task pod, the task will fail
([https://github.com/apache/airflow/blob/a36cfe049a2c5948b24fde7a878fe19cabede5f7/airflow/kubernetes/pod_launcher.py#L214)]
```
```
Rather than the user having to explicitly create result.json as part of their
task, seems more aligned with other operators to me to create
`\{XCOM_MOUNT_PATH}/result.json` when creating the
> KubernetesPodOperator resolve default xcom read behavior or improve
> associated documentation
> --------------------------------------------------------------------------------------------
>
> Key: AIRFLOW-6166
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6166
> Project: Apache Airflow
> Issue Type: Improvement
> Components: documentation, executor-kubernetes, xcom
> Affects Versions: 1.10.7
> Reporter: Cabell Maddux
> Assignee: Daniel Imberman
> Priority: Minor
>
> Hi,
> I'm relatively new to airflow and am testing out Airflow running with the
> Kubernetes executor locally on Minikube based off of the CI configs in the
> repo
> ([https://github.com/apache/airflow/tree/master/scripts/ci/kubernetes/app]).
> Was able to get everything up and running smoothly, however thought default
> behavior for xcom when no `do_xcom_push` argument is provided to a
> `KubernetesPodOperator` was a bit confusing.
> If a user doesn't explicitly create a result.json in the pod default
> `XCOM_MOUNT_PATH` as part of the task pod, obviously the task will fail
> ([https://github.com/apache/airflow/blob/a36cfe049a2c5948b24fde7a878fe19cabede5f7/airflow/kubernetes/pod_launcher.py#L214)]
> ```
> cat: can't open '/airflow/xcom/return.json': No such file or directory
> ```
> Rather than the user having to explicitly create result.json as part of their
> task, maybe more aligned with other operators to me to create
> `\{XCOM_MOUNT_PATH}/result.json` when creating the xcom sidecar -
> https://github.com/apache/airflow/blob/a36cfe049a2c5948b24fde7a878fe19cabede5f7/airflow/kubernetes/pod_generator.py#L36.
> Otherwise, maybe would be nice to document that user of KubernetesPodOperator
> should be careful to explicitly deal with `do_xcom_push` to avoid a bit of
> confusion.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
