aichemzee commented on issue #39184: URL: https://github.com/apache/airflow/issues/39184#issuecomment-2589149626
> If you need to use xcom push, just make the init container yourself in the spark operator template (based on https://github.com/apache/airflow/blob/main/airflow/providers/cncf/kubernetes/utils/xcom_sidecar.py) > > ``` > volumes: > - name: xcom > emptyDir: {} > driver: > volumeMounts: > - name: xcom > mountPath: /airflow/xcom > sidecars: > - name: airflow-xcom-sidecar > image: alpine > command: ["sh", "-c", 'trap "exit 0" INT; while true; do sleep 1; done;'] > volumeMounts: > - name: xcom > mountPath: /airflow/xcom > resources: > requests: > cpu: "1m" > memory: "10Mi" > ``` @creyer I tried this however it still stays stuck for xcom push, do we have to enable something else in the spark operator for this to work? Did you add this in initContainers section or sidecars like you have it above? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
