GitHub user kiranzo added a comment to the discussion: How to setup Airflow to use local file system for XComs?
[Okay, tl;dr](https://stackoverflow.com/a/72343078) - when I used bind mounted volume, `docker compose` took over as a `root` user and overwrote whatever definitions I made in the Docker file, so it had permissions 755, and `airflow` user wasn't allowed to write there. I solved it by turning my XComs directory into a named volume, so the way I created the XCom directory remained the same during the container creation. My Docker file ended up looking like this: ``` ### extended_airflow:1.6.0 FROM apache/airflow:2.10.5-python3.10 COPY requirements.txt /requirements.txt RUN pip install --upgrade pip RUN pip install "cython<3.0.0" wheel RUN pip install --no-cache-dir -r /requirements.txt USER root # apt install extra utils here RUN rm /requirements.txt RUN mkdir -m 777 /opt/airflow/xcoms RUN chown -R airflow:0 /opt/airflow/xcoms USER airflow ``` And my [docker-compose from the official documentation](https://airflow.apache.org/docs/apache-airflow/2.10.5/docker-compose.yaml) had these changes: ``` x-airflow-common: ... environment: AIRFLOW__CORE__XCOM_BACKEND: 'airflow.providers.common.io.xcom.backend.XComObjectStorageBackend' AIRFLOW__COMMON_IO__XCOM_OBJECTSTORAGE_PATH: 'file:///opt/airflow/xcoms' AIRFLOW__COMMON_IO__XCOM_OBJECTSTORAGE_THRESHOLD: 100000000 AIRFLOW__COMMON_IO__XCOM_OBJECTSTORAGE_COMPRESSION: 'gzip' ... volumes: - ./dags:/opt/airflow/dags - ./logs:/opt/airflow/logs - xcoms:/opt/airflow/xcoms ... volumes: airflow-cache-volume: airflow-protected-volume: postgres-db-volume: git-sync-volume: xcoms: ``` GitHub link: https://github.com/apache/airflow/discussions/50926#discussioncomment-13249843 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
