GitHub user hanxdatadog created a discussion: XCom backend confusion for Operator Extra Links
Hi Folks. In our Airflow project we are utilizing the operator extra links feature to provide additional information following the instructions [here](https://airflow.apache.org/docs/apache-airflow/stable/howto/define-extra-link.html). We dynamically set the link values via xcom as the example and we are using a custom xcom backend (XComObjectStorageBackend). The issue we are seeing is, when the tasks are running, the links are resolved to the object storage key, but in the end they will be resolved correctly to the value. After digging into the source code, I realized: 1. when using XComObjectStorageBackend, the xcom db table, and the UI by default are only storing the object s3 path. But the xcom value in db for operator link is eventually resolved into the the value. This behavior is actually "specific" to the operator. 2. We are purposefully only using DB XCom backend for extra links. https://github.com/apache/airflow/pull/48227 and the value is resolved when a task is finished. 3. Although it is only meant to use DB backend, TaskInstance doesn't expose a public method to set xcom to db directly. https://github.com/apache/airflow/blob/main/task-sdk/src/airflow/sdk/execution_time/task_runner.py#L658. This forces us to use direct call to BaseXCom which is inconvenient. My question is, is there any specific reasons why we only use DB xcom backend for operator links? Any objections if I try to enable it for the configured custom backend? GitHub link: https://github.com/apache/airflow/discussions/59914 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
