uranusjr commented on code in PR #37058:
URL: https://github.com/apache/airflow/pull/37058#discussion_r1470830063
##########
airflow/config_templates/config.yml:
##########
@@ -375,6 +375,32 @@ core:
type: string
example: "path.to.CustomXCom"
default: "airflow.models.xcom.BaseXCom"
+ xcom_objectstorage_path:
Review Comment:
I wonder if this should be more general. Having two configs that only make
sense if `xcom_backend` is set to a specific backend seems awkward. I wonder if
we can make the backend take arguments directly… One quick idea is to provide a
backend _factory_ in Airflow instead, and tell the user to actually create the
backend class dynamically…
```python
# $AIRFLOW_HOME/plugins/xcom.py
CustomBackend = create_objectstorage_backend(
storage_path="s3://conn_id@bucket/path",
threshold=1_000_000,
)
```
```ini
# airflow.cfg
xcom_backend = xcom.CustomBackend
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
