amoghrajesh commented on PR #46719: URL: https://github.com/apache/airflow/pull/46719#issuecomment-2661058424
> > Impact on custom XCOM backends. > > Flow: > > > TLDR: Custom XCOM backends store the "path" to the xcom object remotely stored in the DB as a JSON type, but store the data at that "path" without serialisation (as JSON objects). So the interface to interact with this backend is in JSON objects, which is what the new proposed flow above does precisely. > > Not always, example `XComObjectStorageBackend` only stores value in a remote object if it is greater than a certain threshold, for others it serializes the objects into string using `json.dumps` > > https://github.com/apache/airflow/blob/8f63b828ace09b1095229f22b0c6d1f0f85ea81a/providers/common/io/src/airflow/providers/common/io/xcom/backend.py#L116-L118 > > https://github.com/apache/airflow/blob/8f63b828ace09b1095229f22b0c6d1f0f85ea81a/providers/common/io/src/airflow/providers/common/io/xcom/backend.py#L126-L133 Thanks! So, yeah, thats true. We still are keeping the same behaviour even with this change, for smaller xcom values, we store in DB in a serialised manner, and for larger data, we store the path in a serialised manner. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
