jolin1337 commented on code in PR #61777:
URL: https://github.com/apache/airflow/pull/61777#discussion_r2802672351
##########
providers/common/io/src/airflow/providers/common/io/xcom/backend.py:
##########
@@ -117,9 +117,13 @@ def serialize_value( # type: ignore[override]
run_id: str | None = None,
map_index: int | None = None,
) -> bytes | str:
- # We will use this serialized value to write to the object store.
- s_val = json.dumps(value, cls=XComEncoder)
- s_val_encoded = s_val.encode("utf-8")
+ if isinstance(value, bytes):
+ # Store raw bytes as-is
+ s_val_encoded = value
Review Comment:
Would you say this is the recommended and perfered way of storing raw bytes
data? to encode and decode on each task_run? Seams to me that this will add
unnecessary overhead but that is just my opinion. Does this require some sort
of discussion before implemented in to the common.io provider perhaps? In case
more people would like to see this feature it might be good idea to implement
or if it is only us then we could solve it as you suggests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
