ashb commented on a change in pull request #16554:
URL: https://github.com/apache/airflow/pull/16554#discussion_r656022723
##########
File path: airflow/serialization/serialized_objects.py
##########
@@ -186,13 +186,18 @@ def serialize_to_json(
if cls._is_excluded(value, key, object_to_serialize):
continue
- if key in decorated_fields:
- serialized_object[key] = cls._serialize(value)
- else:
- value = cls._serialize(value)
- if isinstance(value, dict) and "__type" in value:
+ value = cls._serialize(value)
+ if key not in decorated_fields and isinstance(value, dict) and
"__type" in value:
+ # Extra check to make sure for custom operators or dags that
the non-standard
+ # serialized_fields still support non-primitive types (e.g.
dicts).
+ if isinstance(object_to_serialize, DAG) and key in
DAG.get_serialized_fields():
+ value = value["__var"]
+ elif (
+ isinstance(object_to_serialize, BaseOperator)
+ and key in BaseOperator.get_serialized_fields()
+ ):
Review comment:
> why we are not always surrounding complex types in a dict with __var
and __type keys
Simply put: to reduce the size of the output JSON -- in testing of large
DAGs this made a surprising difference to the size of the output JSON, and it
was for cases where we _know_ the type of a column so we don't have to decorate
it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
