dstandish commented on a change in pull request #19267:
URL: https://github.com/apache/airflow/pull/19267#discussion_r741608516
##########
File path: airflow/serialization/serialized_objects.py
##########
@@ -409,30 +406,59 @@ def _value_is_hardcoded_default(cls, attrname: str,
value: Any, instance: Any) -
return True
return False
+ @classmethod
+ def _serialize_param(cls, param: Param):
+ return dict(
+ __class=f"{param.__module__}.{param.__class__.__name__}",
+ default=cls._serialize(param.value),
+ description=cls._serialize(param.description),
Review comment:
So I think it ends up being the same because string is primitive type
##########
File path: airflow/serialization/serialized_objects.py
##########
@@ -409,30 +406,59 @@ def _value_is_hardcoded_default(cls, attrname: str,
value: Any, instance: Any) -
return True
return False
+ @classmethod
+ def _serialize_param(cls, param: Param):
+ return dict(
+ __class=f"{param.__module__}.{param.__class__.__name__}",
+ default=cls._serialize(param.value),
+ description=cls._serialize(param.description),
Review comment:
yeah just double checked and when it is primitive type, `_serialize`
doesn't do anything
though, maybe we should not serialize it so as not to accidentally support
something users should not be doing anyway. so on second thought, probably
should change this to put the raw value (in which case, json-serialization
will fail)
##########
File path: airflow/serialization/serialized_objects.py
##########
@@ -409,30 +406,59 @@ def _value_is_hardcoded_default(cls, attrname: str,
value: Any, instance: Any) -
return True
return False
+ @classmethod
+ def _serialize_param(cls, param: Param):
+ return dict(
+ __class=f"{param.__module__}.{param.__class__.__name__}",
+ default=cls._serialize(param.value),
+ description=cls._serialize(param.description),
Review comment:
ok just sat down to not `_serialize` the `description` attr. and on
second thought, i think it's better to run it through `_serialize` after all.
like, that's precisely what `_serialize` is there for. there's a reason that
`_serialize` has handling for primitive types, and so we should use it; to do
otherwise is essentially to duplicate serialization logic. not running through
`_serialize` would not prevent a user from using other json-serializable types
(only would prevent from using non-json-ser types like `set`) -- if we really
want to clamp down on types, probably the right place do do that is in
`Parame.__init__`.
so, i think i'll leave this as is, but let me know if you think otherwise.
thanks
##########
File path: airflow/serialization/serialized_objects.py
##########
@@ -409,30 +406,59 @@ def _value_is_hardcoded_default(cls, attrname: str,
value: Any, instance: Any) -
return True
return False
+ @classmethod
+ def _serialize_param(cls, param: Param):
+ return dict(
+ __class=f"{param.__module__}.{param.__class__.__name__}",
+ default=cls._serialize(param.value),
+ description=cls._serialize(param.description),
Review comment:
ok just sat down to not `_serialize` the `description` attr. and on
second thought, i think it's better to run it through `_serialize` after all.
like, that's precisely what `_serialize` is there for. there's a reason that
`_serialize` has handling for primitive types, and so we should use it; to do
otherwise is essentially to duplicate serialization logic. also, not running
through `_serialize` would not achieve the goal of preventing a user from using
non-string values (it only would prevent from using non-json-serializable types
such as `set`) -- if we really want to clamp down on types, probably the right
place do do that is in `Param.__init__`.
so, i think i'll leave this as is, but let me know if you think otherwise.
thanks
##########
File path: .pre-commit-config.yaml
##########
@@ -195,6 +195,15 @@ repos:
- "4"
files: ^chart/values\.schema\.json$|^chart/values_schema\.schema\.json$
pass_filenames: true
+ - id: pretty-format-json
+ name: Serialization schema
+ args:
+ - --autofix
+ - --no-sort-keys
+ - --indent
+ - "2"
Review comment:
yeah i'll split it out.
but yeah it was previously not processed in pre-commit. but it should be to
ensure consistent formatting, even if that means slightly less pretty.
and yeah i just chose indent of 2 because that's what the file already had.
##########
File path: airflow/serialization/serialized_objects.py
##########
@@ -325,7 +324,7 @@ def _serialize(cls, var: Any) -> Any: # Unfortunately
there is no support for r
elif isinstance(var, TaskGroup):
return SerializedTaskGroup.serialize_task_group(var)
elif isinstance(var, Param):
- return cls._encode(var.dump(), type_=DAT.PARAM)
+ return cls._encode(cls._serialize_param(var), type_=DAT.PARAM)
Review comment:
can you clarify what you mean @kaxil ? the serializer converts them to
json...
##########
File path: .pre-commit-config.yaml
##########
@@ -195,6 +195,15 @@ repos:
- "4"
files: ^chart/values\.schema\.json$|^chart/values_schema\.schema\.json$
pass_filenames: true
+ - id: pretty-format-json
+ name: Serialization schema
+ args:
+ - --autofix
+ - --no-sort-keys
+ - --indent
+ - "2"
Review comment:
ok updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
