ashb commented on a change in pull request #6715: [AIRFLOW-5945] Make inbuilt
OperatorLinks work when using Serialization
URL: https://github.com/apache/airflow/pull/6715#discussion_r354280914
##########
File path: airflow/serialization/serialized_objects.py
##########
@@ -350,6 +368,53 @@ def _is_excluded(cls, var, attrname, op):
return True
return super()._is_excluded(var, attrname, op)
+ @classmethod
+ def _deserialize_operator_extra_links(
+ cls,
+ encoded_op_links: list
+ ) -> Dict[str, BaseOperatorLink]:
+ """
+ Deserialize Operator Links by importing all the modules containing
operator links
+ defined in the serialized dict and initialize them by passing
arguments using cattr.
+
+ :param encoded_op_links: Serialized Operator Link
+ :return: De-Serialized Operator Link
+ """
+
+ # Extra Operator Links
+ op_predefined_extra_links = {}
+
+ for _operator_links_source in encoded_op_links:
+ _operator_link_class, data =
list(_operator_links_source.items())[0]
+ try:
+ single_link = import_string(_operator_link_class)
+ op_predefined_extra_link: BaseOperatorLink =
cattr.structure(data, single_link)
+ op_predefined_extra_links.update(
+ {op_predefined_extra_link.name: op_predefined_extra_link}
+ )
+ except ImportError:
+ raise ImportError("Error Importing Operator Link:
{}".format(_operator_link_class))
+
+ return op_predefined_extra_links
+
+ @classmethod
+ def _serialize_operator_extra_links(
+ cls,
+ operator_extra_links: Iterable[BaseOperatorLink]
+ ):
+ """
+ Serialize Operator Links. Store the import path of the OperatorLink
and the arguments
+ passed to it. Example
``[{'airflow.gcp.operators.bigquery.BigQueryConsoleLink': {}}]``
+
+ :param operator_extra_links: Operator Link
+ :return: Serialized Operator Link
+ """
+ return [
+ {
+ "{}.{}".format(x.__class__.__module__, x.__class__.__name__):
cattr.unstructure(x)
Review comment:
I wonder if we need to pass the RHS of this through `cls.serialize_to_json`
-- otherwise more complex types than int/str/bool/dict/array can't be
serialized? Or should we just expect that cattr/attr has handled that for us?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services