amoghrajesh commented on code in PR #45448:
URL: https://github.com/apache/airflow/pull/45448#discussion_r1904990716
##########
task_sdk/src/airflow/sdk/definitions/connection.py:
##########
@@ -50,3 +55,32 @@ class Connection:
def get_uri(self): ...
def get_hook(self): ...
+
+ def _get_extra_dejson(self, nested: bool = False) -> dict:
+ """
+ Deserialize extra property to JSON.
+
+ :param nested: Determines whether nested structures are also
deserialized into JSON (default False).
+ """
+ extra = {}
+ log = structlog.get_logger(logger_name="task")
+
+ if self.extra:
+ try:
+ if nested:
+ for key, value in json.loads(self.extra).items():
+ extra[key] = value
+ if isinstance(value, str):
+ with suppress(JSONDecodeError):
+ extra[key] = json.loads(value)
+ else:
+ extra = json.loads(self.extra)
+ except JSONDecodeError:
+ log.error("Failed to deserialize extra property `extra`,
returning empty dictionary")
Review Comment:
We don't want to raise an exception, so a safe fallback would be to return
empty dict with an error log.
##########
task_sdk/src/airflow/sdk/definitions/connection.py:
##########
@@ -50,3 +55,32 @@ class Connection:
def get_uri(self): ...
def get_hook(self): ...
+
+ def _get_extra_dejson(self, nested: bool = False) -> dict:
+ """
+ Deserialize extra property to JSON.
+
+ :param nested: Determines whether nested structures are also
deserialized into JSON (default False).
+ """
+ extra = {}
+ log = structlog.get_logger(logger_name="task")
+
+ if self.extra:
+ try:
+ if nested:
+ for key, value in json.loads(self.extra).items():
+ extra[key] = value
+ if isinstance(value, str):
+ with suppress(JSONDecodeError):
+ extra[key] = json.loads(value)
+ else:
+ extra = json.loads(self.extra)
+ except JSONDecodeError:
+ log.error("Failed to deserialize extra property `extra`,
returning empty dictionary")
+ # Mask sensitive stuff here
Review Comment:
Do we really need to keep this? What plans to migrate masking, will it be in
the server or task sdk?
##########
task_sdk/src/airflow/sdk/definitions/connection.py:
##########
@@ -50,3 +55,32 @@ class Connection:
def get_uri(self): ...
def get_hook(self): ...
+
+ def _get_extra_dejson(self, nested: bool = False) -> dict:
Review Comment:
Ported mainly from
https://github.com/apache/airflow/blob/a6da8df0d7bb1958b7e3ab558e06d16ce72b5327/airflow/models/connection.py#L413
with some minor changes.
The `nested` parameter is never used at all in legacy code. Something worth
nuking and building some intelligence to check for `nested` jsons without a
parameter is a direction I am favouring, thoughts @kaxil @ashb?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
