potiuk commented on code in PR #36205:
URL: https://github.com/apache/airflow/pull/36205#discussion_r1433985142
##########
airflow/providers/databricks/hooks/databricks_sql.py:
##########
@@ -241,13 +267,23 @@ def run(
else:
return results
- @staticmethod
- def _make_serializable(result):
- """Transform the databricks Row objects into JSON-serializable
lists."""
- if isinstance(result, list):
- return [list(row) for row in result]
- elif isinstance(result, Row):
- return list(result)
+ def _make_common_data_structure(
+ self, result: list[T] | T | None
+ ) -> list[T] | T | list[tuple] | tuple | None:
+ """Transform the databricks Row objects into namedtuple."""
+ # Below ignored lines respect namedtuple docstring, but mypy do not
support dynamically
+ # instantiated namedtuple, and will never do:
https://github.com/python/mypy/issues/848
+ if self.return_tuple:
+ if isinstance(result, list) and all(isinstance(item, Row) for item
in result):
Review Comment:
What would be your suggestion here @bolkedebruin ? I guess it is not
blocking but simple optimisation suggestion? If so - is that fine if we
separate it as a follow up issue to optimize it? I guess this is not something
that BLOCKS this pr?
I think we can definitely optimise it in the future, however, again -
perfect is the enemy of good and it's holding back release of a new waves of
providers . My suggestion is to get it out now and separate it as a follow-up
PR.
Is that OK @bolkedebruin ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
