uranusjr commented on a change in pull request #19736:
URL: https://github.com/apache/airflow/pull/19736#discussion_r754981584
##########
File path: airflow/providers/databricks/hooks/databricks.py
##########
@@ -493,3 +504,120 @@ def __init__(self, token: str) -> None:
def __call__(self, r: PreparedRequest) -> PreparedRequest:
r.headers['Authorization'] = 'Bearer ' + self.token
return r
+
+
+class DatabricksAsyncHook(DatabricksHook):
+ """
+ Async version of the ``DatabricksHook``
+ Implements only necessary methods used further in Databricks Triggers.
+ """
+
+ def __init__(self, *args: Any, **kwargs: Any) -> None:
+ super().__init__(*args, **kwargs)
+
+ async def __aenter__(self):
+ self._session = aiohttp.ClientSession()
Review comment:
> > you must move the abstraction to the trigger instead.
>
> My understanding is that when there're multiple `triggerer` processes,
each trigger will be executed as a separate async task with its own
`ClientSession` instance. So even if we move it to
`DatabricksExecutionTrigger`, it'd still create a session for each trigger run.
Sorry, I think I meant to say trigger**er** instead of trigger 🙂
Your general summary is correct though; I don’t think it’s particularly
worthwhile to make that optimisation right now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
