Seokyun-Ha commented on code in PR #34071:
URL: https://github.com/apache/airflow/pull/34071#discussion_r1333986359
##########
airflow/providers/databricks/hooks/databricks.py:
##########
@@ -490,6 +567,41 @@ def start_cluster(self, json: dict) -> None:
"""
self._do_api_call(START_CLUSTER_ENDPOINT, json)
+ def activate_cluster(self, json: dict, polling: int, timeout: int | None =
None) -> None:
+ """
+ Start the cluster, and wait for it to be ready.
+
+ :param json: json dictionary containing cluster specification.
+ :param polling: polling interval in seconds.
+ :param timeout: timeout in seconds.
+ """
+ cluster_id = json["cluster_id"]
+
+ api_called = False
+ time_start = time.time()
+
+ while True:
Review Comment:
Yes, maybe. I now understand your suggestion. I think
calling`active_cluster()` method without `polling` and `timeout` is same as
calling `start_cluster()`. So, if I change the code it looks like this
```py
def start_cluster(self, json: dict, polling: int | None = None, timeout: int
| None = None) -> None:
cluster_id = json["cluster_id"]
api_called = False
time_start = time.time()
while True:
run_state = self.get_cluster_state(cluster_id)
if run_state.is_running:
return
elif run_state.is_terminal:
if api_called:
raise AirflowException(
f"Cluster {cluster_id} start failed with
'{run_state.state}' "
f"state: {run_state.state_message}"
)
# This part changed
# self.start_cluster(json)
self._do_api_call(START_CLUSTER_ENDPOINT, json)
api_called = True
# This part changed
if polling:
# wait for cluster to start
time.sleep(polling)
else:
return
elapsed_time = time.time() - time_start
if timeout and elapsed_time > timeout:
raise AirflowException(f"Cluster {cluster_id} start timed
out after {timeout} seconds")
```
Please, take a look and get some opinions 🙏
\+ I think then we can apply this to `restart_cluster()` method too!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
