[
https://issues.apache.org/jira/browse/BEAM-14130?focusedWorklogId=748929&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-748929
]
ASF GitHub Bot logged work on BEAM-14130:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 28/Mar/22 21:08
Start Date: 28/Mar/22 21:08
Worklog Time Spent: 10m
Work Description: VictorPlusC commented on a change in pull request
#17127:
URL: https://github.com/apache/beam/pull/17127#discussion_r836852439
##########
File path: sdks/python/apache_beam/runners/interactive/interactive_runner.py
##########
@@ -267,9 +267,12 @@ def _get_dataproc_cluster_master_url_if_applicable(
category=DeprecationWarning)
project_id =
(user_pipeline.options.view_as(GoogleCloudOptions).project)
region = (user_pipeline.options.view_as(GoogleCloudOptions).region)
- cluster_name = ie.current_env().clusters.default_cluster_name
- cluster_metadata = MasterURLIdentifier(
- project_id=project_id, region=region, cluster_name=cluster_name)
+ if not project_id:
+ cluster_metadata = ie.current_env().clusters.default_cluster_metadata
+ else:
+ cluster_name = ie.current_env().clusters.default_cluster_name
+ cluster_metadata = MasterURLIdentifier(
+ project_id=project_id, region=region, cluster_name=cluster_name)
Review comment:
This case was added in the scenario the user first uses the JupyterLab
extension to select a default cluster, at this point, we update
ib.clusters.default_cluster_metadata to be the MasterURLIdentifier that is used
by the selected cluster. Afterwards, if they create a new
InteractiveRunner(underlying_runner=FlinkRunner()) pipeline that does not have
a Flink master_url and also does not have a Google Cloud project specified, we
just assign the default Dataproc cluster to that pipeline.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 748929)
Time Spent: 21h (was: 20h 50m)
> Implement Jupyterlab extension for managing Dataproc clusters
> -------------------------------------------------------------
>
> Key: BEAM-14130
> URL: https://issues.apache.org/jira/browse/BEAM-14130
> Project: Beam
> Issue Type: New Feature
> Components: runner-py-interactive
> Reporter: Victor Chen
> Assignee: Victor Chen
> Priority: P2
> Time Spent: 21h
> Remaining Estimate: 0h
>
> - Adds an additional option under the Interactive Beam Jupyterlab extension
> to enable users to manage Interactive Beam clusters
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
