[
https://issues.apache.org/jira/browse/BEAM-13812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victor Chen updated BEAM-13812:
-------------------------------
Fix Version/s: 2.38.0
(was: Not applicable)
> Integrate DataprocClusterManager into Interactive Environment
> -------------------------------------------------------------
>
> Key: BEAM-13812
> URL: https://issues.apache.org/jira/browse/BEAM-13812
> Project: Beam
> Issue Type: New Feature
> Components: runner-py-interactive
> Reporter: Victor Chen
> Assignee: Victor Chen
> Priority: P2
> Fix For: 2.38.0
>
> Time Spent: 29h 50m
> Remaining Estimate: 0h
>
> * Integrates the DataprocClusterManager previously introduced in:
> ** BEAM-13799 Create an Interactive Beam Dataproc package for users to
> manage clusters with
> * Adds code to automatically create a Dataproc cluster configured with
> Docker and Flink if a user instantiates an InteractiveRunner using
> underlying_runner='FlinkRunner'
> * By default, the created Dataproc cluster will be called
> 'interactive-beam-cluster' and will be cleaned up when the Interactive
> Environment is interrupted
> * Introduces the Clusters class in interactive_beam.py, which acts as a user
> interface that provides the user with the following capabilities:
> ** Changing the default name of the clusters created by Interactive Beam
> ** Cleaning up a cluster by providing a corresponding pipeline, an option to
> force the deletion is provided, in the case where more than one pipeline use
> the same cluster
> ** Getting all pipelines that use the provided master_url
> ** Receiving a description back for either the cluster corresponding to a
> specific pipeline, or the descriptions of all pipelines
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
