[
https://issues.apache.org/jira/browse/BEAM-7872?focusedWorklogId=288868&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-288868
]
ASF GitHub Bot logged work on BEAM-7872:
----------------------------------------
Author: ASF GitHub Bot
Created on: 05/Aug/19 11:10
Start Date: 05/Aug/19 11:10
Worklog Time Spent: 10m
Work Description: lgajowy commented on pull request #9213: [BEAM-7872]
Simpler Flink cluster set up in load tests
URL: https://github.com/apache/beam/pull/9213#discussion_r310551498
##########
File path: .test-infra/dataproc/flink_cluster.sh
##########
@@ -137,12 +134,29 @@ function create_cluster() {
gcloud dataproc clusters create $CLUSTER_NAME
--num-workers=$num_dataproc_workers --initialization-actions
$DOCKER_INIT,$BEAM_INIT,$FLINK_INIT --metadata "${metadata}",
--image-version=$image_version --zone=$GCLOUD_ZONE --quiet
}
-function main() {
+# Runs init actions for Docker, Portability framework (Beam) and Flink cluster
+# and opens an SSH tunnel to connect with Flink easily and run Beam jobs.
+function create() {
upload_init_actions
create_cluster
get_leader
[[ -n "${JOB_SERVER_IMAGE:=}" ]] && start_job_server
start_tunnel
}
-main "$@"
\ No newline at end of file
+# Recreates a Flink cluster.
+function restart() {
+ delete
+ create
+}
+
+# Deletes a Flink cluster.
+function delete() {
+ gcloud dataproc clusters delete $CLUSTER_NAME --quiet
+}
+
+function scale() {
Review comment:
Why don't you use `gcloud dataproc update` for scaling the dataproc cluster?
https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/scaling-clusters
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 288868)
Time Spent: 3h 10m (was: 3h)
> Simpler Flink cluster set up in load tests
> ------------------------------------------
>
> Key: BEAM-7872
> URL: https://issues.apache.org/jira/browse/BEAM-7872
> Project: Beam
> Issue Type: Sub-task
> Components: testing
> Reporter: Kamil Wasilewski
> Assignee: Kamil Wasilewski
> Priority: Major
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> Creating a new load test running on Flink runner could be easier by providing
> a single `setUp` function which would encapsulate the process of creating
> Flink cluster and registering teardown steps
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
