[
https://issues.apache.org/jira/browse/BEAM-8251?focusedWorklogId=499532&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-499532
]
ASF GitHub Bot logged work on BEAM-8251:
----------------------------------------
Author: ASF GitHub Bot
Created on: 12/Oct/20 17:07
Start Date: 12/Oct/20 17:07
Worklog Time Spent: 10m
Work Description: ibzib commented on a change in pull request #13072:
URL: https://github.com/apache/beam/pull/13072#discussion_r503424490
##########
File path: sdks/go/gogradle.lock
##########
@@ -5,7 +5,7 @@ dependencies:
build:
- vcs: "git"
name: "cloud.google.com/go"
- commit: "4f6c921ec566a33844f4e7879b31cd8575a6982d"
+ commit: "03869a08dc16b35ad4968e92d34c5a2a2961b205"
Review comment:
This PR shouldn't require any changes to dependencies.
##########
File path: sdks/go/pkg/beam/runners/dataflow/dataflowlib/job.go
##########
@@ -262,3 +272,36 @@ func printOptions(opts *JobOptions, images []string)
[]*displayData {
}
return ret
}
+
+func validateWorkerSettings(opts *JobOptions) error {
Review comment:
Add tests for this function.
##########
File path: sdks/go/pkg/beam/runners/dataflow/dataflowlib/job.go
##########
@@ -262,3 +272,36 @@ func printOptions(opts *JobOptions, images []string)
[]*displayData {
}
return ret
}
+
+func validateWorkerSettings(opts *JobOptions) error {
+ if opts.Zone != "" && opts.WorkerRegion != "" {
+ return errors.New("cannot use option zone with workerRegion;
prefer either workerZone or workerRegion")
+ }
+ if opts.Zone != "" && opts.WorkerZone != "" {
+ return errors.New("cannot use option zone with workerZone;
prefer workerZone")
+ }
+ if opts.WorkerZone != "" && opts.WorkerRegion != "" {
+ return errors.New("workerRegion and workerZone options are
mutually exclusive")
+ }
+
+ hasExperimentWorkerRegion := false
+ for _, experiment := range opts.Experiments {
+ if strings.HasPrefix(experiment, "worker_region") {
+ hasExperimentWorkerRegion = true
+ break
+ }
+ }
+
+ if hasExperimentWorkerRegion && opts.WorkerRegion != "" {
+ return errors.New("experiment worker_region and option
workerRegion are mutually exclusive")
+ }
+ if hasExperimentWorkerRegion && opts.WorkerZone != "" {
+ return errors.New("experiment worker_region and option
workerZone are mutually exclusive")
+ }
+
+ if opts.Zone != "" {
Review comment:
Print a warning that zone is deprecated and worker_zone should be used
instead.
##########
File path: sdks/go/pkg/beam/runners/dataflow/dataflowlib/job.go
##########
@@ -262,3 +272,36 @@ func printOptions(opts *JobOptions, images []string)
[]*displayData {
}
return ret
}
+
+func validateWorkerSettings(opts *JobOptions) error {
+ if opts.Zone != "" && opts.WorkerRegion != "" {
+ return errors.New("cannot use option zone with workerRegion;
prefer either workerZone or workerRegion")
+ }
+ if opts.Zone != "" && opts.WorkerZone != "" {
+ return errors.New("cannot use option zone with workerZone;
prefer workerZone")
+ }
+ if opts.WorkerZone != "" && opts.WorkerRegion != "" {
+ return errors.New("workerRegion and workerZone options are
mutually exclusive")
+ }
+
+ hasExperimentWorkerRegion := false
+ for _, experiment := range opts.Experiments {
+ if strings.HasPrefix(experiment, "worker_region") {
+ hasExperimentWorkerRegion = true
+ break
+ }
+ }
+
+ if hasExperimentWorkerRegion && opts.WorkerRegion != "" {
Review comment:
There also needs to be a check for `hasExperimentWorkerRegion &&
opts.Zone != ""`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 499532)
Time Spent: 2h 40m (was: 2.5h)
> Add worker_region and worker_zone options
> -----------------------------------------
>
> Key: BEAM-8251
> URL: https://issues.apache.org/jira/browse/BEAM-8251
> Project: Beam
> Issue Type: Improvement
> Components: runner-dataflow
> Reporter: Kyle Weaver
> Priority: P3
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> We are refining the way the user specifies worker regions and zones to the
> Dataflow service. We need to add worker_region and worker_zone pipeline
> options that will be preferred over the old experiments=worker_region and
> --zone flags. I will create subtasks for adding these options to each SDK.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
