angoenka commented on code in PR #17271:
URL: https://github.com/apache/beam/pull/17271#discussion_r844479607
##########
sdks/python/apache_beam/runners/dataflow/internal/apiclient.py:
##########
@@ -1174,11 +1174,22 @@ def _use_unified_worker(pipeline_options):
debug_options = pipeline_options.view_as(DebugOptions)
use_unified_worker_flag = 'use_unified_worker'
use_runner_v2_flag = 'use_runner_v2'
+ enable_prime_flag = 'enable_prime'
if (debug_options.lookup_experiment(use_runner_v2_flag) and
not debug_options.lookup_experiment(use_unified_worker_flag)):
debug_options.add_experiment(use_unified_worker_flag)
+ dataflow_service_options = pipeline_options.view_as(
Review Comment:
we should also check for disable_runner_v2 and disable_unified_worker
options before opting into dataflow prime.
##########
sdks/python/apache_beam/runners/dataflow/internal/apiclient.py:
##########
@@ -1174,11 +1174,22 @@ def _use_unified_worker(pipeline_options):
debug_options = pipeline_options.view_as(DebugOptions)
use_unified_worker_flag = 'use_unified_worker'
use_runner_v2_flag = 'use_runner_v2'
+ enable_prime_flag = 'enable_prime'
if (debug_options.lookup_experiment(use_runner_v2_flag) and
not debug_options.lookup_experiment(use_unified_worker_flag)):
debug_options.add_experiment(use_unified_worker_flag)
+ dataflow_service_options = pipeline_options.view_as(
Review Comment:
Also, should we add enable_prime to both debug option and
dataflow_service_option when we are enabling it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
