rohdesamuel commented on code in PR #17736:
URL: https://github.com/apache/beam/pull/17736#discussion_r883119026
##########
sdks/python/apache_beam/runners/interactive/interactive_beam.py:
##########
@@ -418,15 +424,19 @@ def create(
raise ValueError(
'Unknown cluster identifier: %s. Cannot create or reuse'
'a Dataproc cluster.')
- elif cluster_metadata.region == 'global':
- # The global region is unsupported as it will be eventually deprecated.
- raise ValueError('Clusters in the global region are not supported.')
- elif not cluster_metadata.region:
+ if not cluster_metadata.region:
_LOGGER.info(
'No region information was detected, defaulting Dataproc cluster '
'region to: us-central1.')
cluster_metadata.region = 'us-central1'
+ elif cluster_metadata.region == 'global':
+ # The global region is unsupported as it will be eventually deprecated.
+ raise ValueError('Clusters in the global region are not supported.')
# else use the provided region.
+ if cluster_metadata.num_workers and cluster_metadata.num_workers < 2:
Review Comment:
Why are two workers required? Also can you please make the magic number a
constant?
##########
sdks/python/apache_beam/runners/interactive/dataproc/types.py:
##########
@@ -50,5 +79,8 @@ def __eq__(self, other):
return self.__key() == other.__key()
return False
+ def rename(self):
Review Comment:
Somewhat of an awkward name for something that sets the cluster name to the
default. Maybe "reset_name" or something similar would be better?
##########
sdks/python/apache_beam/runners/interactive/interactive_beam.py:
##########
@@ -475,7 +485,9 @@ def cleanup(
'options is deprecated since First stable release. References to '
'<pipeline>.options will not be supported',
category=DeprecationWarning)
- p.options.view_as(FlinkRunnerOptions).flink_master = '[auto]'
+ p_flink_options = p.options.view_as(FlinkRunnerOptions)
+ p_flink_options.flink_master = '[auto]'
+ p_flink_options.flink_version = None
Review Comment:
Why do we reset the flink version here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
