damccorm commented on code in PR #25739:
URL: https://github.com/apache/beam/pull/25739#discussion_r1127981595
##########
sdks/python/apache_beam/runners/dataflow/internal/apiclient.py:
##########
@@ -506,7 +506,13 @@ def __init__(self, options, proto_pipeline):
# Labels.
if self.google_cloud_options.labels:
self.proto.labels = dataflow.Job.LabelsValue()
- for label in self.google_cloud_options.labels:
+ labels = self.google_cloud_options.labels
+ if len(labels) == 1:
+ # labels can be passed as cmd args
+ # --labels=name=wrench,age=32,job=dataflow and this is parsed by
+ # argparse as 'labels' : ["name=wrench,age=32,job=dataflow"]
+ labels = labels[0].split(',')
Review Comment:
I think we should prefer the json format currently specified in
https://cloud.google.com/dataflow/docs/reference/pipeline-options#python rather
than just doing comma separated since that is consistent with how Java handles
this. Whenever we can fairly easily remain consistent, we should do so.
##########
sdks/python/apache_beam/runners/dataflow/internal/apiclient.py:
##########
@@ -506,7 +506,13 @@ def __init__(self, options, proto_pipeline):
# Labels.
if self.google_cloud_options.labels:
self.proto.labels = dataflow.Job.LabelsValue()
- for label in self.google_cloud_options.labels:
+ labels = self.google_cloud_options.labels
+ if len(labels) == 1:
Review Comment:
Could you please add a test for this here -
https://github.com/apache/beam/blob/380359ce9ebd284ecd9bc4c91efdd5f30da21425/sdks/python/apache_beam/runners/dataflow/internal/apiclient_test.py#L786?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
