[
https://issues.apache.org/jira/browse/BEAM-5509?focusedWorklogId=149242&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-149242
]
ASF GitHub Bot logged work on BEAM-5509:
----------------------------------------
Author: ASF GitHub Bot
Created on: 28/Sep/18 17:46
Start Date: 28/Sep/18 17:46
Worklog Time Spent: 10m
Work Description: tweise closed pull request #6514: [BEAM-5509] Fix int
pipeline option serialization in portable_runner
URL: https://github.com/apache/beam/pull/6514
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/sdks/python/apache_beam/runners/job/utils.py
b/sdks/python/apache_beam/runners/job/utils.py
index 5a247cd2729..3e347517972 100644
--- a/sdks/python/apache_beam/runners/job/utils.py
+++ b/sdks/python/apache_beam/runners/job/utils.py
@@ -27,7 +27,7 @@
def dict_to_struct(dict_obj):
- return json_format.Parse(json.dumps(dict_obj), struct_pb2.Struct())
+ return json_format.ParseDict(dict_obj, struct_pb2.Struct())
def struct_to_dict(struct_obj):
diff --git a/sdks/python/apache_beam/runners/portability/portable_runner.py
b/sdks/python/apache_beam/runners/portability/portable_runner.py
index ec537266324..15ae84d341a 100644
--- a/sdks/python/apache_beam/runners/portability/portable_runner.py
+++ b/sdks/python/apache_beam/runners/portability/portable_runner.py
@@ -136,7 +136,8 @@ def run_pipeline(self, pipeline):
del transform_proto.subtransforms[:]
# TODO: Define URNs for options.
- options = {'beam:option:' + k + ':v1': v
+ # convert int values: https://issues.apache.org/jira/browse/BEAM-5509
+ options = {'beam:option:' + k + ':v1': (str(v) if type(v) == int else v)
for k, v in pipeline._options.get_all_options().items()
if v is not None}
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 149242)
Time Spent: 0.5h (was: 20m)
> Python pipeline_options doesn't handle int type
> -----------------------------------------------
>
> Key: BEAM-5509
> URL: https://issues.apache.org/jira/browse/BEAM-5509
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-harness
> Reporter: Thomas Weise
> Assignee: Thomas Weise
> Priority: Major
> Labels: portability-flink
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The int option supplied at the command line is turned into a decimal during
> serialization and then the parser in SDK harness fails to restore it as int.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
