[
https://issues.apache.org/jira/browse/BEAM-5509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16632172#comment-16632172
]
Thomas Weise commented on BEAM-5509:
------------------------------------
A bit more digging shows that the culprit is really the conversion from dict to
struct in
google/protobuf/json_format.py(582)_ConvertValueMessage
{code:java}
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/bdb.py(400)run()
-> exec cmd in globals, locals
<string>(1)<module>()
/Users/tweise/src/beam/sdks/python/flink-example.py(23)<module>()
-> | beam.Map(lambda x: logging.info("1Got %s", x) or (x, 1))
/Users/tweise/src/beam/sdks/python/apache_beam/pipeline.py(414)__exit__()
-> self.run().wait_until_finish()
/Users/tweise/src/beam/sdks/python/apache_beam/pipeline.py(394)run()
-> self.to_runner_api(), self.runner, self._options).run(False)
/Users/tweise/src/beam/sdks/python/apache_beam/pipeline.py(407)run()
-> return self.runner.run_pipeline(self)
/Users/tweise/src/beam/sdks/python/apache_beam/runners/portability/portable_runner.py(165)run_pipeline()
-> prepare_response = send_prepare_request()
/Users/tweise/src/beam/sdks/python/apache_beam/runners/portability/portable_runner.py(159)send_prepare_request()
-> pipeline_options=job_utils.dict_to_struct(options)))
/Users/tweise/src/beam/sdks/python/apache_beam/runners/job/utils.py(30)dict_to_struct()
-> return json_format.ParseDict(dict_obj, struct_pb2.Struct())
/Users/tweise/python-ve/beam/lib/python2.7/site-packages/google/protobuf/json_format.py(404)ParseDict()
-> parser.ConvertMessage(js_dict, message)
/Users/tweise/python-ve/beam/lib/python2.7/site-packages/google/protobuf/json_format.py(433)ConvertMessage()
-> methodcaller(_WKTJSONMETHODS[full_name][1], value, message)(self)
/Users/tweise/python-ve/beam/lib/python2.7/site-packages/google/protobuf/json_format.py(605)_ConvertStructMessage()
-> self._ConvertValueMessage(value[key], message.fields[key])
> /Users/tweise/python-ve/beam/lib/python2.7/site-packages/google/protobuf/json_format.py(582)_ConvertValueMessage(){code}
{code:java}
(Pdb) l
577 elif isinstance(value, list):
578 self. _ConvertListValueMessage(value, message.list_value)
579 elif value is None:
580 message.null_value = 0
581 elif isinstance(value, bool):
582 message.bool_value = value
583 elif isinstance(value, six.string_types):
584 message.string_value = value
585 elif isinstance(value, _INT_OR_FLOAT):
586 -> message.number_value = value
587 else:{code}
As we see int and float treated the same way. Is that an upstream bug that
should be filed as such?
> Python pipeline_options doesn't handle int type
> -----------------------------------------------
>
> Key: BEAM-5509
> URL: https://issues.apache.org/jira/browse/BEAM-5509
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-harness
> Reporter: Thomas Weise
> Assignee: Thomas Weise
> Priority: Major
> Labels: portability-flink
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The int option supplied at the command line is turned into a decimal during
> serialization and then the parser in SDK harness fails to restore it as int.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
