[
https://issues.apache.org/jira/browse/BEAM-9446?focusedWorklogId=402514&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-402514
]
ASF GitHub Bot logged work on BEAM-9446:
----------------------------------------
Author: ASF GitHub Bot
Created on: 12/Mar/20 21:53
Start Date: 12/Mar/20 21:53
Worklog Time Spent: 10m
Work Description: mxm commented on issue #11052: [BEAM-9446] Add missing
parallelism and execution mode args.
URL: https://github.com/apache/beam/pull/11052#issuecomment-598434409
What I meant is keeping "unknown" pipeline options instead of discarding
them. If we pass the unknown ones as strings I thought this could work. For
example, if we pass `"123"` the parser on the Java side is smart enough to
figure it is the integer `123`. I haven't tried this out but I'd suggest we do.
Of course this is not very user-friendly. That's why I'd suggest to write
pipeline options in a language-agnostic format which can be read by all SDKs.
Considering the Flink pipeline options, what we usually want to set is what
is called `ExecutionConfig` in Flink. With Flink they are configured directly
in the API. For Beam if the parameters are just strings/ints we could come up
with something reflection based to set arbitrary new settings. I agree this
would be helpful.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 402514)
Time Spent: 1h 50m (was: 1h 40m)
> FlinkRunner discards parallelism and execution_mode_for_batch pipeline options
> ------------------------------------------------------------------------------
>
> Key: BEAM-9446
> URL: https://issues.apache.org/jira/browse/BEAM-9446
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Reporter: Kyle Weaver
> Assignee: Kyle Weaver
> Priority: Major
> Labels: portability-flink
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> I need these options for TFX, but they're being discarded (I believe they are
> normally supplied by the job server).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
