[
https://issues.apache.org/jira/browse/BEAM-8471?focusedWorklogId=345437&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-345437
]
ASF GitHub Bot logged work on BEAM-8471:
----------------------------------------
Author: ASF GitHub Bot
Created on: 18/Nov/19 17:29
Start Date: 18/Nov/19 17:29
Worklog Time Spent: 10m
Work Description: tweise commented on pull request #10139: [BEAM-8471]
Flink native job submission for portable pipelines
URL: https://github.com/apache/beam/pull/10139
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 345437)
Time Spent: 5h 50m (was: 5h 40m)
> Flink native job submission for portable pipelines
> --------------------------------------------------
>
> Key: BEAM-8471
> URL: https://issues.apache.org/jira/browse/BEAM-8471
> Project: Beam
> Issue Type: Improvement
> Components: runner-flink
> Reporter: Thomas Weise
> Assignee: Thomas Weise
> Priority: Major
> Labels: portability-flink
> Fix For: 2.18.0
>
> Time Spent: 5h 50m
> Remaining Estimate: 0h
>
> There are currently two methods to run a portable pipeline written in a
> non-JVM language to Flink:
> 1) Run the SDK client entry point which will submit the job server, which in
> turn will submit to a Flink cluster using the Flink remote environment
> 2) Run the SDK client entry point to generate a Flink jar file that can be
> used to start the Flink job using any of the Flink client tooling available.
> Either approach requires the SDK client and the job server dependency to be
> present on the client. This doesn't work well in environments such as
> FlinkK8sOperator that rely on the Flink REST API jar run endpoint (see
> [https://docs.google.com/document/d/1z3LNrRtr8kkiFHonZ5JJM_L4NWNBBNcqRc_yAf6G0VI/edit#heading=h.x7hki4bhh18l]).
> This improvement is to provide a new Flink entry point (main method) that
> invokes the SDK client entry point to generate the pipeline and submits the
> resulting Flink job like any other Flink native driver program would, via the
> optimizer plan environment ("[auto]").
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
