[
https://issues.apache.org/jira/browse/BEAM-8183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16941330#comment-16941330
]
Ankur Goenka commented on BEAM-8183:
------------------------------------
Flink has some neat feature of picking the pipeline on the fly.
I don't think this is a very common usecase with Beam though.
Given Beam Job Submission api work on a single pipeline at a time it will be
very convoluted work flow to introduce multiple pipelines in a single jar.
Will it be easier to just store pipelines as separate jar in global storage
(hdfs etc) and pass the right jar at the time of pipeline submission?
In case the submission happen through a service then will it be easier and less
error prone to just keep these jars separately on the service and submit the
right jar to flink based on the parameter?
> Optionally bundle multiple pipelines into a single Flink jar
> ------------------------------------------------------------
>
> Key: BEAM-8183
> URL: https://issues.apache.org/jira/browse/BEAM-8183
> Project: Beam
> Issue Type: New Feature
> Components: runner-flink
> Reporter: Kyle Weaver
> Assignee: Kyle Weaver
> Priority: Major
> Labels: portability-flink
>
> [https://github.com/apache/beam/pull/9331#issuecomment-526734851]
> "With Flink you can bundle multiple entry points into the same jar file and
> specify which one to use with optional flags. It may be desirable to allow
> inclusion of multiple pipelines for this tool also, although that would
> require a different workflow. Absent this option, it becomes quite convoluted
> for users that need the flexibility to choose which pipeline to launch at
> submission time."
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
