[
https://issues.apache.org/jira/browse/FLINK-10934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077769#comment-17077769
]
Yang Wang commented on FLINK-10934:
-----------------------------------
[~felixzheng] Thanks for your attention about this ticket. The design of K8s
perjob(maybe we call it "application" mode is more reasonable) is aligned with
FLIP-85[1]. It is just a specific implementation for K8s deployment.
Comparing with YARN application mode, the way to distribute user jars,
dependencies is different.
* On Yarn deployment, we could register the local or HDFS jar/files as
LocalResource.
And let Yarn to localize the resource to workdir, when the entrypoint is
launched, all
the jars and dependencies exist locally. So the entrypoint will *NOT* do the
real fetching,
* For K8s deployment, the jars need to be built in image or fetched by
init-container. Then the following code path will be exact same as Yarn.
For the implementation, i will introduce a new
{{KubernetesApplicationClusterEntrypoint}} which will use the new introduced
FLINK-16655 to submit the job. Since the {{DispatcherBootstrap}} will also be
introduced in FLINK-16656 to make the submission and recovery more common(i.e.
reused by Standalone, YARN, K8s), then the entrypoint is just a slim wrapper.
For the client side, i think a new interface in
{{ClusterDescriptor#deployApplicationCluster}} will be introduced. And we will
provide "flink run-application" command for the CLI. You could refer to this
thread[1] for more discussion details.
Most of the changes will be done in the scope of FLIP-85. Please share more
your thoughts if you still think it is not clear enough.
[1].
[https://cwiki.apache.org/confluence/display/FLINK/FLIP-85+Flink+Application+Mode]
[2].
[http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-85-Delayed-Job-Graph-Generation-td35759.html]
> Support per-job mode in kubernetes
> ----------------------------------
>
> Key: FLINK-10934
> URL: https://issues.apache.org/jira/browse/FLINK-10934
> Project: Flink
> Issue Type: Sub-task
> Components: Deployment / Kubernetes
> Reporter: JIN SUN
> Assignee: Yang Wang
> Priority: Major
>
> The Kubernetes do not have a default distributed storage and provide public
> api to ship files like Yarn local resource. So we could not ship the user
> jars and files on the client side to jobmanager and taskmanager. Also it is
> not a common way on Kubernetes. Instead, users usually build their jars and
> files into the docker image. So when the jobmanager and taskmanager are
> launched, the users jars already existed.
> Even if some users do not want to build the jars into the image, they could
> use the initContainer to download the jars from the storage(http/s3/etc.).
> All in all, the Kubernetes per-job cluster will only support cluster
> deploy-mode.
>
> This ticket depends on FLIP-85(Support cluster deployment). Please reference
> the documentation
> [FLIP-85|https://cwiki.apache.org/confluence/display/FLINK/FLIP-85+Flink+Application+Mode]
> for more information.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
