[
https://issues.apache.org/jira/browse/FLINK-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16183304#comment-16183304
]
Mikhail Pryakhin commented on FLINK-6949:
-----------------------------------------
I propose to add a --yarnship-files CLI option that will submit folders with
files to YARN cluster and add these folders to the application classpath.
At the moment the --yarnship option
[traverses|https://github.com/apache/flink/blob/2eaf92b1f0a0d965c14b65755d25f1a8167de023/flink-yarn/src/main/java/org/apache/flink/yarn/AbstractYarnClusterDescriptor.java#L1003:]
the content of the specified yarnship folder and appends files to the
classpath. In case a ship folder contains non *.class, *.jar or *.zip files,
but for example config/property files this takes no effect if such files are
appended to the application classpath. So it would be great to have an ability
to add the whole folders to the application classpath.
I'm going to add a PR to support this feature.
> Add ability to ship custom resource files to YARN cluster
> ---------------------------------------------------------
>
> Key: FLINK-6949
> URL: https://issues.apache.org/jira/browse/FLINK-6949
> Project: Flink
> Issue Type: Improvement
> Components: Client, YARN
> Affects Versions: 1.3.0
> Reporter: Mikhail Pryakhin
> Assignee: Mikhail Pryakhin
> Priority: Critical
>
> *The problem:*
> When deploying a flink job on YARN it is not possible to specify custom
> resource files to be shipped to YARN cluster.
>
> *The use case description:*
> When running a flink job on multiple environments it becomes necessary to
> pass environment-related configuration files to the job's runtime. It can be
> accomplished by packaging configuration files within the job's jar. But
> having tens of different environments one can easily end up packaging as many
> jars as there are environments. It would be great to have an ability to
> separate configuration files from the job artifacts.
>
> *The possible solution:*
> add the --yarnship-files option to flink cli to specify files that should be
> shipped to the YARN cluster.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
