[
https://issues.apache.org/jira/browse/FLINK-4873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15661882#comment-15661882
]
ASF GitHub Bot commented on FLINK-4873:
---------------------------------------
GitHub user gyfora opened a pull request:
https://github.com/apache/flink/pull/2796
[FLINK-4873] Configurable ship path for YARN cluster resources
This PR makes the resource shipping directory configurable for YARN
clusters by adding a new config option: `yarn.resource.ship.path`
Currently the HDFS home directory is used by default which makes it
practically impossible to use the YARN cluster without HDFS (which is actually
quite feasible if you have a different distributed file system).
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gyfora/flink master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/2796.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2796
----
commit 00380ac5233a5b149672bef8a4a148200bd272d3
Author: Gyula Fora <[email protected]>
Date: 2016-11-11T12:45:29Z
[FLINK-4873] Configurable ship path for YARN cluster resources
----
> Add config option to specify "home directory" for YARN client resource sharing
> ------------------------------------------------------------------------------
>
> Key: FLINK-4873
> URL: https://issues.apache.org/jira/browse/FLINK-4873
> Project: Flink
> Issue Type: Improvement
> Components: YARN Client
> Affects Versions: 1.2.0, 1.1.3
> Reporter: Gyula Fora
>
> The YARN client currently uses FileSystem.getHomeDirectory() to store the jar
> files that needed to be shared on the cluster. This pretty much forces users
> to run HDFS or something compatible with the Hadoop FS api on the cluster.
> In some production environments file systems (distributed or simply shared)
> are simply mounted under the same path and do not require the use of the
> hadoop api for convenience. If we want to run Flink on YARN in these cases we
> would need to be able to define the "home directory" where Flink should copy
> the files for sharing.
> It could be something like:
> yarn.resource.upload.dir in the flink-conf.yaml
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
