Gyula Fora created FLINK-4873:
---------------------------------
Summary: Add config option to specify "home directory" for YARN
client resource sharing
Key: FLINK-4873
URL: https://issues.apache.org/jira/browse/FLINK-4873
Project: Flink
Issue Type: Improvement
Components: YARN Client
Affects Versions: 1.2.0, 1.1.3
Reporter: Gyula Fora
The YARN client currently uses FileSystem.getHomeDirectory() to store the jar
files that needed to be shared on the cluster. This pretty much forces users to
run HDFS or something compatible with the Hadoop FS api on the cluster.
In some production environments file systems (distributed or simply shared) are
simply mounted under the same path and do not require the use of the hadoop api
for convenience. If we want to run Flink on YARN in these cases we would need
to be able to define the "home directory" where Flink should copy the files for
sharing.
It could be something like:
yarn.resource.upload.dir in the flink-conf.yaml
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
