[
https://issues.apache.org/jira/browse/FLINK-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15837036#comment-15837036
]
ASF GitHub Bot commented on FLINK-5631:
---------------------------------------
GitHub user haohui opened a pull request:
https://github.com/apache/flink/pull/3202
[FLINK-5631] [yarn] Support downloading additional jars from non-HDFS…
This PR enables Flink to support downloading additional jars from non-HDFS
paths (e.g., S3, viewfs). More details are available in the jira.
There are two changes in this PR:
* Refactoring the code for localization into a common function
* Use `Path.getFileSystem()` to obtain the FileSystem object instead of
always using the default FileSystem constructed using YARN config.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/haohui/flink FLINK-5631
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3202.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3202
----
commit f7d24c5a0559ca71cd49962c9dc0b4ab81997783
Author: Haohui Mai <[email protected]>
Date: 2017-01-25T02:00:08Z
[FLINK-5631] [yarn] Support downloading additional jars from non-HDFS paths.
----
> [yarn] Support downloading additional jars from non-HDFS paths
> --------------------------------------------------------------
>
> Key: FLINK-5631
> URL: https://issues.apache.org/jira/browse/FLINK-5631
> Project: Flink
> Issue Type: Bug
> Reporter: Haohui Mai
> Assignee: Haohui Mai
>
> Currently the {{YarnResourceManager}} and {{YarnApplicationMasterRunner}}
> always register the additional jars using the YARN filesystem object. This is
> problematic as the paths might require another filesystem.
> To support localizing from non-HDFS paths (e.g., s3, http or viewfs), the
> cleaner approach is to get the filesystem object from the path.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
