[ https://issues.apache.org/jira/browse/FLINK-5815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15885735#comment-15885735 ]
ASF GitHub Bot commented on FLINK-5815: --------------------------------------- Github user tillrohrmann commented on a diff in the pull request: https://github.com/apache/flink/pull/3388#discussion_r103177702 --- Diff: flink-fs-tests/src/test/java/org/apache/flink/hdfstests/FileUtilsTest.java --- @@ -0,0 +1,117 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.hdfstests; + +import org.apache.flink.core.fs.FSDataInputStream; +import org.apache.flink.core.fs.FSDataOutputStream; +import org.apache.flink.core.fs.FileSystem; +import org.apache.flink.core.fs.Path; +import org.apache.flink.test.util.AbstractTestBase; +import org.apache.flink.util.FileUtils; +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.hdfs.MiniDFSCluster; +import org.junit.*; +import org.junit.rules.TemporaryFolder; + +import java.io.File; +import java.io.IOException; +import java.net.URI; + +import static org.junit.Assert.fail; + +/** + * Created by wenlong.lwl on 2017/2/21. --- End diff -- We don't do these comments. > Add resource files configuration for Yarn Mode > ---------------------------------------------- > > Key: FLINK-5815 > URL: https://issues.apache.org/jira/browse/FLINK-5815 > Project: Flink > Issue Type: Improvement > Components: Client, YARN > Affects Versions: 1.3.0 > Reporter: Wenlong Lyu > Assignee: Wenlong Lyu > > Currently in flink, when we want to setup a resource file to distributed > cache, we need to make the file accessible remotely by a url, which is often > difficult to maintain a service like that. What's more, when we want do add > some extra jar files to job classpath, we need to copy the jar files to blob > server when submitting the jobgraph. In yarn, especially in flip-6, the blob > server is not running yet when we try to start a flink job. > Yarn has a efficient distributed cache implementation for application running > on it, what's more we can be easily share the files stored in hdfs in > different application by distributed cache without extra IO operations. > I suggest to introduce -yfiles, -ylibjars -yarchives options to FlinkYarnCLI > to enable yarn user setup their job resource files by yarn distributed cache. > The options is compatible with what is used in mapreduce, which make it easy > to use for yarn user who generally has experience on using mapreduce. -- This message was sent by Atlassian JIRA (v6.3.15#6346)