Github user cresny commented on a diff in the pull request:
https://github.com/apache/flink/pull/2288#discussion_r74774950
--- Diff: flink-yarn/src/main/java/org/apache/flink/yarn/Utils.java ---
@@ -255,4 +257,33 @@ private Utils() {
}
return result;
}
+
+ /**
+ * Recursive directory copy to work around FileSystem implementations
that do not implement it.
+ * @param fs
+ * @param localPath
+ * @param remotePath
+ * @throws IOException
+ */
+ protected static void copyFromLocalFile(final FileSystem fs, final Path
localPath, final Path remotePath) throws IOException {
+ File localFile = new File(localPath.toUri());
+ if (localFile.isDirectory()) {
+ for (File file : localFile.listFiles()) {
+ copyFromLocalFile(fs, new Path("file://" +
file.getAbsolutePath()), new Path(remotePath,file.getName()));
+ }
+ } else {
+ fs.copyFromLocalFile(localPath,remotePath);
+ }
+ }
+
+ public static Path checkScheme(final Path localRsrcPath) throws
IOException {
--- End diff --
Yes on changes. The short answer to what will happen if scheme is
undefined is it will fail. I don't remember offhand exactly why, but I think
may have been because the default file system is S3a, which I suppose would
become the default fs for unqualified paths. I noticed that all but one or two
callers' paths do have qualified scheme. Will add comment.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
