kl0u commented on a change in pull request #12791:
URL: https://github.com/apache/flink/pull/12791#discussion_r452671394
##########
File path:
flink-yarn/src/main/java/org/apache/flink/yarn/YarnClusterDescriptor.java
##########
@@ -267,6 +278,29 @@ public void addShipFiles(List<File> shipFiles) {
this.shipFiles.addAll(shipFiles);
}
+ /**
+ * Adds the given files to the list of archives to ship.
+ **
+ * @param shipArchives archives to ship
+ */
+ public void addShipArchives(List<File> shipArchives) {
+
checkArgument(isArchiveOnlyIncludedInShipArchiveFiles(shipArchives),
"Non-archive files are included.");
+ this.shipArchives.addAll(shipArchives);
+ }
+
+ private static boolean
isArchiveOnlyIncludedInShipArchiveFiles(List<File> shipFiles) {
+ return shipFiles.stream()
+ .filter(File::isFile)
+ .map(File::getName)
+ .map(String::toLowerCase)
+ .allMatch(name -> name.endsWith(".tar.gz") ||
+ name.endsWith(".tar") ||
+ name.endsWith(".tgz") ||
+ name.endsWith(".dst") ||
+ name.endsWith(".jar") ||
Review comment:
Ok let's leave it like this for now :). Could you integrate the rest of
my comments?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
