RocMarshal commented on code in PR #23191:
URL: https://github.com/apache/flink/pull/23191#discussion_r1290829384
##########
flink-yarn/src/main/java/org/apache/flink/yarn/YarnClusterDescriptor.java:
##########
@@ -305,23 +305,26 @@ public void addShipFiles(List<File> shipFiles) {
private void addShipArchives(List<File> shipArchives) {
checkArgument(
isArchiveOnlyIncludedInShipArchiveFiles(shipArchives),
- "Non-archive files are included.");
+ "Directories or non-archive files are included.");
this.shipArchives.addAll(shipArchives);
}
private static boolean isArchiveOnlyIncludedInShipArchiveFiles(List<File>
shipFiles) {
- return shipFiles.stream()
- .filter(File::isFile)
- .map(File::getName)
- .map(String::toLowerCase)
- .allMatch(
- name ->
- name.endsWith(".tar.gz")
- || name.endsWith(".tar")
- || name.endsWith(".tgz")
- || name.endsWith(".dst")
- || name.endsWith(".jar")
- || name.endsWith(".zip"));
+ long archivedFileCount =
+ shipFiles.stream()
+ .filter(File::isFile)
+ .map(File::getName)
+ .map(String::toLowerCase)
Review Comment:
Could these two lines be inlined?
Because there's no filters between the two `maps`
##########
flink-yarn/src/main/java/org/apache/flink/yarn/YarnClusterDescriptor.java:
##########
@@ -305,23 +305,26 @@ public void addShipFiles(List<File> shipFiles) {
private void addShipArchives(List<File> shipArchives) {
checkArgument(
isArchiveOnlyIncludedInShipArchiveFiles(shipArchives),
- "Non-archive files are included.");
+ "Directories or non-archive files are included.");
this.shipArchives.addAll(shipArchives);
}
private static boolean isArchiveOnlyIncludedInShipArchiveFiles(List<File>
shipFiles) {
- return shipFiles.stream()
- .filter(File::isFile)
- .map(File::getName)
- .map(String::toLowerCase)
- .allMatch(
- name ->
- name.endsWith(".tar.gz")
- || name.endsWith(".tar")
- || name.endsWith(".tgz")
- || name.endsWith(".dst")
- || name.endsWith(".jar")
- || name.endsWith(".zip"));
+ long archivedFileCount =
+ shipFiles.stream()
+ .filter(File::isFile)
+ .map(File::getName)
+ .map(String::toLowerCase)
+ .filter(
+ name ->
+ name.endsWith(".tar.gz")
+ || name.endsWith(".tar")
+ || name.endsWith(".tgz")
+ || name.endsWith(".dst")
+ || name.endsWith(".jar")
+ || name.endsWith(".zip"))
Review Comment:
Is there a possibility of frequent changes in the suffix format here?
If so, would you consider introducing a parameter to specify the suffix
format?
Please correct me if I'm wrong in my limited read.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
