[
https://issues.apache.org/jira/browse/FLINK-32809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Fan resolved FLINK-32809.
-----------------------------
Fix Version/s: 1.18.0
Resolution: Fixed
> YarnClusterDescriptor.isarchiveonlyincludedinShipArchiveFiles dose not work
> as expected
> ---------------------------------------------------------------------------------------
>
> Key: FLINK-32809
> URL: https://issues.apache.org/jira/browse/FLINK-32809
> Project: Flink
> Issue Type: Bug
> Components: Deployment / YARN
> Affects Versions: 1.18.0
> Reporter: junzhong qin
> Assignee: junzhong qin
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.18.0
>
>
> YarnClusterDescriptor.isarchiveonlyincludedinShipArchiveFiles(List<File>
> shipFiles) check wether the shipFiles are all archive files, but it dose not
> work as expected.
> {code:java}
> public static boolean isArchiveOnlyIncludedInShipArchiveFiles(List<File>
> shipFiles) {
> return shipFiles.stream()
> .filter(File::isFile)
> .map(File::getName)
> .map(String::toLowerCase)
> .allMatch(
> name ->
> name.endsWith(".tar.gz")
> || name.endsWith(".tar")
> || name.endsWith(".tgz")
> || name.endsWith(".dst")
> || name.endsWith(".jar")
> || name.endsWith(".zip"));
> } {code}
> When we pass a directory and an archive file it should return false but it
> returns true.
> {code:java}
> // dir1 is a directory and archive.zip is an archive file
> List<File> files = Arrays.asList(new File("/tmp/dir1"), new
> File("/tmp/archive.zip"));
> boolean result = isArchiveOnlyIncludedInShipArchiveFiles(files);
> System.out.println(result); // Print true but is should print false{code}
> If flink user want to add directory as ship file, they should use
> configuration: yarn.ship-files .
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
