[
https://issues.apache.org/jira/browse/FLINK-32775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17751820#comment-17751820
]
Archit Goyal commented on FLINK-32775:
--------------------------------------
Rootcause : The parent directory is not being added when using
{{yarn.provided.lib.dirs}} but when using {{{}yarn.ship-files{}}}, the parent
dir is added.{{{}yarn.provided.lib.dirs{}}}
([Link|https://github.com/apache/flink/blob/255d83087c6a6432270e6886ffdcf85dae00c241/flink-yarn/src/main/java/org/apache/flink/yarn/YarnApplicationFileUploader.java#L343C1-L344C1])
should construct the classpath similar as {{yarn.ship-files}}
([Link|https://github.com/apache/flink/blob/255d83087c6a6432270e6886ffdcf85dae00c241/flink-yarn/src/main/java/org/apache/flink/yarn/YarnApplicationFileUploader.java#L233])
where we should add the parent dir also, followed by jars.
Will raise a PR with the fix.
> Support yarn.provided.lib.dirs to add parent directory to classpath
> -------------------------------------------------------------------
>
> Key: FLINK-32775
> URL: https://issues.apache.org/jira/browse/FLINK-32775
> Project: Flink
> Issue Type: New Feature
> Components: Deployment / YARN
> Reporter: Archit Goyal
> Assignee: Archit Goyal
> Priority: Minor
> Labels: pull-request-available
>
> Currently with {*}yarn.provided.lib.dirs{*}, Flink libs can be copied to HDFS
> location in each cluster and when set Flink tries to reuse the same jars
> avoiding uploading it every time and YARN also caches it in the nodes.
>
> This works fine with jars but if we try to add the xml file parent directory
> to path, Flink job submission fails. If I add the parent directory of the xml
> to the
> {noformat}
> yarn.ship-files{noformat}
> Flink job is submitted successfully.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
