[ https://issues.apache.org/jira/browse/SPARK-44845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mridul Muralidharan reassigned SPARK-44845: ------------------------------------------- Assignee: zheju_he > spark job copies jars repeatedly if fs.defaultFS and application jar are same > url > --------------------------------------------------------------------------------- > > Key: SPARK-44845 > URL: https://issues.apache.org/jira/browse/SPARK-44845 > Project: Spark > Issue Type: Bug > Components: YARN > Affects Versions: 3.4.1 > Reporter: zheju_he > Assignee: zheju_he > Priority: Major > > In the org.apache.spark.deploy.yarn.Client#compareUri method, > hdfs://hadoop81:8020 and hdfs://192.168.0.81:8020 are regarded as different > file systems (hadoop81 corresponds to 192.168.0.81). The specific reason is > that in the last pr, different URIs of user information are also regarded as > different file systems. Uri.getauthority is used to determine the user > information, but authority contains the host so the URI above must be > different from authority. To determine whether the user authentication > information is different, you only need to determine URI.getUserInfo. > > the last pr and issue link: > https://issues.apache.org/jira/browse/SPARK-22587 > https://github.com/apache/spark/pull/19885 -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org