[ https://issues.apache.org/jira/browse/SPARK-23669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
jin xing updated SPARK-23669: ----------------------------- Environment: (was: In our cluster, there are lots of UDF jars, some of them have the same filename but different path, for example: ``` hdfs://A/B/udf.jar -> udfA hdfs://C/D/udf.jar -> udfB ``` When user uses udfA and udfB in same sql, executor will fetch both \{hdfs://A/B/udf.jar} and \{hdfs://C/D/udf.jar}. There will be a conflict for the same name. Can we config to fetch jars and save with a filename with MD5 prefix, so there will be no conflict.) > Executors fetch jars and name the jars with md5 prefix > ------------------------------------------------------ > > Key: SPARK-23669 > URL: https://issues.apache.org/jira/browse/SPARK-23669 > Project: Spark > Issue Type: New Feature > Components: Spark Core > Affects Versions: 2.3.0 > Reporter: jin xing > Priority: Minor > > In our cluster, there are lots of UDF jars, some of them have the same > filename but different path, for example: > ``` > hdfs://A/B/udf.jar -> udfA > hdfs://C/D/udf.jar -> udfB > ``` > When user uses udfA and udfB in same sql, executor will fetch both > \{hdfs://A/B/udf.jar} and \{hdfs://C/D/udf.jar}. There will be a conflict for > the same name. > > Can we config to fetch jars and save with a filename with MD5 prefix, so > there will be no conflict. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org