[jira] [Assigned] (SPARK-23669) Executors fetch jars and name the jars with md5 prefix
[ https://issues.apache.org/jira/browse/SPARK-23669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23669: Assignee: (was: Apache Spark) > Executors fetch jars and name the jars with md5 prefix > -- > > Key: SPARK-23669 > URL: https://issues.apache.org/jira/browse/SPARK-23669 > Project: Spark > Issue Type: New Feature > Components: Spark Core >Affects Versions: 2.3.0 >Reporter: jin xing >Priority: Minor > > In our cluster, there are lots of UDF jars, some of them have the same > filename but different path, for example: > ``` > hdfs://A/B/udf.jar -> udfA > hdfs://C/D/udf.jar -> udfB > ``` > When user uses udfA and udfB in same sql, executor will fetch both > *hdfs://A/B/udf.jar* and *hdfs://C/D/udf.jar.* There will be a conflict for > the same name. > > Can we config to fetch jars and save with a filename with MD5 prefix, so > there will be no conflict. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-23669) Executors fetch jars and name the jars with md5 prefix
[ https://issues.apache.org/jira/browse/SPARK-23669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23669: Assignee: Apache Spark > Executors fetch jars and name the jars with md5 prefix > -- > > Key: SPARK-23669 > URL: https://issues.apache.org/jira/browse/SPARK-23669 > Project: Spark > Issue Type: New Feature > Components: Spark Core >Affects Versions: 2.3.0 >Reporter: jin xing >Assignee: Apache Spark >Priority: Minor > > In our cluster, there are lots of UDF jars, some of them have the same > filename but different path, for example: > ``` > hdfs://A/B/udf.jar -> udfA > hdfs://C/D/udf.jar -> udfB > ``` > When user uses udfA and udfB in same sql, executor will fetch both > *hdfs://A/B/udf.jar* and *hdfs://C/D/udf.jar.* There will be a conflict for > the same name. > > Can we config to fetch jars and save with a filename with MD5 prefix, so > there will be no conflict. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
