jin xing created SPARK-23669:

             Summary: Executors fetch jars and name the jars with md5 prefix
                 Key: SPARK-23669
                 URL: https://issues.apache.org/jira/browse/SPARK-23669
             Project: Spark
          Issue Type: New Feature
          Components: Spark Core
    Affects Versions: 2.3.0
         Environment: In our cluster, there are lots of UDF jars, some of them 
have the same filename but different path, for example:


hdfs://A/B/udf.jar  -> udfA

hdfs://C/D/udf.jar -> udfB


When user uses udfA and udfB in same sql, executor will fetch both 
\{hdfs://A/B/udf.jar} and \{hdfs://C/D/udf.jar}. There will be a conflict for 
the same name. 


Can we config to fetch jars and save with a filename with MD5 prefix, so there 
will be no conflict.
            Reporter: jin xing

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to