Naved Alam created SPARK-27541:
----------------------------------

             Summary: Refresh class definitions for jars added via addJar()
                 Key: SPARK-27541
                 URL: https://issues.apache.org/jira/browse/SPARK-27541
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core
    Affects Versions: 2.2.3
            Reporter: Naved Alam


Currently, if a class is loaded by the executor, its definition cannot be 
updated (because classloaders won't load an already loaded class again). For 
use cases with long running sparkContexts, this becomes a problem when there 
are requirements to update the definition of one of these classes.

There should be a spark property which when turned on, allows the executors to 
refresh the definitions of these classes if they were to be added as a new jar 
using the addJar API.

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to