Venkata Sai Akhil Gudesa created SPARK-44078:
------------------------------------------------
Summary: Add support for classloader/resource isolation
Key: SPARK-44078
URL: https://issues.apache.org/jira/browse/SPARK-44078
Project: Spark
Issue Type: New Feature
Components: Connect, Spark Core
Affects Versions: 3.5.0
Reporter: Venkata Sai Akhil Gudesa
A current limitation of Scala UDFs is that a Spark cluster would only be able
to support a single REPL at a time due to the fact that classloaders of
different Spark Sessions (and therefore, Spark Connect sessions) aren't
isolated from each other. Without isolation, REPL-generated classfiles as well
as user-added JARs may conflict if there are multiple users of the cluster.
Thus, we need a mechanism to support isolated sessions (i.e isolated
resources/classloader) so that each REPL user does not conflict with other
users on the same cluster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
