xupefei commented on code in PR #48320:
URL: https://github.com/apache/spark/pull/48320#discussion_r1793356682
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala:
##########
@@ -210,7 +210,9 @@ abstract class StreamExecution(
this, s"spark.streaming.${Option(name).getOrElse(id)}")
/** Isolated spark session to run the batches with. */
- private val sparkSessionForStream = sparkSession.cloneSession()
+ private val sparkSessionForStream = {
+ sparkSession.cloneSession(reuseArtifactManager = true)
Review Comment:
Let's get rid of this optimization to simplify the logic. If we see
performance problems in streaming, we could rethink this again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
