attilapiros commented on issue #24189: [SPARK-27253][SQL] Prioritizes parent session's SQLConf over SparkConf when cloning a session URL: https://github.com/apache/spark/pull/24189#issuecomment-478124516 I have checked the changes in the resource allocation point of view and there is no resource leak. As in `SessionStateSuite.scala` the `activeSession` stopped in the `afterAll()` (which stops the underlying `SparkContext`) as the `forkedSession` session shares the `SparkContext` with the `activeSession` no separate `stop()` call is necessary. Running only the new UT test proves this as well: ``` [info] - SPARK-27253 forked new session should not discard SQLConf overrides (627 milliseconds) 20:34:39.580 WARN org.apache.spark.sql.SessionStateSuite: ===== POSSIBLE THREAD LEAK IN SUITE o.a.s.sql.SessionStateSuite, thread names: process reaper ===== ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
