gaborgsomogyi commented on issue #25709: [SPARK-29007][STREAMING][MLLIB] Enforce not leaking SparkContext in tests which creates new StreamingContext with new SparkContext URL: https://github.com/apache/spark/pull/25709#issuecomment-529363320 I think the question is the granularity of defensiveness. In the Structured Streaming area there were huge execution times because each and every test has created its own `SparkSession`. That's the main reason why `SharedSparkSession` was introduced. In some cases here we're switching to per test `StreamingContext` creation. Do we have a rough number what does that mean in terms of execution time?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
