HeartSaVioR commented on issue #25709: [SPARK-29007][STREAMING][MLLIB] Enforce not leaking SparkContext in tests which creates new StreamingContext with new SparkContext URL: https://github.com/apache/spark/pull/25709#issuecomment-529387575 Unfortunately no, I don't have the number, and actually I expect these numbers would fluctuate unless we have some way to correctly measure it. Both LocalJavaStreamingContext and LocalSparkContext deal with cleanup per test - I agree it would be generally slower then before if test suites deal with cleanup per suite, but that's also obvious that it would be safer. So that sounds me as preference or policy. Let's hear about further opinions - I'm open to roll back and just apply withStreamingContext or even call static function for these suites.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
