Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21639#discussion_r198210412
--- Diff: core/src/main/scala/org/apache/spark/TestUtils.scala ---
@@ -173,21 +173,23 @@ private[spark] object TestUtils {
* Run some code involving jobs submitted to the given context and
assert that the jobs spilled.
*/
def assertSpilled[T](sc: SparkContext, identifier: String)(body: => T):
Unit = {
- val spillListener = new SpillListener
- sc.addSparkListener(spillListener)
- body
- assert(spillListener.numSpilledStages > 0, s"expected $identifier to
spill, but did not")
+ withListener(sc, new SpillListener) { listener =>
+ val ret = body
--- End diff --
I saw the return type in the closure, but the method itself returns `Unit`,
so all that can be cleaned up.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
