wypoon commented on a change in pull request #23767: [SPARK-26329][CORE] Faster
polling of executor memory metrics.
URL: https://github.com/apache/spark/pull/23767#discussion_r309337589
##########
File path: core/src/test/scala/org/apache/spark/executor/ExecutorSuite.scala
##########
@@ -421,10 +418,9 @@ class ExecutorSuite extends SparkFunSuite
private def createResultTaskDescription(
serializer: SerializerInstance,
- resultFunc: (TaskContext, Iterator[Int]) => Int,
+ taskBinary: Broadcast[Array[Byte]],
rdd: RDD[Int],
stageId: Int): TaskDescription = {
- val taskBinary = sc.broadcast(serializer.serialize((rdd,
resultFunc)).array())
Review comment:
I don't actually understand why, but when the call to `sc.broadcast` happens
inside the helper, there were some problems with the broadcast. If the entire
`ExecutorSuite` were run, all the tests would pass. However, if I ran each test
in it individually, then the tests that used this helper would fail, due to
said problem.
With this change, all the tests in the suite pass whether run together or
individually.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
