JiexingLi commented on code in PR #38371:
URL: https://github.com/apache/spark/pull/38371#discussion_r1009014221
##########
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala:
##########
@@ -3089,13 +3089,14 @@ class DAGSchedulerSuite extends SparkFunSuite with
TempLocalSparkContext with Ti
submit(finalRdd, Array(0, 1), properties = new Properties())
// Finish the first 2 shuffle map stages.
- completeShuffleMapStageSuccessfully(0, 0, 2)
+ completeShuffleMapStageSuccessfully(0, 0, 2, Seq("hostA", "hostB"))
Review Comment:
Yes. The background is that: Wenchen and I were reading the related code,
but felt confused about why line 3098 says: "Executor lost on hostB, both of
stage 0 and 1 should be reran.".
After digging around, we found that stage0 completes in "hostA", "hostB"
(the default hosts somewhere in the code). But it is not very obvious for
reader to tell this.
So adding Seq("hostA", "hostB") to reader to easily get this. I further
updated the comments too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
