Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4510#discussion_r24627100
--- Diff:
core/src/test/scala/org/apache/spark/ui/jobs/JobProgressListenerSuite.scala ---
@@ -290,8 +291,11 @@ class JobProgressListenerSuite extends FunSuite with
LocalSparkContext with Matc
stage0Data = listener.stageIdToData.get((0, 0)).get
stage1Data = listener.stageIdToData.get((1, 0)).get
- assert(stage0Data.shuffleReadBytes == 402)
- assert(stage1Data.shuffleReadBytes == 602)
+ // Task 1235 contributed (100+1)+(100+9) = 210 shuffle bytes, and task
1234 contributed
+ // (300+1)+(300+9) = 610 total shuffle bytes, so the total for the
stage is 820.
+ assert(stage0Data.shuffleReadTotalBytes == 820)
+ // Task 1236 contributed 410 shuffle bytes, and task 1237 contributed
810 shuffle bytes.
+ assert(stage1Data.shuffleReadTotalBytes == 1220)
--- End diff --
hm these should all have `===`, but we can fix this outside of this patch
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
