vanzin commented on a change in pull request #25299: [SPARK-27651][Core] Avoid
the network when shuffle blocks are fetched from the same host
URL: https://github.com/apache/spark/pull/25299#discussion_r308937544
##########
File path: core/src/main/scala/org/apache/spark/status/api/v1/api.scala
##########
@@ -344,6 +348,7 @@ class ShuffleReadMetricDistributions private[spark](
val readRecords: IndexedSeq[Double],
val remoteBlocksFetched: IndexedSeq[Double],
val localBlocksFetched: IndexedSeq[Double],
+ val hostLocalBlocksFetched: IndexedSeq[Double],
Review comment:
Hmmm.
I wonder what happens if:
- you run the current version of the SHS with the disk enabled
- look at an app's stage in the UI, which will cache this information
- shut down the SHS, and bring up the version with this change
- the data in the disk store doesn't have this field, so what will happen?
My feeling is that this field will be `null`, which may cause some problems.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
