Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/12113#discussion_r58253255
--- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala ---
@@ -477,12 +605,16 @@ private[spark] class MapOutputTrackerWorker(conf:
SparkConf) extends MapOutputTr
private[spark] object MapOutputTracker extends Logging {
val ENDPOINT_NAME = "MapOutputTracker"
+ private val DIRECT = 0
+ private val BROADCAST = 1
// Serialize an array of map output locations into an efficient byte
format so that we can send
// it to reduce tasks. We do this by compressing the serialized bytes
using GZIP. They will
// generally be pretty compressible because many map outputs will be on
the same hostname.
- def serializeMapStatuses(statuses: Array[MapStatus]): Array[Byte] = {
+ def serializeMapStatuses(statuses: Array[MapStatus], broadcastManager:
BroadcastManager,
+ isLocal: Boolean, minBroadcastSize: Int): (Array[Byte],
Broadcast[Array[Byte]]) = {
--- End diff --
Both values are used in getSerializedMapOutputStatuses
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
