Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/21898#discussion_r206716080
--- Diff: core/src/main/scala/org/apache/spark/BarrierTaskContextImpl.scala
---
@@ -39,8 +44,53 @@ private[spark] class BarrierTaskContextImpl(
taskMemoryManager, localProperties, metricsSystem, taskMetrics)
with BarrierTaskContext {
- // TODO SPARK-24817 implement global barrier.
- override def barrier(): Unit = {}
+ private val barrierCoordinator: RpcEndpointRef = {
+ val env = SparkEnv.get
+ RpcUtils.makeDriverRef("barrierSync", env.conf, env.rpcEnv)
+ }
+
+ private val timer = new Timer("Barrier task timer for barrier() calls.")
+
+ private var barrierEpoch = 0
+
+ private lazy val numTasks = localProperties.getProperty("numTasks",
"0").toInt
--- End diff --
Shall we use `getTaskInfos().size` instead?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
