Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r206710204 --- Diff: core/src/main/scala/org/apache/spark/BarrierTaskContext.scala --- @@ -27,6 +27,33 @@ trait BarrierTaskContext extends TaskContext { * Sets a global barrier and waits until all tasks in this stage hit this barrier. Similar to * MPI_Barrier function in MPI, the barrier() function call blocks until all tasks in the same * stage have reached this routine. + * + * This function is expected to be called by EVERY tasks in the same barrier stage in the SAME + * pattern, otherwise you may get a SparkException. Some examples of misuses listed below: + * 1. Only call barrier() function on a subset of all the tasks in the same barrier stage, it + * shall lead to time out of the function call. + * rdd.barrier().mapPartitions { (iter, context) => --- End diff -- This won't be rendered correctly in ScalaDoc/JavaDoc. See https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/functions.scala#L160.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org