[
https://issues.apache.org/jira/browse/SPARK-24817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16567301#comment-16567301
]
Erik Erlandson commented on SPARK-24817:
----------------------------------------
Thanks [~jiangxb] - I'd expect that design to work out-of-box on the k8s
backend.
ML-specific code seems like it will have needs that are harder to predict, by
definition. If it can use IP addresses in the cluster space, it should work
regardless. If it wants fqdn, then perhaps additional pod configurations will
be required.
> Implement BarrierTaskContext.barrier()
> --------------------------------------
>
> Key: SPARK-24817
> URL: https://issues.apache.org/jira/browse/SPARK-24817
> Project: Spark
> Issue Type: New Feature
> Components: Spark Core
> Affects Versions: 2.4.0
> Reporter: Jiang Xingbo
> Priority: Major
>
> Implement BarrierTaskContext.barrier(), to support global sync between all
> the tasks in a barrier stage. The global sync shall finish immediately once
> all tasks in the same barrier stage reaches the same barrier.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
