[
https://issues.apache.org/jira/browse/SPARK-25376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-25376:
----------------------------------
Description:
[~irashid] You mentioned that there are couple scenarios we should handle in
barrier execution mode but we didn't in 2.4. Could you elaborate here?
One scenario we are aware of is that speculation is not supported by barrier
mode. Hence a barrier mode might hang in case of hardware issues on one node. I
don't have a good proposal here except letting users set a timeout for the
barrier stage. Would like to hear your thoughts.
You also mentioned multi-tenancy issues. Could you say more?
cc: [~jiangxb1987]
was:
[~irashid] You mentioned that there are couple scenarios we should handle in
barrier execution mode but we didn't in 2.4. Could you elaborate here?
cc: [~jiangxb1987]
> Scenarios we should handle but missed for barrier execution mode
> ----------------------------------------------------------------
>
> Key: SPARK-25376
> URL: https://issues.apache.org/jira/browse/SPARK-25376
> Project: Spark
> Issue Type: Story
> Components: Spark Core
> Affects Versions: 2.4.0
> Reporter: Xiangrui Meng
> Priority: Major
>
> [~irashid] You mentioned that there are couple scenarios we should handle in
> barrier execution mode but we didn't in 2.4. Could you elaborate here?
> One scenario we are aware of is that speculation is not supported by barrier
> mode. Hence a barrier mode might hang in case of hardware issues on one node.
> I don't have a good proposal here except letting users set a timeout for the
> barrier stage. Would like to hear your thoughts.
> You also mentioned multi-tenancy issues. Could you say more?
> cc: [~jiangxb1987]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
