[
https://issues.apache.org/jira/browse/TEZ-14?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14101322#comment-14101322
]
Chen He edited comment on TEZ-14 at 8/20/14 4:00 AM:
-----------------------------------------------------
Mantri considers the network traffic when it chooses the node which runs the
speculative task. It is good at that time. However, right now, for a very large
data center, network traffic changes frequently. Maybe we can employ the
Software Defined Networks here, to do the network traffic estimation.
https://www.opennetworking.org/sdn-resources/sdn-library/whitepapers
Or maybe introduce that layer in Yarn and Tez speculative tasks can get network
info from Yarn. Just a spark of idea.
was (Author: airbots):
Mantri considers the network traffic when it chooses the node which runs the
speculative task. It is good at that time. However, right now, for a very large
data center, network traffic changes frequently. Maybe we can employ the
Software Defined Networks here, to do the network traffic estimation.
> Support for speculation of slow tasks
> -------------------------------------
>
> Key: TEZ-14
> URL: https://issues.apache.org/jira/browse/TEZ-14
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Bikas Saha
> Assignee: Bikas Saha
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
