[jira] [Updated] (TEZ-3816) Ability to automatically speculate single-task vertices
[ https://issues.apache.org/jira/browse/TEZ-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Muhammad Samir Khan updated TEZ-3816: - Attachment: TEZ-3816.003.patch Added a couple of comments and speculationValue returns ON_SCHEDULE if the single task has not timed out yet. > Ability to automatically speculate single-task vertices > --- > > Key: TEZ-3816 > URL: https://issues.apache.org/jira/browse/TEZ-3816 > Project: Apache Tez > Issue Type: Improvement >Reporter: Muhammad Samir Khan >Assignee: Muhammad Samir Khan > Attachments: TEZ-3816.001.patch, TEZ-3816.002.patch, > TEZ-3816.003.patch > > > When a single-task vertex is unlucky, it lands on a very slow node. > Speculation doesn't currently apply when there are no other tasks to compare > with. It would be good to have a configurable timeout after which the tasks > automatically speculate. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (TEZ-3816) Ability to automatically speculate single-task vertices
[ https://issues.apache.org/jira/browse/TEZ-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Muhammad Samir Khan updated TEZ-3816: - Attachment: TEZ-3816.002.patch Thanks [~sseth] for the comments. I updated the patch to put the logic in LegacySpeculator. Also added a unit test. If there is an existing speculation attempt, then speculationValue() will return ALREADY_SPECULATING which will prevent more speculations. Tested the updated patch with mrrsleep. > Ability to automatically speculate single-task vertices > --- > > Key: TEZ-3816 > URL: https://issues.apache.org/jira/browse/TEZ-3816 > Project: Apache Tez > Issue Type: Improvement >Reporter: Muhammad Samir Khan >Assignee: Muhammad Samir Khan > Attachments: TEZ-3816.001.patch, TEZ-3816.002.patch > > > When a single-task vertex is unlucky, it lands on a very slow node. > Speculation doesn't currently apply when there are no other tasks to compare > with. It would be good to have a configurable timeout after which the tasks > automatically speculate. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (TEZ-3816) Ability to automatically speculate single-task vertices
[ https://issues.apache.org/jira/browse/TEZ-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Muhammad Samir Khan updated TEZ-3816: - Description: When a single-task vertex is unlucky, it lands on a very slow node. Speculation doesn't currently apply when there are no other tasks to compare with. It would be good to have a configurable timeout after which the tasks automatically speculate. (was: When a single-task vertex is unlucky, it lands on a very slow node. Speculation doesn't currently apply when there are no other tasks to compare with. It would be good to either have a configurable timeout after which the tasks automatically speculate.) > Ability to automatically speculate single-task vertices > --- > > Key: TEZ-3816 > URL: https://issues.apache.org/jira/browse/TEZ-3816 > Project: Apache Tez > Issue Type: Improvement >Reporter: Muhammad Samir Khan >Assignee: Muhammad Samir Khan > Attachments: TEZ-3816.001.patch > > > When a single-task vertex is unlucky, it lands on a very slow node. > Speculation doesn't currently apply when there are no other tasks to compare > with. It would be good to have a configurable timeout after which the tasks > automatically speculate. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (TEZ-3816) Ability to automatically speculate single-task vertices
[ https://issues.apache.org/jira/browse/TEZ-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Muhammad Samir Khan updated TEZ-3816: - Attachment: TEZ-3816.001.patch Added ability to LegacyTaskRuntimeEstimator to speculate based on timeout for single task vertices. > Ability to automatically speculate single-task vertices > --- > > Key: TEZ-3816 > URL: https://issues.apache.org/jira/browse/TEZ-3816 > Project: Apache Tez > Issue Type: Improvement >Reporter: Muhammad Samir Khan >Assignee: Muhammad Samir Khan > Attachments: TEZ-3816.001.patch > > > When a single-task vertex is unlucky, it lands on a very slow node. > Speculation doesn't currently apply when there are no other tasks to compare > with. It would be good to either have a configurable timeout after which the > tasks automatically speculate. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
