[
https://issues.apache.org/jira/browse/TEZ-14?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14216609#comment-14216609
]
Siddharth Seth commented on TEZ-14:
-----------------------------------
bq. There are multiple reasons I havent made this change. Its not yet clear
that this is an issue. This only adds 1 new event per physically running task.
n events - one for each update from a task, and tasks are the largest number of
entities in the system. The separation of TaskAttempts interactive with
Vertices directly is definitely broken IMO.
Whether the speculator belongs in the Vertex itself is debatable. Wondering if
there's any use of having access to data across vertices - at least information
on what percentage is scheduled etc. This is going to evolve.
It should be fairly simple to setup an EventHandler for Speculation events
(still via the same dispatcher thread) - or just send them via the AppContext
(synchronous and hand-off) to the speculator(s).
I don't think TEZ-93 needs to be a pre-requisite for this. Agree with Hitesh on
TEZ-93 being a 0.7 item rather than 0.6. Can be fairly destabilizing,
especially if we start using multiple dispatchers for the current flow.
> Support for speculation of slow tasks
> -------------------------------------
>
> Key: TEZ-14
> URL: https://issues.apache.org/jira/browse/TEZ-14
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Bikas Saha
> Assignee: Bikas Saha
> Attachments: TEZ-14.1.patch, TEZ-14.2.patch, TEZ-14.3.patch,
> TEZ-14.4.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
