[
https://issues.apache.org/jira/browse/TEZ-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kuhu Shukla updated TEZ-3476:
-----------------------------
Attachment: TEZ-3476.001.patch
> Need a way to account for container localization.
> -------------------------------------------------
>
> Key: TEZ-3476
> URL: https://issues.apache.org/jira/browse/TEZ-3476
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Eric Payne
> Assignee: Kuhu Shukla
> Priority: Major
> Attachments: TEZ-3476.001.patch
>
>
> Tez task attempt start times don't reflect time spent in localization.
> In the MapReduce framework, the time spent in localization was included in
> the total runtime of each task attempt. But since Tez reuses containers, the
> time spent localizing for a container is not captured. The start time of the
> first attempt in that container will only be set after the localization has
> completed.
> The result is that attempts can appear as if they are not being run even
> though there are resources available in the queue. An attempt can be assigned
> to a container, but if the container is on a slow node and it takes a long
> time to localize, the attempt state will remain pending until localization
> completes.
> The impact risk is that tasks will not speculate during localization since
> they haven't started
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
