Eric Payne created TEZ-3476:
-------------------------------
Summary: Need a way to account for container localization.
Key: TEZ-3476
URL: https://issues.apache.org/jira/browse/TEZ-3476
Project: Apache Tez
Issue Type: Bug
Reporter: Eric Payne
Tez task attempt start times don't reflect time spent in localization.
In the MapReduce framework, the time spent in localization was included in the
total runtime of each task attempt. But since Tez reuses containers, the time
spent localizing for a container is not captured. The start time of the first
attempt in that container will only be set after the localization has completed.
The result is that attempts can appear as if they are not being run even though
there are resources available in the queue. An attempt can be assigned to a
container, but if the container is on a slow node and it takes a long time to
localize, the attempt state will remain pending until localization completes.
The impact risk is that tasks will not speculate during localization since they
haven't started
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
