Eric Payne created TEZ-3476:

             Summary: Need a way to account for container localization.
                 Key: TEZ-3476
             Project: Apache Tez
          Issue Type: Bug
            Reporter: Eric Payne

Tez task attempt start times don't reflect time spent in localization.

In the MapReduce framework, the time spent in localization was included in the 
total runtime of each task attempt. But since Tez reuses containers, the time 
spent localizing for a container is not captured. The start time of the first 
attempt in that container will only be set after the localization has completed.

The result is that attempts can appear as if they are not being run even though 
there are resources available in the queue. An attempt can be assigned to a 
container, but if the container is on a slow node and it takes a long time to 
localize, the attempt state will remain pending until localization completes.

The impact risk is that tasks will not speculate during localization since they 
haven't started

This message was sent by Atlassian JIRA

Reply via email to