[
https://issues.apache.org/jira/browse/TEZ-1610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Balamohan updated TEZ-1610:
----------------------------------
Attachment: TEZ-1610.2.patch
Attaching v2 of the patch which has the following counters.
{code}
/**
* Total time available to fetchers in the task.
* (ie. total shuffle time in task * number of fetchers)
*/
FETCHER_TIME_MILLIS,
/**
* Total time spent by all fetchers in a task in downloading shuffle data
* This is a cumulative time of all fetchers in the task
*/
FETCHER_TIME_MILLIS_IN_SHUFFLE,
/**
* Time taken to merge shuffled data after the fetchers have been shutdown.
* Represented in milliseconds.
*/
MERGED_INPUT_READY_DELTA,
/**
* Timestamp of the event seen in shuffle side
*/
LAST_SHUFFLE_EVENT_TIMESTAMP
{code}
> additional task counters for fetchers
> -------------------------------------
>
> Key: TEZ-1610
> URL: https://issues.apache.org/jira/browse/TEZ-1610
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Attachments: TEZ-1610.1.patch, TEZ-1610.2.patch
>
>
> - ShuffleFinishTime (per source)
> - Merge time (depending on broadcast/scatter-gather shuffle)
> This would be helpful in determining when shuffle started/ended for different
> sources in a task.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
