[
https://issues.apache.org/jira/browse/TEZ-2137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14367764#comment-14367764
]
Hitesh Shah commented on TEZ-2137:
----------------------------------
At this point, these counters are only useful at a task level. We should look
at how to make these counters usable at the vertex level.
Current counter aggregation ( which does a simple sum ) is next to useless for
timestamp information.
Ver few users are likely to dig into counters at each task level but most will
look more at aggregates at the DAG and vertex level. Only the analyser will
likely make use of these counters at the granular level.
Given the above, how much memory footprint are we adding for these new
counters? Should we have better logic on how to keep memory in check even as we
add more and more information at the task level ( needed only for later deep
analysis )?
\cc [~bikassaha]
> Add task counter to understand sorter final merge time
> ------------------------------------------------------
>
> Key: TEZ-2137
> URL: https://issues.apache.org/jira/browse/TEZ-2137
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Attachments: TEZ-2137.1.patch, TEZ-2137.2.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
