[
https://issues.apache.org/jira/browse/TEZ-1610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240850#comment-14240850
]
Siddharth Seth commented on TEZ-1610:
-------------------------------------
Looks good. Nits - up to you if you want to address them.
- startTime should be final in Shuffle
- In both InputEventHandlers, can updateEventReceivedTime() just be called when
handleEvent / processDataMovementEvent is invoked. One more thing to consider
is that the event processing takes some time - so events may actual have come
in (batch), but would be processed one at a time (lightweight operations). Is
the counter time meant to reflect when the event was received or when the event
was processed (practically they'll be the same).
- In ShuffleScheduler and ShuffleManager - is this check required ? "if
(lastEventReceived.getValue() < relativeTime)".
> additional task counters for fetchers
> -------------------------------------
>
> Key: TEZ-1610
> URL: https://issues.apache.org/jira/browse/TEZ-1610
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Attachments: TEZ-1610.1.patch, TEZ-1610.2.patch, TEZ-1610.4.patch
>
>
> - ShuffleFinishTime (per source)
> - Merge time (depending on broadcast/scatter-gather shuffle)
> This would be helpful in determining when shuffle started/ended for different
> sources in a task.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
