[
https://issues.apache.org/jira/browse/TEZ-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16270638#comment-16270638
]
Rajesh Balamohan commented on TEZ-3805:
---------------------------------------
In case of hive with LLAP, I observed misses for a specific query which was
being benchmarked. This analyzer got created to nail down such issues without
having to dig down the logs often. Finally, it turned out that Hive LLAP task
scheduler does not add location hints and that was causing all sorts of issues.
This analyzer helped in creating
https://issues.apache.org/jira/browse/HIVE-17180 to add "location hints" in
Hive/LLAP task scheduler.
For regular tez jobs with Hive, I haven't come across yet with scenarios
wherein scheduler would miss locality.
> Analyzer: Add an analyzer to find out scheduling misses in 1:1 edges
> --------------------------------------------------------------------
>
> Key: TEZ-3805
> URL: https://issues.apache.org/jira/browse/TEZ-3805
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Fix For: 0.9.1
>
> Attachments: TEZ-3805.1.patch
>
>
> When 1:1 edge is used, it would be helpful to find out whether downstream
> tasks ran on the same location provided in the hints by the runtime.
> One of the recent feature in upstream project (hive) used 1:1 edge. Instead
> of checking the logs, it would be useful to have an analyzer to churn out the
> details.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
