Pulkit Sharma created HIVE-22312:
------------------------------------
Summary: MapJoinCounterHook doesnot work for tez
Key: HIVE-22312
URL: https://issues.apache.org/jira/browse/HIVE-22312
Project: Hive
Issue Type: Bug
Components: Hive
Affects Versions: All Versions
Reporter: Pulkit Sharma
In [HIVE-1792|https://issues.apache.org/jira/browse/HIVE-1792],
MapJoinCounterHook hook was added to track joins that get converted to map
join. This hook gets List of Tasks from hookContext and check Tag associated
with each task. For mr, we create Conditional tasks in case of joins and add
tags for respective join conversions. This does not work in case of tez as we
only create TezTask(no Conditional Task is created) which can handle multiple
joins in contrast to one Conditional Task per join in mr.
The current approach will fail even if we add tag to TezTask as it can have
multiple joins of the same type which will require counter.
One possible solution for tez, is to parse query-plan after query completion
which we get from hookContext to get workGraph. Using workGraph, we can walk
through Operator Tree to find join conversions.
If this approach looks good, I can raise Pull Request
cc [~ashutoshc] [~jcamachorodriguez] [~pxiong]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
