[
https://issues.apache.org/jira/browse/HIVE-30276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-30276:
----------------------------------
Description:
A partial aggregate is pushed below a join only if it is expected to reduce its
input. The current check is per aggregate key: the push is skipped if
{code}
[NDV pushed key] > hive.optimize.partial.aggr.join.transpose.max.key.ndv.ratio
* [input row count]
{code}
is true for any single pushed key.
The number of groups depends on the NDV of all the keys together. With several
keys, each key can pass the check on its own while the combination is close to
unique. The pushed GROUP BY then aggregates almost nothing: it pays the hash
aggregation cost and outputs about as many rows as it reads.
> Partial GROUP BY join transpose: check the combined NDV of the pushed-down
> aggregate keys
> -----------------------------------------------------------------------------------------
>
> Key: HIVE-30276
> URL: https://issues.apache.org/jira/browse/HIVE-30276
> Project: Hive
> Issue Type: Sub-task
> Components: Physical Optimizer
> Reporter: Krisztian Kasa
> Priority: Major
>
> A partial aggregate is pushed below a join only if it is expected to reduce
> its input. The current check is per aggregate key: the push is skipped if
> {code}
> [NDV pushed key] >
> hive.optimize.partial.aggr.join.transpose.max.key.ndv.ratio * [input row
> count]
> {code}
> is true for any single pushed key.
> The number of groups depends on the NDV of all the keys together. With
> several keys, each key can pass the check on its own while the combination is
> close to unique. The pushed GROUP BY then aggregates almost nothing: it pays
> the hash aggregation cost and outputs about as many rows as it reads.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)