[ 
https://issues.apache.org/jira/browse/HIVE-30261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Krisztian Kasa updated HIVE-30261:
----------------------------------
    Description: 
In a star join followed by a GROUP BY, the fact table rows go through the 
shuffle and the join before any aggregation happens. When the join keys and the 
GROUP BY keys have few distinct values on the fact side, the rows could be 
partially aggregated below the join first. Then far fewer rows are shuffled, 
joined and aggregated.

Criteria to meet for push-down:
* The join is an inner join. Every input to the join is a ReduceSink operator, 
and for an n-way join, every condition is an inner join.
* The GROUP BY is above the join with only Select operators between them. It 
has keys, so a full aggregate cannot be pushed down.
* GROUP BYs with grouping sets and DISTINCT aggregations are not supported.
* Only SUM, $SUM0, COUNT, MIN, and MAX aggregate functions are supported. These 
are the ones that support partial aggregation.
* Every key and every argument is deterministic and references at most one join 
input.
* Every aggregate function argument references the same join input, and the 
GROUP BY is pushed down to that input.
* If no argument has columns (e.g., COUNT(\*)), the GROUP BY is pushed down to 
the larger join input.


  was:
In a star join followed by a GROUP BY, the fact table rows go through the 
shuffle and the join before any aggregation happens. When the join keys and the 
GROUP BY keys have few distinct values on the fact side, the rows could be 
partially aggregated below the join first. Then far fewer rows are shuffled, 
joined and aggregated.

Criteria to meet for push-down:
* The join is an inner join. Every input to the join is a ReduceSink operator, 
and for an n-way join, every condition is an inner join.
* The GROUP BY is above the join with only Select operators between them. It 
has keys, so a full aggregate cannot be pushed down.
* GROUP BYs with grouping sets and DISTINCT aggregations are not supported.
* Only SUM, $SUM0, COUNT, MIN, and MAX aggregate functions are supported. These 
are the ones that support partial aggregation.
* Every key and every argument is deterministic and references at most one join 
input.
* Every aggregate function argument references the same join input, and the 
GROUP BY is pushed down to that input.
* If no argument has columns (e.g., COUNT(*)), the GROUP BY is pushed down to 
the larger join input.



> Push a partial aggregation below an inner join
> ----------------------------------------------
>
>                 Key: HIVE-30261
>                 URL: https://issues.apache.org/jira/browse/HIVE-30261
>             Project: Hive
>          Issue Type: Improvement
>          Components: Physical Optimizer
>            Reporter: Krisztian Kasa
>            Assignee: Krisztian Kasa
>            Priority: Major
>
> In a star join followed by a GROUP BY, the fact table rows go through the 
> shuffle and the join before any aggregation happens. When the join keys and 
> the GROUP BY keys have few distinct values on the fact side, the rows could 
> be partially aggregated below the join first. Then far fewer rows are 
> shuffled, joined and aggregated.
> Criteria to meet for push-down:
> * The join is an inner join. Every input to the join is a ReduceSink 
> operator, and for an n-way join, every condition is an inner join.
> * The GROUP BY is above the join with only Select operators between them. It 
> has keys, so a full aggregate cannot be pushed down.
> * GROUP BYs with grouping sets and DISTINCT aggregations are not supported.
> * Only SUM, $SUM0, COUNT, MIN, and MAX aggregate functions are supported. 
> These are the ones that support partial aggregation.
> * Every key and every argument is deterministic and references at most one 
> join input.
> * Every aggregate function argument references the same join input, and the 
> GROUP BY is pushed down to that input.
> * If no argument has columns (e.g., COUNT(\*)), the GROUP BY is pushed down 
> to the larger join input.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to