Sahil Takiar has uploaded a new change for review.

Change subject: IMPALA-4232: qgen: Hive does not support aggregates inside 
specific analytic clauses

IMPALA-4232: qgen: Hive does not support aggregates inside specific analytic 

Hive does not support aggregates inside the following analytic clauses:

* AVG ... OVER
* LAG ... OVER
* MAX ... OVER
* MIN ... OVER
* SUM ... OVER

This patch modifies the qgen code so that it doesn't create aggregates inside 
the above
analytic clauses, if the HiveProfile is used. A new method called
get_analytics_cannot_contain_aggs() is added to the DefaultProfile and to the
HiveProfile. The implementation in the DefaultProfile returns an empty list. The
implementation in the HiveProfile returns the list of methods above. The
QueryGenerator._create_agg_or_analytic_tree() method is modified so that it 
checks the
get_analytics_cannot_contain_aggs method when populating the possible function 
types of
the next child in the function tree. If it finds any of these functions already 
exist in
the tree, it ensures that the next child function cannot be an aggregate 

Misc Changes:

A few miscellaneous changes were made that popped up during testing:

* Fixed a possible NPE in _create_boolean_func_tree
* Fixed a possible NPE in _populate_func_with_vals
* Disabled ONLY_USE_EQUALITY_JOIN_PREDICATES for the HiveProfile, this should 
have been
done in IMPALA-4101; Hive doesn't support all equality-joins, so specific types 
need to
be disabled, see IMPALA-4101 for more details


* Unit tests added: and
* All unit tests pass
* Tested locally against Hive
* Tested against Impala via Leopard
* Tested against Impala via the discrepancy searcher

Change-Id: Ie1096c4cde7ea52a52b39e31cd93242da53b549f
M tests/comparison/
M tests/comparison/
A tests/comparison/tests/hive/
A tests/comparison/tests/
4 files changed, 192 insertions(+), 9 deletions(-)

  git pull ssh:// refs/changes/81/4581/1
To view, visit
To unsubscribe, visit

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie1096c4cde7ea52a52b39e31cd93242da53b549f
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Sahil Takiar <>

Reply via email to