-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71820/
-----------------------------------------------------------
(Updated Nov. 27, 2019, 10:20 a.m.)
Review request for hive, Jesús Camacho Rodríguez and Zoltan Haindrich.
Bugs: HIVE-20150
https://issues.apache.org/jira/browse/HIVE-20150
Repository: hive-git
Description (updated)
-------
TopNKey pushdown
================
1. Apply patch:
https://issues.apache.org/jira/secure/attachment/12941630/HIVE-20150.11.patch
2. TopNKey introduction depends only from Reduce Sink with topn property >= 0
3. Implement TopNKey operator pushdown through: projection, group by, redeuce
sink, left outer join, other topnkey
4. Add sort order and null sort order direction check when determining if the
topnkey op can be pushed
5. Implement handling cases when topnkey op and the parent op has a common key
prefix only.
6. turn off topnkey optimization by default
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 4393a2825e
ql/src/java/org/apache/hadoop/hive/ql/optimizer/CommonKeyPrefix.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/optimizer/TopNKeyProcessor.java
4b4cf99503
ql/src/java/org/apache/hadoop/hive/ql/optimizer/TopNKeyPushdownProcessor.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java bf58bd8bb8
ql/src/test/org/apache/hadoop/hive/ql/optimizer/TestCommonKeyPrefix.java
PRE-CREATION
ql/src/test/queries/clientpositive/topnkey.q 057b6a45ba
ql/src/test/queries/clientpositive/vector_topnkey.q 85c5880cd6
ql/src/test/results/clientpositive/llap/bucket_groupby.q.out 0c051c926b
ql/src/test/results/clientpositive/llap/check_constraint.q.out 9f2c9a1cd0
ql/src/test/results/clientpositive/llap/constraints_optimization.q.out
b6d210becf
ql/src/test/results/clientpositive/llap/enforce_constraint_notnull.q.out
9343e078b7
ql/src/test/results/clientpositive/llap/explainuser_1.q.out 735296f814
ql/src/test/results/clientpositive/llap/explainuser_2.q.out b90fb55fd1
ql/src/test/results/clientpositive/llap/external_jdbc_table_perf.q.out
545cce75a9
ql/src/test/results/clientpositive/llap/limit_pushdown.q.out 3fdd77d802
ql/src/test/results/clientpositive/llap/limit_pushdown3.q.out efa8c38d7c
ql/src/test/results/clientpositive/llap/offset_limit.q.out 23f2de46e5
ql/src/test/results/clientpositive/llap/offset_limit_ppd_optimizer.q.out
4ecb7bc46d
ql/src/test/results/clientpositive/llap/orc_struct_type_vectorization.q.out
0eac389eb7
ql/src/test/results/clientpositive/llap/parquet_complex_types_vectorization.q.out
4362fb6f2e
ql/src/test/results/clientpositive/llap/parquet_map_type_vectorization.q.out
24468c9a1b
ql/src/test/results/clientpositive/llap/parquet_struct_type_vectorization.q.out
45890a1890
ql/src/test/results/clientpositive/llap/topnkey.q.out 1e77587f82
ql/src/test/results/clientpositive/llap/vector_cast_constant.q.out cc2dc47280
ql/src/test/results/clientpositive/llap/vector_char_2.q.out f7e76e5a8b
ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_limit.q.out
6fd15e7101
ql/src/test/results/clientpositive/llap/vector_groupby_reduce.q.out
d6325982e3
ql/src/test/results/clientpositive/llap/vector_string_concat.q.out a8019be7aa
ql/src/test/results/clientpositive/llap/vector_topnkey.q.out c140bdfd37
ql/src/test/results/clientpositive/llap/vectorization_limit.q.out 7326adf522
ql/src/test/results/clientpositive/perf/tez/query10.q.out 30628f4829
ql/src/test/results/clientpositive/perf/tez/query15.q.out 418bffddca
ql/src/test/results/clientpositive/perf/tez/query17.q.out e7eda149e8
ql/src/test/results/clientpositive/perf/tez/query27.q.out d915989654
ql/src/test/results/clientpositive/perf/tez/query35.q.out 1936f22b61
ql/src/test/results/clientpositive/perf/tez/query40.q.out 61f3d7c748
ql/src/test/results/clientpositive/perf/tez/query43.q.out 2e6f86c0a0
ql/src/test/results/clientpositive/perf/tez/query45.q.out ab7181d7a4
ql/src/test/results/clientpositive/perf/tez/query50.q.out 2a287ce6d3
ql/src/test/results/clientpositive/perf/tez/query66.q.out 23ffd637a3
ql/src/test/results/clientpositive/perf/tez/query69.q.out d27f4060f6
ql/src/test/results/clientpositive/tez/topnkey.q.out 45947d0be8
ql/src/test/results/clientpositive/tez/vector_topnkey.q.out d179013e28
ql/src/test/results/clientpositive/topnkey.q.out cecbe89b1c
Diff: https://reviews.apache.org/r/71820/diff/2/
Changes: https://reviews.apache.org/r/71820/diff/1-2/
Testing
-------
Run q tests using TestMiniLlapLocalCliDriver
topnkey.q
vector_topnkey.q
Thanks,
Krisztian Kasa
