-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71820/
-----------------------------------------------------------

(Updated Nov. 29, 2019, 1:27 p.m.)


Review request for hive, Jesús Camacho Rodríguez and Zoltan Haindrich.


Bugs: HIVE-20150
    https://issues.apache.org/jira/browse/HIVE-20150


Repository: hive-git


Description
-------

TopNKey pushdown
================
1. Apply patch: 
https://issues.apache.org/jira/secure/attachment/12941630/HIVE-20150.11.patch
2. TopNKey introduction depends only from Reduce Sink with topn property >= 0
3. Implement TopNKey operator pushdown through: projection, group by, redeuce 
sink, left outer join, other topnkey
4. Add sort order and null sort order direction check when determining if the 
topnkey op can be pushed
5. Implement handling cases when topnkey op and the parent op has a common key 
prefix only.
6. turn off topnkey optimization by default


Diffs (updated)
-----

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 4393a2825e 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/CommonKeyPrefix.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/TopNKeyProcessor.java 
0d6cf3c755 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/TopNKeyPushdownProcessor.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java bf58bd8bb8 
  ql/src/test/org/apache/hadoop/hive/ql/optimizer/TestCommonKeyPrefix.java 
PRE-CREATION 
  ql/src/test/queries/clientpositive/topnkey.q 057b6a45ba 
  ql/src/test/queries/clientpositive/vector_topnkey.q 85c5880cd6 
  ql/src/test/results/clientpositive/llap/bucket_groupby.q.out 0c051c926b 
  ql/src/test/results/clientpositive/llap/check_constraint.q.out 9f2c9a1cd0 
  ql/src/test/results/clientpositive/llap/constraints_optimization.q.out 
b6d210becf 
  ql/src/test/results/clientpositive/llap/enforce_constraint_notnull.q.out 
9343e078b7 
  ql/src/test/results/clientpositive/llap/explainuser_1.q.out 735296f814 
  ql/src/test/results/clientpositive/llap/explainuser_2.q.out b90fb55fd1 
  ql/src/test/results/clientpositive/llap/external_jdbc_table_perf.q.out 
545cce75a9 
  ql/src/test/results/clientpositive/llap/filter_union.q.out 0df77762a0 
  ql/src/test/results/clientpositive/llap/limit_pushdown.q.out 3fdd77d802 
  ql/src/test/results/clientpositive/llap/limit_pushdown3.q.out efa8c38d7c 
  ql/src/test/results/clientpositive/llap/llap_decimal64_reader.q.out 
ffe5f6fb22 
  ql/src/test/results/clientpositive/llap/offset_limit.q.out 23f2de46e5 
  ql/src/test/results/clientpositive/llap/offset_limit_ppd_optimizer.q.out 
4ecb7bc46d 
  ql/src/test/results/clientpositive/llap/orc_struct_type_vectorization.q.out 
0eac389eb7 
  
ql/src/test/results/clientpositive/llap/parquet_complex_types_vectorization.q.out
 4362fb6f2e 
  ql/src/test/results/clientpositive/llap/parquet_map_type_vectorization.q.out 
24468c9a1b 
  
ql/src/test/results/clientpositive/llap/parquet_struct_type_vectorization.q.out 
45890a1890 
  ql/src/test/results/clientpositive/llap/semijoin_reddedup.q.out 0e9723b8f3 
  ql/src/test/results/clientpositive/llap/subquery_ALL.q.out d910c1a79d 
  ql/src/test/results/clientpositive/llap/subquery_ANY.q.out 91472d631e 
  ql/src/test/results/clientpositive/llap/topnkey.q.out 1e77587f82 
  ql/src/test/results/clientpositive/llap/vector_cast_constant.q.out cc2dc47280 
  ql/src/test/results/clientpositive/llap/vector_char_2.q.out f7e76e5a8b 
  
ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_limit.q.out
 6fd15e7101 
  ql/src/test/results/clientpositive/llap/vector_groupby_reduce.q.out 
d6325982e3 
  ql/src/test/results/clientpositive/llap/vector_mr_diff_schema_alias.q.out 
4d417b9c3d 
  ql/src/test/results/clientpositive/llap/vector_reduce_groupby_decimal.q.out 
97a211cfc6 
  ql/src/test/results/clientpositive/llap/vector_string_concat.q.out a8019be7aa 
  ql/src/test/results/clientpositive/llap/vector_topnkey.q.out c140bdfd37 
  ql/src/test/results/clientpositive/llap/vectorization_limit.q.out 7326adf522 
  ql/src/test/results/clientpositive/perf/tez/cbo_query14.q.out e9308cd709 
  ql/src/test/results/clientpositive/perf/tez/cbo_query77.q.out 02caf99f7d 
  ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query14.q.out 
43e1b2b5c2 
  ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query77.q.out 
2f75361df1 
  ql/src/test/results/clientpositive/perf/tez/constraints/query10.q.out 
7ad55b5c5f 
  ql/src/test/results/clientpositive/perf/tez/constraints/query14.q.out 
9b07ceb3a7 
  ql/src/test/results/clientpositive/perf/tez/constraints/query15.q.out 
7c0e6cf8cd 
  ql/src/test/results/clientpositive/perf/tez/constraints/query17.q.out 
44fd104a9b 
  ql/src/test/results/clientpositive/perf/tez/constraints/query25.q.out 
1ca55ca290 
  ql/src/test/results/clientpositive/perf/tez/constraints/query26.q.out 
90a56acafc 
  ql/src/test/results/clientpositive/perf/tez/constraints/query27.q.out 
58b2b86cdf 
  ql/src/test/results/clientpositive/perf/tez/constraints/query29.q.out 
073ae52f1f 
  ql/src/test/results/clientpositive/perf/tez/constraints/query35.q.out 
4ca9d3cb61 
  ql/src/test/results/clientpositive/perf/tez/constraints/query37.q.out 
83759fd6d8 
  ql/src/test/results/clientpositive/perf/tez/constraints/query40.q.out 
8dff7e8984 
  ql/src/test/results/clientpositive/perf/tez/constraints/query43.q.out 
f4ead2363c 
  ql/src/test/results/clientpositive/perf/tez/constraints/query45.q.out 
b73dc876da 
  ql/src/test/results/clientpositive/perf/tez/constraints/query49.q.out 
db3129dca0 
  ql/src/test/results/clientpositive/perf/tez/constraints/query5.q.out 
629d2ff013 
  ql/src/test/results/clientpositive/perf/tez/constraints/query50.q.out 
0e210fb5cc 
  ql/src/test/results/clientpositive/perf/tez/constraints/query60.q.out 
f2b494e706 
  ql/src/test/results/clientpositive/perf/tez/constraints/query66.q.out 
4889093f21 
  ql/src/test/results/clientpositive/perf/tez/constraints/query69.q.out 
71bb17e27a 
  ql/src/test/results/clientpositive/perf/tez/constraints/query7.q.out 
e4e28188dd 
  ql/src/test/results/clientpositive/perf/tez/constraints/query76.q.out 
08c26cf74f 
  ql/src/test/results/clientpositive/perf/tez/constraints/query77.q.out 
0f1eafc4d8 
  ql/src/test/results/clientpositive/perf/tez/constraints/query8.q.out 
f036baf048 
  ql/src/test/results/clientpositive/perf/tez/constraints/query80.q.out 
d4e913cb4f 
  ql/src/test/results/clientpositive/perf/tez/constraints/query82.q.out 
185629c47c 
  ql/src/test/results/clientpositive/perf/tez/constraints/query99.q.out 
edcf1c20d7 
  ql/src/test/results/clientpositive/perf/tez/query10.q.out 30628f4829 
  ql/src/test/results/clientpositive/perf/tez/query14.q.out abfc5e85d4 
  ql/src/test/results/clientpositive/perf/tez/query15.q.out 418bffddca 
  ql/src/test/results/clientpositive/perf/tez/query17.q.out e7eda149e8 
  ql/src/test/results/clientpositive/perf/tez/query25.q.out 201de7c376 
  ql/src/test/results/clientpositive/perf/tez/query26.q.out 4d26c4be5b 
  ql/src/test/results/clientpositive/perf/tez/query27.q.out d915989654 
  ql/src/test/results/clientpositive/perf/tez/query29.q.out 8d947e280b 
  ql/src/test/results/clientpositive/perf/tez/query35.q.out 1936f22b61 
  ql/src/test/results/clientpositive/perf/tez/query37.q.out 53b1cee165 
  ql/src/test/results/clientpositive/perf/tez/query40.q.out 61f3d7c748 
  ql/src/test/results/clientpositive/perf/tez/query43.q.out 2e6f86c0a0 
  ql/src/test/results/clientpositive/perf/tez/query45.q.out ab7181d7a4 
  ql/src/test/results/clientpositive/perf/tez/query49.q.out aefacb03f4 
  ql/src/test/results/clientpositive/perf/tez/query5.q.out a345d5caf1 
  ql/src/test/results/clientpositive/perf/tez/query50.q.out 2a287ce6d3 
  ql/src/test/results/clientpositive/perf/tez/query60.q.out 87eaa70d50 
  ql/src/test/results/clientpositive/perf/tez/query66.q.out 23ffd637a3 
  ql/src/test/results/clientpositive/perf/tez/query69.q.out d27f4060f6 
  ql/src/test/results/clientpositive/perf/tez/query7.q.out c00b543a0e 
  ql/src/test/results/clientpositive/perf/tez/query76.q.out c49be8e699 
  ql/src/test/results/clientpositive/perf/tez/query77.q.out ca68776a34 
  ql/src/test/results/clientpositive/perf/tez/query8.q.out b35b381a82 
  ql/src/test/results/clientpositive/perf/tez/query80.q.out e831f68dcf 
  ql/src/test/results/clientpositive/perf/tez/query82.q.out c481ce47fe 
  ql/src/test/results/clientpositive/perf/tez/query99.q.out fadd30cd14 
  ql/src/test/results/clientpositive/tez/topnkey.q.out 45947d0be8 
  ql/src/test/results/clientpositive/tez/vector_topnkey.q.out d179013e28 
  ql/src/test/results/clientpositive/topnkey.q.out cecbe89b1c 


Diff: https://reviews.apache.org/r/71820/diff/4/

Changes: https://reviews.apache.org/r/71820/diff/3-4/


Testing
-------

Run q tests using TestMiniLlapLocalCliDriver
topnkey.q
vector_topnkey.q


Thanks,

Krisztian Kasa

Reply via email to