-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65304/
-----------------------------------------------------------
(Updated Jan. 31, 2018, 6:16 a.m.)
Review request for hive, Ashutosh Chauhan, Gopal V, and Jesús Camacho Rodríguez.
Changes
-------
- Updated config setting descriptions in HiveConf
- A lot of fixes in HiveRelOpMaterizationValidator per review comments, plus
tests to test out some of the Hive RelNodes that might be encountered
(HiveSemiJoin, HiveUnion, HiveSortLimit, HiveIntersect, ..)
Bugs: HIVE-18513
https://issues.apache.org/jira/browse/HIVE-18513
Repository: hive-git
Description
-------
- For queries that result in MR/Tez/Spark jobs on the cluster, save the
temporary query results to a cache directory where they can be re-used.
- Add QueryResultsCache to manage cached results. Currently cache invalidation
is time-based, update-based cache invalidation needs to be added later.
- Driver/SemanticAnalyzer/Calcite planner changes to lookup queries in the
cache and use in place of the query plan.
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java b7d3e99
common/src/java/org/apache/hadoop/hive/ql/log/PerfLogger.java 2767bca
data/conf/hive-site.xml 01f83d1
data/conf/llap/hive-site.xml cdda875
data/conf/perf-reg/spark/hive-site.xml 497a61f
data/conf/perf-reg/tez/hive-site.xml 012369f
data/conf/rlist/hive-site.xml 9de00e5
data/conf/spark/local/hive-site.xml fd0e6a0
data/conf/spark/standalone/hive-site.xml 1e5bd65
data/conf/spark/yarn-client/hive-site.xml a9a788b
data/conf/tez/hive-site.xml 4519678
itests/hive-blobstore/src/test/resources/hive-site.xml 038db0d
itests/src/test/resources/testconfiguration.properties d86ff58
itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java 4432aca
ql/src/java/org/apache/hadoop/hive/ql/Driver.java 74595b0
ql/src/java/org/apache/hadoop/hive/ql/cache/results/CacheUsage.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/cache/results/QueryResultsCache.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java d7b3e4b
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveCalciteUtil.java
f0dd167
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelOpMaterializationValidator.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 372cfad
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 85a1f34
ql/src/java/org/apache/hadoop/hive/ql/parse/QBParseInfo.java ae2ec3d
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 83dfb47
ql/src/java/org/apache/hadoop/hive/ql/plan/FetchWork.java 7243dc7
ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java d6083e6
ql/src/test/queries/clientpositive/results_cache_1.q PRE-CREATION
ql/src/test/queries/clientpositive/results_cache_2.q PRE-CREATION
ql/src/test/queries/clientpositive/results_cache_capacity.q PRE-CREATION
ql/src/test/queries/clientpositive/results_cache_lifetime.q PRE-CREATION
ql/src/test/queries/clientpositive/results_cache_temptable.q PRE-CREATION
ql/src/test/queries/clientpositive/results_cache_with_masking.q PRE-CREATION
ql/src/test/results/clientpositive/llap/results_cache_1.q.out PRE-CREATION
ql/src/test/results/clientpositive/results_cache_1.q.out PRE-CREATION
ql/src/test/results/clientpositive/results_cache_2.q.out PRE-CREATION
ql/src/test/results/clientpositive/results_cache_capacity.q.out PRE-CREATION
ql/src/test/results/clientpositive/results_cache_lifetime.q.out PRE-CREATION
ql/src/test/results/clientpositive/results_cache_temptable.q.out PRE-CREATION
ql/src/test/results/clientpositive/results_cache_with_masking.q.out
PRE-CREATION
service/src/java/org/apache/hive/service/server/HiveServer2.java 2a528cd
Diff: https://reviews.apache.org/r/65304/diff/4/
Changes: https://reviews.apache.org/r/65304/diff/3-4/
Testing
-------
qfile tests added.
Thanks,
Jason Dere
