-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65304/
-----------------------------------------------------------
(Updated Jan. 24, 2018, 9:51 p.m.)
Review request for hive, Ashutosh Chauhan, Gopal V, and Jesús Camacho Rodríguez.
Changes
-------
- Disable caching of results if the query did not use CBO
- Initialize the cache during HS2 init
- Update test configs to disable caching by default. Some of the tests have
repeated runs of the same queries to test different join algorithms etc, and
the presence of query caching will mess up those tests. There have been several
qfile tests added which specifically enable the cache.
Bugs: HIVE-18513
https://issues.apache.org/jira/browse/HIVE-18513
Repository: hive-git
Description
-------
- For queries that result in MR/Tez/Spark jobs on the cluster, save the
temporary query results to a cache directory where they can be re-used.
- Add QueryResultsCache to manage cached results. Currently cache invalidation
is time-based, update-based cache invalidation needs to be added later.
- Driver/SemanticAnalyzer/Calcite planner changes to lookup queries in the
cache and use in place of the query plan.
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 0c2cf05
common/src/java/org/apache/hadoop/hive/ql/log/PerfLogger.java 2767bca
data/conf/hive-site.xml 01f83d1
data/conf/llap/hive-site.xml cdda875
data/conf/perf-reg/spark/hive-site.xml 497a61f
data/conf/perf-reg/tez/hive-site.xml 012369f
data/conf/rlist/hive-site.xml 9de00e5
data/conf/spark/local/hive-site.xml fd0e6a0
data/conf/spark/standalone/hive-site.xml 1e5bd65
data/conf/spark/yarn-client/hive-site.xml a9a788b
data/conf/tez/hive-site.xml 4519678
itests/src/test/resources/testconfiguration.properties 1017249
itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java 3f377f9
ql/src/java/org/apache/hadoop/hive/ql/Driver.java 74595b0
ql/src/java/org/apache/hadoop/hive/ql/cache/results/CacheUsage.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/cache/results/QueryResultsCache.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 2e1fd37
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveCalciteUtil.java
f0dd167
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelOpMaterializationValidator.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 372cfad
ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 85a1f34
ql/src/java/org/apache/hadoop/hive/ql/parse/QBParseInfo.java ae2ec3d
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 28e1041
ql/src/java/org/apache/hadoop/hive/ql/plan/FetchWork.java 7243dc7
ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java 8af19d8
ql/src/test/queries/clientpositive/results_cache_1.q PRE-CREATION
ql/src/test/queries/clientpositive/results_cache_2.q PRE-CREATION
ql/src/test/queries/clientpositive/results_cache_capacity.q PRE-CREATION
ql/src/test/queries/clientpositive/results_cache_lifetime.q PRE-CREATION
ql/src/test/queries/clientpositive/results_cache_temptable.q PRE-CREATION
ql/src/test/queries/clientpositive/results_cache_with_masking.q PRE-CREATION
ql/src/test/results/clientpositive/llap/results_cache_1.q.out PRE-CREATION
ql/src/test/results/clientpositive/results_cache_1.q.out PRE-CREATION
ql/src/test/results/clientpositive/results_cache_2.q.out PRE-CREATION
ql/src/test/results/clientpositive/results_cache_capacity.q.out PRE-CREATION
ql/src/test/results/clientpositive/results_cache_lifetime.q.out PRE-CREATION
ql/src/test/results/clientpositive/results_cache_temptable.q.out PRE-CREATION
ql/src/test/results/clientpositive/results_cache_with_masking.q.out
PRE-CREATION
service/src/java/org/apache/hive/service/server/HiveServer2.java 2a528cd
Diff: https://reviews.apache.org/r/65304/diff/2/
Changes: https://reviews.apache.org/r/65304/diff/1-2/
Testing
-------
qfile tests added.
Thanks,
Jason Dere
