Joe McDonnell has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/21698 )
Change subject: IMPALA-13186: Tag query option scope for tuple cache ...................................................................... IMPALA-13186: Tag query option scope for tuple cache Constructs a hash of non-default query options that are relevant to query results; by default query options are included in the hash. Passes this hash to the frontend for inclusion in the tuple cache key on plan leaf nodes (which will be included in parent key calculation). Modifies MurmurHash3 to be re-entrant so the backend can construct a hash incrementally. This is slightly slower but more memory efficient than accumulating all hash inputs in a contiguous array first. Adds TUPLE_CACHE_EXEMPT_QUERY_OPT_FN to mark query options that can be ignored when calculating a tuple cache hash. Adds startup flag 'tuple_cache_exempt_query_options' as a safety valve for query options that might be important to exempt that we missed. Removes duplicate printing logic for query options from child-query.cc in favor of re-using TQueryOptionsToMap, which does the same thing. Cleans up query-options.cc helpers so they're static and reduces duplicate printing logic. Adds test that different values for a relevant query option use different cache entries. Adds startup flag 'tuple_cache_ignore_query_options' to omit query options for testing certain tuple cache failure modes, where we need to use debug actions. Change-Id: I1f4802ad9548749cd43df8848b6f46dca3739ae7 Reviewed-on: http://gerrit.cloudera.org:8080/21698 Reviewed-by: Joe McDonnell <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/service/child-query.cc M be/src/service/impala-server.cc M be/src/service/query-options.cc M be/src/service/query-options.h M be/src/thirdparty/datasketches/MurmurHash3.h M common/thrift/ImpalaService.thrift M common/thrift/Query.thrift M fe/src/main/java/org/apache/impala/planner/PlanNode.java M fe/src/main/java/org/apache/impala/planner/TupleCachePlanner.java M tests/custom_cluster/test_tuple_cache.py 10 files changed, 277 insertions(+), 124 deletions(-) Approvals: Joe McDonnell: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/21698 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I1f4802ad9548749cd43df8848b6f46dca3739ae7 Gerrit-Change-Number: 21698 Gerrit-PatchSet: 18 Gerrit-Owner: Michael Smith <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Kurt Deschler <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Yida Wu <[email protected]>
