[
https://issues.apache.org/jira/browse/IMPALA-13186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17888012#comment-17888012
]
ASF subversion and git services commented on IMPALA-13186:
----------------------------------------------------------
Commit b6b953b48e05b64e7f0c9d1cb2623148671ffce2 in impala's branch
refs/heads/master from Michael Smith
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=b6b953b48 ]
IMPALA-13186: Tag query option scope for tuple cache
Constructs a hash of non-default query options that are relevant to
query results; by default query options are included in the hash.
Passes this hash to the frontend for inclusion in the tuple cache key
on plan leaf nodes (which will be included in parent key calculation).
Modifies MurmurHash3 to be re-entrant so the backend can construct a
hash incrementally. This is slightly slower but more memory efficient
than accumulating all hash inputs in a contiguous array first.
Adds TUPLE_CACHE_EXEMPT_QUERY_OPT_FN to mark query options that can be
ignored when calculating a tuple cache hash.
Adds startup flag 'tuple_cache_exempt_query_options' as a safety valve
for query options that might be important to exempt that we missed.
Removes duplicate printing logic for query options from child-query.cc
in favor of re-using TQueryOptionsToMap, which does the same thing.
Cleans up query-options.cc helpers so they're static and reduces
duplicate printing logic.
Adds test that different values for a relevant query option use
different cache entries. Adds startup flag
'tuple_cache_ignore_query_options' to omit query options for testing
certain tuple cache failure modes, where we need to use debug actions.
Change-Id: I1f4802ad9548749cd43df8848b6f46dca3739ae7
Reviewed-on: http://gerrit.cloudera.org:8080/21698
Reviewed-by: Joe McDonnell <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Tuple cache keys should incorporate information about related query options
> ---------------------------------------------------------------------------
>
> Key: IMPALA-13186
> URL: https://issues.apache.org/jira/browse/IMPALA-13186
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 4.5.0
> Reporter: Joe McDonnell
> Assignee: Michael Smith
> Priority: Major
>
> Currently, the tuple cache key does not include information from the query
> options. Many query options have no impact on the result of a query (e.g.
> idle_session_timeout) or are evaluated purely on the coordinator during
> planning (e.g. broadcast_bytes_limit).
> However, some query options can impact behavior either by controlling how
> certain things are calculated (e.g. decimal_v2) or controlling what
> conditions result in an error. Changing a query option can change the output
> of a query.
> We need some way to incorporate the relevant query options into the tuple
> cache key so there is no correctness issue.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
