[
https://issues.apache.org/jira/browse/IMPALA-13012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17841385#comment-17841385
]
ASF subversion and git services commented on IMPALA-13012:
----------------------------------------------------------
Commit ba32d70891fd68c5c1234ed543b74c51661bf272 in impala's branch
refs/heads/master from Michael Smith
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=ba32d7089 ]
IMPALA-13012: Lower default query_log_max_queued
Sets the query_log_max_queued default such that
query_log_max_queued * num_columns(49) < statement_expression_limit
to avoid triggering e.g.
AnalysisException: Exceeded the statement expression limit (250000)
Statement has 370039 expressions.
Also increases statement_expression_limit for insertion to avoid an
error if query_log_max_queued is changed.
Logs time taken to write to the queries table for help with debugging
and adds histogram "impala-server.completed-queries.write-durations".
Fixes InternalServer so it uses 'default_query_options'.
Change-Id: I6535675307d88cb65ba7d908f3c692e0cf3259d7
Reviewed-on: http://gerrit.cloudera.org:8080/21351
Reviewed-by: Michael Smith <[email protected]>
Tested-by: Michael Smith <[email protected]>
Reviewed-by: Riza Suminto <[email protected]>
> Completed queries write fails regularly under heavy load
> --------------------------------------------------------
>
> Key: IMPALA-13012
> URL: https://issues.apache.org/jira/browse/IMPALA-13012
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 4.4.0
> Reporter: Michael Smith
> Assignee: Michael Smith
> Priority: Critical
>
> Under heavy test load (running EE tests), Impala regularly fails to write
> completed queries with errors like
> {code}
> W0411 19:11:07.764967 32713 workload-management.cc:435] failed to write
> completed queries table="sys.impala_query_log" record_count="10001"
> W0411 19:11:07.764983 32713 workload-management.cc:437] AnalysisException:
> Exceeded the statement expression limit (250000)
> Statement has 370039 expressions.
> {code}
> After a few attempts, it floods logs with an error for each query that could
> not be written
> {code}
> E0411 19:11:24.646953 32713 workload-management.cc:376] could not write
> completed query table="sys.impala_query_log"
> query_id="3142ceb1380b58e6:715b83d900000000"
> {code}
> This seems like poor default behavior. Options for addressing it:
> # Decrease the default for {{query_log_max_queued}}. Inserts are pretty
> constant at 37 expressions per entry. I'm not sure why that isn't 49, since
> that's the number of columns we have; maybe some fields are frequently
> omitted. I would cap {{query_log_max_queued}} to {{statement_expression_limit
> / number_of_columns ~ 5100}}.
> # Allow workload management to {{set statement_expression_limit}} higher
> using a similar formula. This may be relatively safe as the expressions are
> simple.
> # Ideally we would skip expression parsing and construct TExecRequest
> directly, but that's a much larger effort.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
