[
https://issues.apache.org/jira/browse/IMPALA-13012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith updated IMPALA-13012:
-----------------------------------
Fix Version/s: Impala 4.4.0
> Completed queries write fails regularly under heavy load
> --------------------------------------------------------
>
> Key: IMPALA-13012
> URL: https://issues.apache.org/jira/browse/IMPALA-13012
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 4.4.0
> Reporter: Michael Smith
> Assignee: Michael Smith
> Priority: Critical
> Fix For: Impala 4.4.0
>
>
> Under heavy test load (running EE tests), Impala regularly fails to write
> completed queries with errors like
> {code}
> W0411 19:11:07.764967 32713 workload-management.cc:435] failed to write
> completed queries table="sys.impala_query_log" record_count="10001"
> W0411 19:11:07.764983 32713 workload-management.cc:437] AnalysisException:
> Exceeded the statement expression limit (250000)
> Statement has 370039 expressions.
> {code}
> After a few attempts, it floods logs with an error for each query that could
> not be written
> {code}
> E0411 19:11:24.646953 32713 workload-management.cc:376] could not write
> completed query table="sys.impala_query_log"
> query_id="3142ceb1380b58e6:715b83d900000000"
> {code}
> This seems like poor default behavior. Options for addressing it:
> # Decrease the default for {{query_log_max_queued}}. Inserts are pretty
> constant at 37 expressions per entry. I'm not sure why that isn't 49, since
> that's the number of columns we have; maybe some fields are frequently
> omitted. I would cap {{query_log_max_queued}} to {{statement_expression_limit
> / number_of_columns ~ 5100}}.
> # Allow workload management to {{set statement_expression_limit}} higher
> using a similar formula. This may be relatively safe as the expressions are
> simple.
> # Ideally we would skip expression parsing and construct TExecRequest
> directly, but that's a much larger effort.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
