Michael Smith created IMPALA-13012:
--------------------------------------
Summary: Completed queries write fails regularly under heavy load
Key: IMPALA-13012
URL: https://issues.apache.org/jira/browse/IMPALA-13012
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 4.4.0
Reporter: Michael Smith
Under heavy test load (running EE tests), Impala regularly fails to write
completed queries with errors like
{code}
W0411 19:11:07.764967 32713 workload-management.cc:435] failed to write
completed queries table="sys.impala_query_log" record_count="10001"
W0411 19:11:07.764983 32713 workload-management.cc:437] AnalysisException:
Exceeded the statement expression limit (250000)
Statement has 370039 expressions.
{code}
After a few attempts, it floods logs with an error for each query that could
not be written
{code}
E0411 19:11:24.646953 32713 workload-management.cc:376] could not write
completed query table="sys.impala_query_log"
query_id="3142ceb1380b58e6:715b83d900000000"
{code}
This seems like poor default behavior. Options for addressing it:
# Decrease the default for {{query_log_max_queued}}. Inserts are pretty
constant at 37 expressions per entry. I'm not sure why that isn't 49, since
that's the number of columns we have; maybe some fields are frequently omitted.
I would cap {{query_log_max_queued}} to {{statement_expression_limit /
number_of_columns ~ 5100}}.
# Allow workload management to {{set statement_expression_limit}} higher using
a similar formula. This may be relatively safe as the expressions are simple.
# Ideally we would skip expression parsing and construct TExecRequest directly,
but that's a much larger effort.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
