Riza Suminto created IMPALA-12029:
-------------------------------------
Summary: Query can be under parallelized in multi executor group
set setup
Key: IMPALA-12029
URL: https://issues.apache.org/jira/browse/IMPALA-12029
Project: IMPALA
Issue Type: Improvement
Components: Frontend
Affects Versions: Impala 4.2.0
Reporter: Riza Suminto
In multiple executor group set setup, Frontend will try to match a query with
the smallest executor group set that can fit the memory and cpu requirement of
the compiled query. There are kind of query where the compiled plan will fit to
any executor group set but not necessarily deliver the best performance. An
example for this is Impala's COMPUTE STATS query. It does full table scan and
aggregate the stats, have fairly simple query plan shape, but can benefit from
higher scan parallelism.
Planner needs to give additional feedback to Frontend that the query might be
under parallelized under current executor group. Frontend can then make
judgement whether to assign the compiled plan to current executor group anyway,
or try step up to the next larger executor group and increase parallelism.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
