Riza Suminto created IMPALA-12988:
-------------------------------------
Summary: Calculate an unbounded version of CpuAsk
Key: IMPALA-12988
URL: https://issues.apache.org/jira/browse/IMPALA-12988
Project: IMPALA
Issue Type: Improvement
Components: Frontend
Reporter: Riza Suminto
CpuAsk is calculated through recursive call beginning at
Planner.computeBlockingAwareCores(), which called after
Planner.computeEffectiveParallelism(). It does blocking operator analysis over
selected degree of parallelism that decided during
computeEffectiveParallelism() traversal. That selected degree of parallelism,
however, is already bounded by min and max parallelism config, derived from
PROCESSING_COST_MIN_THREADS and MAX_FRAGMENT_INSTANCES_PER_NODE options
accordingly.
It is beneficial to have another version of CpuAsk that is not bounded by min
and max parallelism config. It should purely based on the fragment's
ProcessingCost and query plan relationship constraint (ie., num JOIN BUILDER
fragment should be equal as num JOIN fragment for partitioned join). During
executor group set selection, Frontend should use the unbounded CpuAsk number
to avoid assigning query to small executor group set prematurely.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
