Abhishek Rawat created IMPALA-15391:
---------------------------------------
Summary: Enable multi-threaded execution by default
Key: IMPALA-15391
URL: https://issues.apache.org/jira/browse/IMPALA-15391
Project: IMPALA
Issue Type: Task
Reporter: Abhishek Rawat
As Impala’s multi-threaded execution has matured, enabling it by default would
allow the engine to better utilize modern large, high-core-count compute
instances. Providing multi-threading out of the box would also make default
performance more representative when comparing Impala with other analytical
engines that enable intra-node parallelism by default.
Rather than relying on a single static {{MT_DOP}} value, the default level of
parallelism should ideally scale with available hardware resources and be
validated against representative benchmark workloads. Longer term, query
planning should move toward {{{}COMPUTE_PROCESSING_COST{}}}-based parallelism,
allowing Impala to determine an appropriate level of fragment-instance
parallelism based on estimated work and available CPU resources.
The optimal defaults may differ for single-query or low-concurrency workloads
versus highly concurrent workloads. Tuning and resource-allocation behavior for
these scenarios can be evaluated separately in a follow-up JIRA.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
