[
https://issues.apache.org/jira/browse/IMPALA-6115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Csaba Ringhofer updated IMPALA-6115:
------------------------------------
Labels: metadata performance (was: metadata perfomance)
> Investigate scheduling options for block metadata loading threads
> -----------------------------------------------------------------
>
> Key: IMPALA-6115
> URL: https://issues.apache.org/jira/browse/IMPALA-6115
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog
> Affects Versions: Impala 2.11.0
> Reporter: Bharath Vissapragada
> Priority: Minor
> Labels: metadata, performance
>
> In the current implementation of IMPALA-5429, we spin up an executor pool of
> threads every time we load the table and close it once the load has
> finished/failed. We need to investigate the following options and see if they
> can give better throughput.
> 1. Single thread pool for the process lifetime to minimize the startup and
> teardown latency and all the block metadata loading tasks are submitted to it.
> 2. Clustered scheduling of threads rather than submitting them all at once.
> 3. Any other scheduling options that guarantee fairness to all the tables and
> are more performant.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
