[
https://issues.apache.org/jira/browse/IMPALA-9792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong reassigned IMPALA-9792:
-------------------------------------
Assignee: Bikramjeet Vig
> Split Kudu scan ranges into smaller chunks for greater paralellelism
> --------------------------------------------------------------------
>
> Key: IMPALA-9792
> URL: https://issues.apache.org/jira/browse/IMPALA-9792
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Tim Armstrong
> Assignee: Bikramjeet Vig
> Priority: Major
> Labels: kudu, multithreading
>
> We currently use one thread to scan each tablet, which may underparallelise
> queries in many cases. Kudu added an API in KUDU-2437 and KUDU-2670 to split
> tokens at a finer granularity.
> See
> https://github.com/apache/kudu/commit/22a6faa44364dec3a171ec79c15b814ad9277d8f#diff-a4afa9dba99c7612b2cb9176134ff2b0
> The major downside is that the planner has to do an extra RPC to a tserver
> for each tablet being scanned in order to figure out key range splits. Maybe
> we can tie this to mt_dop >= 2, or use some heuristics to avoid these RPCs
> for smaller tables.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
