[
https://issues.apache.org/jira/browse/IMPALA-15358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith resolved IMPALA-15358.
------------------------------------
Fix Version/s: Impala 5.0.0
Resolution: Fixed
> Cap writers to Kudu partitions
> ------------------------------
>
> Key: IMPALA-15358
> URL: https://issues.apache.org/jira/browse/IMPALA-15358
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Reporter: Michael Smith
> Assignee: Michael Smith
> Priority: Major
> Fix For: Impala 5.0.0
>
>
> For INSERT/UPSERT/UPDATE/DELETE into a partitioned Kudu table, the planner
> inserts a KUDU-partitioned exchange ahead of the KuduTableSink. At runtime,
> KrpcDataStreamSender routes each row to channel (partition_index %
> num_channels), where partition_index is the actual Kudu tablet index (bounded
> by the table's tablet count). The writer fragment's instance count, however,
> is sized like any other fragment (mt_dop / cost-based scaling) with no
> awareness of the target table's partitioning. Whenever the scheduler assigns
> more writer instances than the table has tablets, every instance index >=
> tablet count never receives rows, causing severe work skew.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
