[
https://issues.apache.org/jira/browse/IMPALA-8005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anurag Mantripragada resolved IMPALA-8005.
------------------------------------------
Fix Version/s: Impala 3.4.0
Resolution: Fixed
> Randomize partitioning exchanges destinations
> ---------------------------------------------
>
> Key: IMPALA-8005
> URL: https://issues.apache.org/jira/browse/IMPALA-8005
> Project: IMPALA
> Issue Type: Improvement
> Components: Distributed Exec
> Affects Versions: Impala 3.1.0
> Reporter: Michael Ho
> Assignee: Anurag Mantripragada
> Priority: Major
> Labels: ramp-up
> Fix For: Impala 3.4.0
>
>
> Currently, we use the same hash seed for partitioning exchanges at the
> sender. For a table with skew in distribution in the shuffling keys, multiple
> queries using the same shuffling keys for exchanges will end up hashing to
> the same destination fragments running on particular host and potentially
> overloading that host.
> We should consider using the query id or other query specific information to
> seed the hashing function to randomize the destinations for different
> queries. Thanks to [~tlipcon] for pointing this problem out.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
