[
https://issues.apache.org/jira/browse/HIVE-17114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16094329#comment-16094329
]
liyunzhang_intel edited comment on HIVE-17114 at 7/20/17 8:40 AM:
------------------------------------------------------------------
[~lirui]: thanks for your explanation, but i also want to ask that the
situation happened on query with 1 join or multiple joins?
[~gopalv]: thanks for your link and detail query in HIVE-7074.
was (Author: kellyzly):
[~lirui]: thanks for your explanation, but i also want to ask that the
situation happened on join once or join mutiple times?
[~gopalv]: thanks for your link and detail query in HIVE-7074.
> HoS: Possible skew in shuffling when data is not really skewed
> --------------------------------------------------------------
>
> Key: HIVE-17114
> URL: https://issues.apache.org/jira/browse/HIVE-17114
> Project: Hive
> Issue Type: Bug
> Reporter: Rui Li
> Assignee: Rui Li
> Priority: Minor
> Attachments: HIVE-17114.1.patch, HIVE-17114.2.patch
>
>
> Observed in HoS and may apply to other engines as well.
> When we join 2 tables on a single int key, we use the key itself as hash code
> in {{ObjectInspectorUtils.hashCode}}:
> {code}
> case INT:
> return ((IntObjectInspector) poi).get(o);
> {code}
> Suppose the keys are different but are all some multiples of 10. And if we
> choose 10 as #reducers, the shuffle will be skewed.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
