[
https://issues.apache.org/jira/browse/HIVE-964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069997#comment-14069997
]
wangmeng commented on HIVE-964:
-------------------------------
if the two join tables have the same big skew key on one value (for example
,select * from table A join B on A.id=b.id, both table A and B have a
lot of keys on id=1, in this case ,map join will OOM), how to fix this
case? Will it rollback to common join ?
> handle skewed keys for a join in a separate job
> -----------------------------------------------
>
> Key: HIVE-964
> URL: https://issues.apache.org/jira/browse/HIVE-964
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Namit Jain
> Assignee: He Yongqiang
> Fix For: 0.6.0
>
> Attachments: hive-964-2009-12-17.txt, hive-964-2009-12-28-2.patch,
> hive-964-2009-12-29-4.patch, hive-964-2010-01-08.patch,
> hive-964-2010-01-13-2.patch, hive-964-2010-01-14-3.patch,
> hive-964-2010-01-15-4.patch
>
>
> The skewed keys can be written to a temporary table or file, and a followup
> conditional task can be used to perform the join on those keys.
> As a first step, JDBM can be used for those keys
--
This message was sent by Atlassian JIRA
(v6.2#6252)
