[ https://issues.apache.org/jira/browse/HIVE-12492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gopal V updated HIVE-12492: --------------------------- Description: After 4 million keys, the map-join implementation seems to suffer from a performance degradation. The hashtable build & probe time makes this very inefficient, even if the data is very compact (i.e 2 ints). Falling back onto the shuffle join or bucket map-join is useful after 2^22 items. (Note: this fixes a statsutil issue - due to the extra clone() in the column stats path) was: After 4 million keys, the map-join implementation seems to suffer from a performance degradation. The hashtable build & probe time makes this very inefficient, even if the data is very compact (i.e 2 ints). Falling back onto the shuffle join or bucket map-join is useful after 2^22 items. > MapJoin: 4 million unique integers seems to be a probe plateau > -------------------------------------------------------------- > > Key: HIVE-12492 > URL: https://issues.apache.org/jira/browse/HIVE-12492 > Project: Hive > Issue Type: Improvement > Components: Query Planning > Affects Versions: 1.3.0, 1.2.1, 2.0.0 > Reporter: Gopal V > Assignee: Jesus Camacho Rodriguez > Labels: TODOC2.2 > Fix For: 2.2.0 > > Attachments: HIVE-12492.01.patch, HIVE-12492.02.patch, > HIVE-12492.patch > > > After 4 million keys, the map-join implementation seems to suffer from a > performance degradation. > The hashtable build & probe time makes this very inefficient, even if the > data is very compact (i.e 2 ints). > Falling back onto the shuffle join or bucket map-join is useful after 2^22 > items. > (Note: this fixes a statsutil issue - due to the extra clone() in the column > stats path) -- This message was sent by Atlassian JIRA (v6.3.15#6346)