[ https://issues.apache.org/jira/browse/HIVE-10163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388188#comment-14388188 ]
Gopal V commented on HIVE-10163: -------------------------------- {{WritableComparator::get()}} --> {{ReflectionUtils.setConf()}} is pointless, but it misses the static synchronized block inside HADOOP-11771 because the default argument is NULL. There is no difference between each iteration of the same row-keys, since the schema does not vary between JOIN keys in the same operator. > CommonMergeJoinOperator calls WritableComparator.get() in the inner loop > ------------------------------------------------------------------------ > > Key: HIVE-10163 > URL: https://issues.apache.org/jira/browse/HIVE-10163 > Project: Hive > Issue Type: Bug > Components: Query Processor > Affects Versions: 1.2.0 > Reporter: Gopal V > Labels: JOIN, Performance > > The CommonMergeJoinOperator wastes CPU looking up the correct comparator for > each WritableComparable in each row. > {code} > @SuppressWarnings("rawtypes") > private int compareKeys(List<Object> k1, List<Object> k2) { > int ret = 0; > .... > ret = WritableComparator.get(key_1.getClass()).compare(key_1, key_2); > if (ret != 0) { > return ret; > } > } > {code} > The slow part of that get() is deep within {{ReflectionUtils.setConf}}, where > it tries to use reflection to set the Comparator config for each row being > compared. -- This message was sent by Atlassian JIRA (v6.3.4#6332)