> On May 12, 2015, 6:26 a.m., Alexander Pivovarov wrote: > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/ReduceSinkMapJoinProc.java, > > line 107 > > <https://reviews.apache.org/r/34059/diff/1/?file=955672#file955672line107> > > > > ReduceSinkOperator uses Object.hashCode() and equals() methods. > > HashSet algo relies on hashCode/equals methods > > Jason Dere wrote: > So that means equals() only works if it is the exact same > ReduceSinkOperator object. This should be ok for our usage, if we are > referring to the same ReduceSinkOperator, we should be using that exact same > object. > > Alexander Pivovarov wrote: > Do you want to use IdentityHashMap then? > This class implements the Map interface with a hash table, using > reference-equality in place of object-equality when comparing keys (and > values). In other words, in an IdentityHashMap, two keys k1 and k2 are > considered equal if and only if (k1==k2)
We're using a Set here as opposed to a Map. I'll change to use Sets.newIdentityHashSet() from Guava. - Jason ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34059/#review83362 ----------------------------------------------------------- On May 11, 2015, 9:48 p.m., Jason Dere wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/34059/ > ----------------------------------------------------------- > > (Updated May 11, 2015, 9:48 p.m.) > > > Review request for hive, Matt McCline and Vikram Dixit Kumaraswamy. > > > Bugs: HIVE-10673 > https://issues.apache.org/jira/browse/HIVE-10673 > > > Repository: hive-git > > > Description > ------- > > Reduce-side hash join (using MapJoinOperator), where the Tez inputs to the > reducer are unsorted. > > > Diffs > ----- > > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java eff4d30 > itests/src/test/resources/testconfiguration.properties eeb46cc > ql/src/java/org/apache/hadoop/hive/ql/exec/MapJoinOperator.java b1352f3 > ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java d7f1b42 > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/KeyValuesAdapter.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/KeyValuesFromKeyValue.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/KeyValuesFromKeyValues.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/ReduceRecordProcessor.java > 545d7c6 > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/ReduceRecordSource.java > cdabe3a > > ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorMapJoinOperator.java > 15c747e > > ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinCommonOperator.java > a9082eb > ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java > d42b643 > ql/src/java/org/apache/hadoop/hive/ql/optimizer/MapJoinProcessor.java > 4d84f0f > ql/src/java/org/apache/hadoop/hive/ql/optimizer/ReduceSinkMapJoinProc.java > f7e1dbc > ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezProcContext.java adc31ae > ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezUtils.java 241e9d7 > ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezWork.java 6db8220 > ql/src/java/org/apache/hadoop/hive/ql/plan/BaseWork.java a342738 > ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDescUtils.java fb3c4a3 > ql/src/java/org/apache/hadoop/hive/ql/plan/MapJoinDesc.java cee9100 > ql/src/test/queries/clientpositive/tez_dynpart_hashjoin_1.q PRE-CREATION > ql/src/test/queries/clientpositive/tez_dynpart_hashjoin_2.q PRE-CREATION > ql/src/test/queries/clientpositive/tez_vector_dynpart_hashjoin_1.q > PRE-CREATION > ql/src/test/results/clientpositive/tez/tez_dynpart_hashjoin_1.q.out > PRE-CREATION > ql/src/test/results/clientpositive/tez/tez_dynpart_hashjoin_2.q.out > PRE-CREATION > ql/src/test/results/clientpositive/tez/tez_vector_dynpart_hashjoin_1.q.out > PRE-CREATION > > Diff: https://reviews.apache.org/r/34059/diff/ > > > Testing > ------- > > q-file tests added > > > Thanks, > > Jason Dere > >
