----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28727/#review64093 -----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/exec/SparkHashTableSinkOperator.java <https://reviews.apache.org/r/28727/#comment106515> It seems that we are converting Path to String, and then convert back to Path again later. This is a little concerning. I don't quite understand what are we doing. Also, the null handling also seems concerning. ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinTableContainerSerDe.java <https://reviews.apache.org/r/28727/#comment106516> Be aware of different hadoop versions. ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java <https://reviews.apache.org/r/28727/#comment106518> Are we only get the first instance of such operator? There could be more than one instance in a work. ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/SparkMapJoinOptimizer.java <https://reviews.apache.org/r/28727/#comment106521> Can we have some comments on what this methods do, as it's not trivial? ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/SparkMapJoinOptimizer.java <https://reviews.apache.org/r/28727/#comment106522> Do we assume the set always contains only entry? Overall, the patch looks good to me. Besides the comments above, I'm not very confident that I understand enough to let this go in. Maybe a face to face discussion will help. - Xuefu Zhang On Dec. 4, 2014, 11:38 p.m., Jimmy Xiang wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/28727/ > ----------------------------------------------------------- > > (Updated Dec. 4, 2014, 11:38 p.m.) > > > Review request for hive and Xuefu Zhang. > > > Bugs: HIVE-8638 > https://issues.apache.org/jira/browse/HIVE-8638 > > > Repository: hive-git > > > Description > ------- > > Patch v3 that works when bucket number matches > > > Diffs > ----- > > itests/src/test/resources/testconfiguration.properties 09c667e > ql/src/java/org/apache/hadoop/hive/ql/exec/SparkHashTableSinkOperator.java > cfc1501 > > ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinTableContainerSerDe.java > 2f9e55a > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java > 4054173 > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/SparkBucketJoinProcCtx.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/SparkMapJoinOptimizer.java > 8b78123 > ql/src/test/queries/clientpositive/bucket_map_join_spark1.q PRE-CREATION > ql/src/test/queries/clientpositive/bucket_map_join_spark2.q PRE-CREATION > ql/src/test/results/clientpositive/bucket_map_join_spark1.q.out > PRE-CREATION > ql/src/test/results/clientpositive/bucket_map_join_spark2.q.out > PRE-CREATION > ql/src/test/results/clientpositive/spark/bucket_map_join_spark1.q.out > PRE-CREATION > ql/src/test/results/clientpositive/spark/bucket_map_join_spark2.q.out > PRE-CREATION > > Diff: https://reviews.apache.org/r/28727/diff/ > > > Testing > ------- > > > Thanks, > > Jimmy Xiang > >
