> On Feb. 9, 2018, 5:59 p.m., Jason Dere wrote: > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java > > Lines 582 (patched) > > <https://reviews.apache.org/r/64688/diff/2/?file=1954740#file1954740line582> > > > > Thought different number of buckets was supposed to work as long as the > > buckets were a multiple of each other. So this case doesn't work even if > > the num small buckets is a multiple of the big table buckets?
If this case is broken in the Hive-on-Tez case, can you open a followup bug for this issue? - Jason ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64688/#review197145 ----------------------------------------------------------- On Feb. 8, 2018, 10:09 p.m., Deepak Jaiswal wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64688/ > ----------------------------------------------------------- > > (Updated Feb. 8, 2018, 10:09 p.m.) > > > Review request for hive, Ashutosh Chauhan and Jason Dere. > > > Repository: hive-git > > > Description > ------- > > Bucket based Join : Handle buckets with no splits. > > The current logic in CustomPartitionVertex assumes that there is a split for > each bucket whereas in Tez, we can have no splits for empty buckets. > Also falls back to reduceside join if small table has more buckets than big > table. > > Disallow loading files in bucketed tables if the file name format is not like > 000000_0, 000001_0_copy_1 etc. > > > Diffs > ----- > > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/CustomPartitionVertex.java > 26afe90faa > > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/CustomVertexConfiguration.java > ef5e7edcd6 > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java 9885038588 > ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java > dc698c8de8 > ql/src/java/org/apache/hadoop/hive/ql/parse/LoadSemanticAnalyzer.java > 54f5bab6de > ql/src/test/queries/clientpositive/auto_sortmerge_join_2.q e5fdcb57e4 > ql/src/test/queries/clientpositive/auto_sortmerge_join_4.q abf09e5534 > ql/src/test/queries/clientpositive/auto_sortmerge_join_5.q b85c4a7aa3 > ql/src/test/queries/clientpositive/auto_sortmerge_join_7.q bd780861e3 > ql/src/test/results/clientnegative/bucket_mapjoin_mismatch1.q.out > b9c2e6f827 > ql/src/test/results/clientpositive/auto_sortmerge_join_2.q.out 5cfc35aa73 > ql/src/test/results/clientpositive/auto_sortmerge_join_4.q.out 0d586fd26b > ql/src/test/results/clientpositive/auto_sortmerge_join_5.q.out 45704d1253 > ql/src/test/results/clientpositive/auto_sortmerge_join_7.q.out 1959075912 > ql/src/test/results/clientpositive/llap/auto_sortmerge_join_2.q.out > 054b0d00be > ql/src/test/results/clientpositive/llap/auto_sortmerge_join_4.q.out > 95d329862c > ql/src/test/results/clientpositive/llap/auto_sortmerge_join_5.q.out > e711715aa5 > ql/src/test/results/clientpositive/llap/auto_sortmerge_join_7.q.out > 53c685cb11 > ql/src/test/results/clientpositive/spark/auto_sortmerge_join_2.q.out > 8cfa113794 > ql/src/test/results/clientpositive/spark/auto_sortmerge_join_4.q.out > fce5e0cfc4 > ql/src/test/results/clientpositive/spark/auto_sortmerge_join_5.q.out > 8250eca099 > ql/src/test/results/clientpositive/spark/auto_sortmerge_join_7.q.out > eb813c1734 > > > Diff: https://reviews.apache.org/r/64688/diff/2/ > > > Testing > ------- > > > Thanks, > > Deepak Jaiswal > >
