----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17439/#review33117 -----------------------------------------------------------
src/org/apache/pig/backend/hadoop/executionengine/tez/WeightedRangePartitionerTez.java <https://reviews.apache.org/r/17439/#comment62402> In what scenario is quantiles map empty? If we don't initialize weightedParts, quantiles, etc won't we encounter NPE later? - Rohini Palaniswamy On Jan. 28, 2014, 3:44 a.m., Cheolsoo Park wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/17439/ > ----------------------------------------------------------- > > (Updated Jan. 28, 2014, 3:44 a.m.) > > > Review request for pig, Alex Bain, Daniel Dai, Mark Wagner, and Rohini > Palaniswamy. > > > Bugs: PIG-3728 > https://issues.apache.org/jira/browse/PIG-3728 > > > Repository: pig-git > > > Description > ------- > > The patch fixes TestSkewedJoin in tez mode. Note I changed > SkewedPartitionerTez and WeightedRangePartitionerTez to allow null sampleMap > to make testSkewedJoinEmptyInput pass. It used to throw a runtime exception, > but now it assumes empty sample input. > > I also fixed MiniCluster in hadoop20. They were missing the getExecType() > method. > > > Diffs > ----- > > shims/test/hadoop20/org/apache/pig/test/MiniCluster.java 3e64a5d > shims/test/hadoop20/org/apache/pig/test/TezMiniCluster.java 98a580c > > src/org/apache/pig/backend/hadoop/executionengine/tez/POPartitionRearrangeTez.java > b17b38b > src/org/apache/pig/backend/hadoop/executionengine/tez/PigProcessor.java > 2b6c899 > > src/org/apache/pig/backend/hadoop/executionengine/tez/SkewedPartitionerTez.java > 5fe202b > > src/org/apache/pig/backend/hadoop/executionengine/tez/WeightedRangePartitionerTez.java > 36441fb > test/org/apache/pig/test/TestAccumulator.java 516d5c5 > test/org/apache/pig/test/TestSkewedJoin.java be91d6b > test/tez-tests 0d32d23 > > Diff: https://reviews.apache.org/r/17439/diff/ > > > Testing > ------- > > * TestSkewedJoin passes in both tez and mr mode. > * ant test-tez passes except TestCustomPartitioner (known). > * All e2e tests pass. > > > Thanks, > > Cheolsoo Park > >
