----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17529/#review33264 -----------------------------------------------------------
Ship it! +1 for the rest - Daniel Dai On Jan. 30, 2014, 8:28 a.m., Rohini Palaniswamy wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/17529/ > ----------------------------------------------------------- > > (Updated Jan. 30, 2014, 8:28 a.m.) > > > Review request for pig, Cheolsoo Park and Daniel Dai. > > > Bugs: PIG-3732 > https://issues.apache.org/jira/browse/PIG-3732 > > > Repository: pig > > > Description > ------- > > Orderby has 4 vertices and changes done are as below. > > Load Vertex -> Partitioner Vertex > - Was RoundRobinPartitioner with sorted shuffle and parallelism of > Partitioner Vertex was same as reducer vertex (i.e PARALLEL clause). Now > ONE_TO_ONE unsorted edge between Load Vertex and Partitioner Vertex with > Partitioner Vertex having same parallelism as Load Vertex. Will get the > performance numbers for both cases by Friday. > Load Vertex -> Sampler Vertex > Sampler Vertex -> Partitioner Vertex (Broadcast edge) > - The POPackage->POForeach->POLocalRearrange in Partitioner Vertex has > been replaced by POIdentityInOutTez > Partitioner Vertex -> Reducer Vertex > > Need to attempt this for Skewed Join as well. > > > This patch also sets credential on DAG which is required after TEZ-395 > > > Diffs > ----- > > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/partitioners/WeightedRangePartitioner.java > 1562426 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/PhysicalOperator.java > 1562426 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/POIdentityInOutTez.java > PRE-CREATION > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/POLocalRearrangeTez.java > 1562426 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/POPartitionRearrangeTez.java > 1562426 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/POShuffleTezLoad.java > 1562426 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/PigProcessor.java > 1562426 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezCompiler.java > 1562426 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezDAG.java > 1562426 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/WeightedRangePartitionerTez.java > 1562426 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/impl/io/NullablePartitionWritable.java > 1562426 > > http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/impl/io/PigNullableWritable.java > 1562426 > > http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/TEZC16.gld > 1562426 > > http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/TEZC7.gld > 1562426 > > Diff: https://reviews.apache.org/r/17529/diff/ > > > Testing > ------- > > test-tez and tez.conf e2e tests pass > > > Thanks, > > Rohini Palaniswamy > >
