----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17529/ -----------------------------------------------------------
Review request for pig, Cheolsoo Park and Daniel Dai.
Bugs: PIG-3732
https://issues.apache.org/jira/browse/PIG-3732
Repository: pig
Description
-------
Orderby has 4 vertices and changes done are as below.
Load Vertex -> Partitioner Vertex
- Was RoundRobinPartitioner with sorted shuffle and parallelism of
Partitioner Vertex was same as reducer vertex (i.e PARALLEL clause). Now
ONE_TO_ONE unsorted edge between Load Vertex and Partitioner Vertex with
Partitioner Vertex having same parallelism as Load Vertex. Will get the
performance numbers for both cases by Friday.
Load Vertex -> Sampler Vertex
Sampler Vertex -> Partitioner Vertex (Broadcast edge)
- The POPackage->POForeach->POLocalRearrange in Partitioner Vertex has
been replaced by POIdentityInOutTez
Partitioner Vertex -> Reducer Vertex
Need to attempt this for Skewed Join as well.
This patch also sets credential on DAG which is required after TEZ-395
Diffs
-----
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/partitioners/WeightedRangePartitioner.java
1562426
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/PhysicalOperator.java
1562426
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/POIdentityInOutTez.java
PRE-CREATION
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/POLocalRearrangeTez.java
1562426
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/POPartitionRearrangeTez.java
1562426
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/POShuffleTezLoad.java
1562426
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/PigProcessor.java
1562426
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezCompiler.java
1562426
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/TezDAG.java
1562426
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/backend/hadoop/executionengine/tez/WeightedRangePartitionerTez.java
1562426
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/impl/io/NullablePartitionWritable.java
1562426
http://svn.apache.org/repos/asf/pig/branches/tez/src/org/apache/pig/impl/io/PigNullableWritable.java
1562426
http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/TEZC16.gld
1562426
http://svn.apache.org/repos/asf/pig/branches/tez/test/org/apache/pig/test/data/GoldenFiles/TEZC7.gld
1562426
Diff: https://reviews.apache.org/r/17529/diff/
Testing
-------
test-tez and tez.conf e2e tests pass
Thanks,
Rohini Palaniswamy
