-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/376/
-----------------------------------------------------------
Review request for pig and Richard Ding.
Summary
-------
The below script when run in local mode gives me a different output. It looks
like in local mode I have to store a relation obtained through streaming in
order to use it afterwards.
For example consider the below script :
DEFINE MySTREAMUDF `test.sh`;
A = LOAD 'myinput' USING PigStorage() AS (myId:chararray, data2, data3,data4 );
B = STREAM A THROUGH MySTREAMUDF AS (wId:chararray, num:int);
--STORE B into 'output.B';
C = JOIN B by wId LEFT OUTER, A by myId;
D = FOREACH C GENERATE B::wId,B::num,data4 ;
D = STREAM D THROUGH MySTREAMUDF AS (f1:chararray,f2:int);
--STORE D into 'output.D';
E = foreach B GENERATE wId,num;
F = DISTINCT E;
G = GROUP F ALL;
H = FOREACH G GENERATE COUNT_STAR(F) as TotalCount;
I = CROSS D,H;
STORE I into 'output.I';
This addresses bug PIG-1831.
https://issues.apache.org/jira/browse/PIG-1831
Diffs
-----
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/PigMapBase.java
1065894
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/PigMapReduce.java
1065894
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/partitioners/SkewedPartitioner.java
1065894
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POCollectedGroup.java
1065894
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POCombinerPackage.java
1065894
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/PODistinct.java
1065894
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POJoinPackage.java
1065894
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POMergeCogroup.java
1065894
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POMergeJoin.java
1065894
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POPackage.java
1065894
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POPartitionRearrange.java
1065894
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POSort.java
1065894
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/util/MapRedUtil.java
1065894
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/streaming/HadoopExecutableManager.java
1065894
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/builtin/Distinct.java
1065894
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/data/InternalCachedBag.java
1065894
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/data/InternalDistinctBag.java
1065894
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/data/InternalSortedBag.java
1065894
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/builtin/DefaultIndexableLoader.java
1065894
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/io/FileLocalizer.java
1065894
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestFRJoin.java
1065894
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestFinish.java
1065894
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/utils/FILTERFROMFILE.java
1065894
Diff: https://reviews.apache.org/r/376/diff
Testing
-------
Thanks,
Daniel
