Author: pradeepkth
Date: Tue Apr 20 16:24:34 2010
New Revision: 935981

URL: http://svn.apache.org/viewvc?rev=935981&view=rev
Log:
PIG-1290: WeightedRangePartitioner should not check if input is empty if 
quantile file is empty (pradeepkth)

Modified:
    hadoop/pig/branches/branch-0.6/CHANGES.txt
    
hadoop/pig/branches/branch-0.6/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/partitioners/WeightedRangePartitioner.java

Modified: hadoop/pig/branches/branch-0.6/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/hadoop/pig/branches/branch-0.6/CHANGES.txt?rev=935981&r1=935980&r2=935981&view=diff
==============================================================================
--- hadoop/pig/branches/branch-0.6/CHANGES.txt (original)
+++ hadoop/pig/branches/branch-0.6/CHANGES.txt Tue Apr 20 16:24:34 2010
@@ -156,6 +156,9 @@ PIG-922: Logical optimizer: push up proj
 
 BUG FIXES
 
+PIG-1290: WeightedRangePartitioner should not check if input is empty if
+quantile file is empty (pradeepkth)
+
 PIG-1364: Public javadoc on apache site still on 0.2, needs to be updated for 
each version release (gates)
 
 PIG-1346:  In unit tests Util.executeShellCommand relies on java commands

Modified: 
hadoop/pig/branches/branch-0.6/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/partitioners/WeightedRangePartitioner.java
URL: 
http://svn.apache.org/viewvc/hadoop/pig/branches/branch-0.6/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/partitioners/WeightedRangePartitioner.java?rev=935981&r1=935980&r2=935981&view=diff
==============================================================================
--- 
hadoop/pig/branches/branch-0.6/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/partitioners/WeightedRangePartitioner.java
 (original)
+++ 
hadoop/pig/branches/branch-0.6/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/partitioners/WeightedRangePartitioner.java
 Tue Apr 20 16:24:34 2010
@@ -103,17 +103,12 @@ public class WeightedRangePartitioner im
                             new DiscreteProbabilitySampleGenerator(probVec));
                 }
             }
-            else
-            {
-                ArrayList<Pair<FileSpec, Boolean>> inp = 
(ArrayList<Pair<FileSpec, 
Boolean>>)ObjectSerializer.deserialize(job.get("pig.inputs", ""));
-                String inputFileName = inp.get(0).first.getFileName();
-                long inputSize = FileLocalizer.getSize(inputFileName);
-                if (inputSize!=0)
-                {
-                    throw new RuntimeException("Empty samples file and 
non-empty input file");
-                }
-                // Otherwise, we do not put anything to weightedParts
-            }
+            // else - the quantiles file is empty - unless we have a bug, the 
+            // input must also be empty in which case we don't need to put
+            // anything in weightedParts since getPartition() should never get
+            // called. If the quantiles file is empty due to either a bug or
+            // a transient failure situation on the dfs, then weightedParts 
will
+            // not be populated and the job will fail in getPartition()
         }catch (Exception e){
             throw new RuntimeException(e);
         }


Reply via email to