Author: gates
Date: Fri May 15 21:08:27 2009
New Revision: 775340
URL: http://svn.apache.org/viewvc?rev=775340&view=rev
Log:
PIG-619: Create one InputSplit even when the input file is zero length so that
hadoop runs maps and creates output for the next job .
Modified:
hadoop/pig/trunk/CHANGES.txt
hadoop/pig/trunk/src/org/apache/pig/backend/executionengine/PigSlicer.java
Modified: hadoop/pig/trunk/CHANGES.txt
URL:
http://svn.apache.org/viewvc/hadoop/pig/trunk/CHANGES.txt?rev=775340&r1=775339&r2=775340&view=diff
==============================================================================
--- hadoop/pig/trunk/CHANGES.txt (original)
+++ hadoop/pig/trunk/CHANGES.txt Fri May 15 21:08:27 2009
@@ -24,6 +24,10 @@
IMPROVEMENTS
+PIG-619: Create one InputSplit even when the input file is zero length
+ so that hadoop runs maps and creates output for the next
+ job (gates).
+
PIG-693: Proposed improvements to pig's optimizer (sms)
PIG-700: To automate the pig patch test process (gkesavan via sms)
Modified:
hadoop/pig/trunk/src/org/apache/pig/backend/executionengine/PigSlicer.java
URL:
http://svn.apache.org/viewvc/hadoop/pig/trunk/src/org/apache/pig/backend/executionengine/PigSlicer.java?rev=775340&r1=775339&r2=775340&view=diff
==============================================================================
--- hadoop/pig/trunk/src/org/apache/pig/backend/executionengine/PigSlicer.java
(original)
+++ hadoop/pig/trunk/src/org/apache/pig/backend/executionengine/PigSlicer.java
Fri May 15 21:08:27 2009
@@ -100,6 +100,12 @@
// Anything that ends with a ".gz" we must process as a
complete
// file
slices.add(new PigSlice(name, funcSpec, 0, size));
+ } else if (size == 0) {
+ // add one empty slice. This is a total hack to deal with the
+ // case where hadoop isn't starting maps for
empty arrays of
+ // InputSplits. See PIG-619. This should be
removed
+ // once we determine why this is.
+ slices.add(new PigSlice(name, funcSpec, 0, bs));
} else {
while (pos < size) {
if (pos + bs > size) {
