Author: sms
Date: Tue Mar  3 04:24:10 2009
New Revision: 749526

URL: http://svn.apache.org/viewvc?rev=749526&view=rev
Log:
PIG-691: BinStorage skips tuples when ^A is present in data (pradeepkth via sms)

Modified:
    hadoop/pig/trunk/CHANGES.txt
    hadoop/pig/trunk/src/org/apache/pig/builtin/BinStorage.java

Modified: hadoop/pig/trunk/CHANGES.txt
URL: 
http://svn.apache.org/viewvc/hadoop/pig/trunk/CHANGES.txt?rev=749526&r1=749525&r2=749526&view=diff
==============================================================================
--- hadoop/pig/trunk/CHANGES.txt (original)
+++ hadoop/pig/trunk/CHANGES.txt Tue Mar  3 04:24:10 2009
@@ -440,3 +440,5 @@
     PIG-655: Comparison of schemas of bincond operands is flawed (sms via
     pradeepkth)
 
+    PIG-691: BinStorage skips tuples when ^A is present in data (pradeepkth
+    via sms)

Modified: hadoop/pig/trunk/src/org/apache/pig/builtin/BinStorage.java
URL: 
http://svn.apache.org/viewvc/hadoop/pig/trunk/src/org/apache/pig/builtin/BinStorage.java?rev=749526&r1=749525&r2=749526&view=diff
==============================================================================
--- hadoop/pig/trunk/src/org/apache/pig/builtin/BinStorage.java (original)
+++ hadoop/pig/trunk/src/org/apache/pig/builtin/BinStorage.java Tue Mar  3 
04:24:10 2009
@@ -70,11 +70,18 @@
             if (in == null || in.getPosition() >=end) {
                 return null;
             }
-            b = (byte) in.read();
-            if(b != RECORD_1 && b != -1) {
-                continue;
+            // check if we saw RECORD_1 in our last attempt
+            // this can happen if we have the following 
+            // sequence RECORD_1-RECORD_1-RECORD_2-RECORD_3
+            // After reading the second RECORD_1 in the above
+            // sequence, we should not look for RECORD_1 again
+            if(b != RECORD_1) {
+                b = (byte) in.read();
+                if(b != RECORD_1 && b != -1) {
+                    continue;
+                }
+                if(b == -1) return null;
             }
-            if(b == -1) return null;
             b = (byte) in.read();
             if(b != RECORD_2 && b != -1) {
                 continue;


Reply via email to