Author: sms
Date: Tue Mar 3 04:24:10 2009
New Revision: 749526
URL: http://svn.apache.org/viewvc?rev=749526&view=rev
Log:
PIG-691: BinStorage skips tuples when ^A is present in data (pradeepkth via sms)
Modified:
hadoop/pig/trunk/CHANGES.txt
hadoop/pig/trunk/src/org/apache/pig/builtin/BinStorage.java
Modified: hadoop/pig/trunk/CHANGES.txt
URL:
http://svn.apache.org/viewvc/hadoop/pig/trunk/CHANGES.txt?rev=749526&r1=749525&r2=749526&view=diff
==============================================================================
--- hadoop/pig/trunk/CHANGES.txt (original)
+++ hadoop/pig/trunk/CHANGES.txt Tue Mar 3 04:24:10 2009
@@ -440,3 +440,5 @@
PIG-655: Comparison of schemas of bincond operands is flawed (sms via
pradeepkth)
+ PIG-691: BinStorage skips tuples when ^A is present in data (pradeepkth
+ via sms)
Modified: hadoop/pig/trunk/src/org/apache/pig/builtin/BinStorage.java
URL:
http://svn.apache.org/viewvc/hadoop/pig/trunk/src/org/apache/pig/builtin/BinStorage.java?rev=749526&r1=749525&r2=749526&view=diff
==============================================================================
--- hadoop/pig/trunk/src/org/apache/pig/builtin/BinStorage.java (original)
+++ hadoop/pig/trunk/src/org/apache/pig/builtin/BinStorage.java Tue Mar 3
04:24:10 2009
@@ -70,11 +70,18 @@
if (in == null || in.getPosition() >=end) {
return null;
}
- b = (byte) in.read();
- if(b != RECORD_1 && b != -1) {
- continue;
+ // check if we saw RECORD_1 in our last attempt
+ // this can happen if we have the following
+ // sequence RECORD_1-RECORD_1-RECORD_2-RECORD_3
+ // After reading the second RECORD_1 in the above
+ // sequence, we should not look for RECORD_1 again
+ if(b != RECORD_1) {
+ b = (byte) in.read();
+ if(b != RECORD_1 && b != -1) {
+ continue;
+ }
+ if(b == -1) return null;
}
- if(b == -1) return null;
b = (byte) in.read();
if(b != RECORD_2 && b != -1) {
continue;
