CSVLoader does not interpret empty fields to NULLs
--------------------------------------------------
Key: PIG-2567
URL: https://issues.apache.org/jira/browse/PIG-2567
Project: Pig
Issue Type: Bug
Components: piggybank
Affects Versions: 0.8.1
Reporter: Ido Hadanny
Priority: Minor
when specifying an empty field like so "a,,b"
CSVLoader turns them into empty strings.
while PigStorage converts them to nulls.
PigStorage:
private void readField(byte[] buf, int start, int end) {
if (start == end) {
// NULL value
mProtoTuple.add(null);
} else {
mProtoTuple.add(new DataByteArray(buf, start, end));
}
}
CSVLoader:
private void readField(ByteBuffer buf, int fieldID) {
158
if (mRequiredColumns==null || (mRequiredColumns.length>fieldID &&
mRequiredColumns[fieldID])) {
159
byte[] bytes = new byte[buf.position()];
160
buf.rewind();
161
buf.get(bytes, 0, bytes.length);
162
mProtoTuple.add(new DataByteArray(bytes));
163
}
164
buf.clear();
165
}
166
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
