Santhosh Srinivasan commented on PIG-544:

Another use case where scalars also generate errors:


grunt> a = load 'student_tab.data';
grunt> store a into 'student_tab.bin' using BinStorage();
grunt> a = load 'student_tab.bin' using BinStorage() as (name: int, age: int, 
gpa: float);
grunt> dump a;

2008-11-25 16:02:40,986 [main] ERROR 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher - Error 
message from task (map) 
task_200809241441_24635_m_000000java.lang.RuntimeException : Unexpected data 
type 74 found in stream.         at 
  at org.apache.pig.builtin.BinStorage.bytesToInteger(BinStorage.java:169)      

> Utf8StorageConverter.java does not always produce NULLs when data is malformed
> ------------------------------------------------------------------------------
>                 Key: PIG-544
>                 URL: https://issues.apache.org/jira/browse/PIG-544
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Olga Natkovich
> It does so for scalar types but not for complext types and not for the fields 
> inside of the complext types.
> This is because it uses different code to parse scalar types by themselves 
> and scalar types inside of a complex type. It should really use the same (its 
> own code to do so.)
> The code it is currently uses, is inside of TextDataParser.jjt and is also 
> used to parse constants so we need to be careful if we want to make changes 
> to it.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

Reply via email to