[ 
https://issues.apache.org/jira/browse/PIG-544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12650810#action_12650810
 ] 

Santhosh Srinivasan commented on PIG-544:
-----------------------------------------

Another use case where scalars also generate errors:

{code}

grunt> a = load 'student_tab.data';
grunt> store a into 'student_tab.bin' using BinStorage();
grunt> a = load 'student_tab.bin' using BinStorage() as (name: int, age: int, 
gpa: float);
grunt> dump a;

2008-11-25 16:02:40,986 [main] ERROR 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher - Error 
message from task (map) 
task_200809241441_24635_m_000000java.lang.RuntimeException : Unexpected data 
type 74 found in stream.         at 
org.apache.pig.data.DataReaderWriter.readDatum(DataReaderWriter.java:115)       
  at org.apache.pig.builtin.BinStorage.bytesToInteger(BinStorage.java:169)      
   at 
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast.getNext(POCast.java:143)
 
....
{code}

> Utf8StorageConverter.java does not always produce NULLs when data is malformed
> ------------------------------------------------------------------------------
>
>                 Key: PIG-544
>                 URL: https://issues.apache.org/jira/browse/PIG-544
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Olga Natkovich
>
> It does so for scalar types but not for complext types and not for the fields 
> inside of the complext types.
> This is because it uses different code to parse scalar types by themselves 
> and scalar types inside of a complex type. It should really use the same (its 
> own code to do so.)
> The code it is currently uses, is inside of TextDataParser.jjt and is also 
> used to parse constants so we need to be careful if we want to make changes 
> to it.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply via email to