[
https://issues.apache.org/jira/browse/PIG-63?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12560544#action_12560544
]
Alan Gates commented on PIG-63:
-------------------------------
When I apply this patch and run the tests, they fail with the following error
message:
08/01/18 11:50:16 ERROR apache.pig: Error message from task (map)
tip_200801071337_2639_m_000000 java.lang.NullPointerException
at
org.apache.pig.impl.io.BufferedPositionedInputStream.readLine(BufferedPositionedInputStream.java:146)
at org.apache.pig.builtin.PigStorage.getNext(PigStorage.java:62)
at
org.apache.pig.impl.mapreduceExec.PigInputFormat$PigRecordReader.next(PigInputFormat.java:188)
at
org.apache.pig.impl.mapreduceExec.PigInputFormat$PigRecordReader.next(PigInputFormat.java:131)
at org.apache.hadoop.mapred.MapTask$1.next(MapTask.java:174)
at
org.apache.pig.impl.mapreduceExec.PigMapReduce.run(PigMapReduce.java:114)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:192)
at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1760)
> PigStorage does not properly handle UTF8 data
> ---------------------------------------------
>
> Key: PIG-63
> URL: https://issues.apache.org/jira/browse/PIG-63
> Project: Pig
> Issue Type: Bug
> Reporter: Olga Natkovich
> Assignee: Benjamin Reed
> Attachments: utf8.patch, utf8.patch
>
>
> From Ben:
> I just checked the code and the problem seems to be PigStorage. getNext() uses
> readLine() which does not handle UTF8 correctly. putNext() also uses default
> encoder rather than UTF8 explicitly.
> Internally and in BinStorage UTF8 appears to be handled correctly.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
