[
https://issues.apache.org/jira/browse/PIG-771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12703165#action_12703165
]
Daniel Dai commented on PIG-771:
--------------------------------
Seems it works fine on my computer either dump it on the screen or store it
into a file. My OS encoding is UTF8. Here is a log:
grunt> dan...@ubuntu-daniel1:~/pig$ cat chinese.txt
中文测试
dan...@ubuntu-daniel1:~/pig$ java -jar pig.jar
2009-04-27 11:14:10,889 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to
hadoop file system at: file:///
2009-04-27 11:14:11,046 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics -
Initializing JVM Metrics with processName=JobTracker, sessionId=
grunt> A = load '/home/daniel/pig/chinese.txt' using PigStorage() as (str:
chararray);
grunt> dump A;
2009-04-27 11:14:18,767 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics -
Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2009-04-27 11:14:18,774 [Thread-4] WARN org.apache.hadoop.mapred.JobClient -
Use GenericOptionsParser for parsing the arguments. Applications should
implement Tool for the same.
2009-04-27 11:14:19,289 [Thread-10] INFO org.apache.hadoop.mapred.MapTask -
numReduceTasks: 0
2009-04-27 11:14:19,364 [Thread-10] INFO
org.apache.hadoop.mapred.LocalJobRunner -
2009-04-27 11:14:19,368 [Thread-10] INFO org.apache.hadoop.mapred.TaskRunner -
Task 'attempt_local_0001_m_000000_0' done.
2009-04-27 11:14:19,372 [Thread-10] INFO org.apache.hadoop.mapred.TaskRunner -
Saved output of task 'attempt_local_0001_m_000000_0' to
file:/tmp/temp-480716160/tmp2138119751
2009-04-27 11:14:24,220 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 0% complete
2009-04-27 11:14:24,222 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- 100% complete
2009-04-27 11:14:24,222 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- Success!
(中文测试)
grunt>
> PigDump does not properly output Chinese UTF8 characters - they are displayed
> as question marks ??
> --------------------------------------------------------------------------------------------------
>
> Key: PIG-771
> URL: https://issues.apache.org/jira/browse/PIG-771
> Project: Pig
> Issue Type: Bug
> Reporter: David Ciemiewicz
>
> PigDump does not properly output Chinese UTF8 characters.
> The reason for this is that the function Tuple.toString() is called.
> DefaultTuple implements Tuple.toString() and it calls Object.toString() on
> the opaque object d.
> Instead, I think that the code should be changed instead to call the new
> DataType.toString() function.
> {code}
> @Override
> public String toString() {
> StringBuilder sb = new StringBuilder();
> sb.append('(');
> for (Iterator<Object> it = mFields.iterator(); it.hasNext();) {
> Object d = it.next();
> if(d != null) {
> if(d instanceof Map) {
> sb.append(DataType.mapToString((Map<Object, Object>)d));
> } else {
> sb.append(DataType.toString(d)); // <<< Change this one
> line
> if(d instanceof Long) {
> sb.append("L");
> } else if(d instanceof Float) {
> sb.append("F");
> }
> }
> } else {
> sb.append("");
> }
> if (it.hasNext())
> sb.append(",");
> }
> sb.append(')');
> return sb.toString();
> }
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
