MIN on strings (undeclared) gives strange error in store
--------------------------------------------------------
Key: PIG-333
URL: https://issues.apache.org/jira/browse/PIG-333
Project: Pig
Issue Type: Bug
Affects Versions: types_branch
Reporter: Pradeep Kamath
Priority: Minor
Fix For: types_branch
Script which causes error:
{code}
a = load '/user/pig/tests/data/singlefile/votertab10k' as (name, age,
registration, contribution);
b = group a all;
c = foreach b generate MIN(a.name), MAX(a.name);
store c into '/tmp';
{code}
Error:
{noformat}
2008-07-23 11:31:15,415 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher - 0.0%
complete
2008-07-23 11:31:19,167 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher - 50.0%
complete
2008-07-23 11:31:43,431 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher - 100.0%
complete
2008-07-23 11:31:45,956 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher -
Unsuccessful attempt. Completed 0.0% of the job
2008-07-23 11:31:45,969 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher - Error
message from task (map) tip_200807231111_0002_m_000000
2008-07-23 11:31:45,974 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher - Error
message from task (reduce) tip_200807231111_0002_r_000000 java.io.IOException:
Cannot store a non-flat tuple using PigStorage
at org.apache.pig.builtin.PigStorage.putNext(PigStorage.java:163)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:117)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:90)
at org.apache.hadoop.mapred.ReduceTask$3.collect(ReduceTask.java:373)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:170)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:85)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:391)
at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
java.io.IOException: Cannot store a non-flat tuple using PigStorage
at org.apache.pig.builtin.PigStorage.putNext(PigStorage.java:163)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:117)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:90)
at org.apache.hadoop.mapred.ReduceTask$3.collect(ReduceTask.java:373)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:170)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:85)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:391)
at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
java.io.IOException: Cannot store a non-flat tuple using PigStorage
at org.apache.pig.builtin.PigStorage.putNext(PigStorage.java:163)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:117)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:90)
at org.apache.hadoop.mapred.ReduceTask$3.collect(ReduceTask.java:373)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:170)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:85)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:391)
at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
java.io.IOException: Cannot store a non-flat tuple using PigStorage
at org.apache.pig.builtin.PigStorage.putNext(PigStorage.java:163)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:117)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:90)
at org.apache.hadoop.mapred.ReduceTask$3.collect(ReduceTask.java:373)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:170)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:85)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:391)
at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
{noformat}
MIN in the default case converts the ByteArray input to a Double and tries to
compute a MIN of the doubles. Since in the above query the column actually
contains strings, the conversion from string to double should fail with a
NumberFormat exception - instead the error shows up differently in the store
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
