[jira] [Commented] (HIVE-6477) Aggregation functions for tiny/smallint broken with parquet
[
https://issues.apache.org/jira/browse/HIVE-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13911237#comment-13911237
]
Hive QA commented on HIVE-6477:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12630585/HIVE-6477.2.patch
{color:green}SUCCESS:{color} +1 5177 tests passed
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1482/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1482/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12630585
Aggregation functions for tiny/smallint broken with parquet
---
Key: HIVE-6477
URL: https://issues.apache.org/jira/browse/HIVE-6477
Project: Hive
Issue Type: Bug
Components: Serializers/Deserializers
Affects Versions: 0.13.0
Reporter: Szehon Ho
Assignee: Szehon Ho
Labels: parquet
Attachments: HIVE-6477.2.patch, HIVE-6477.patch
Given the following table:
{noformat}
CREATE TABLE IF NOT EXISTS commontypesagg (
id int,
bool_col boolean,
tinyint_col tinyint,
smallint_col smallint,
int_col int,
bigint_col bigint,
float_col float,
double_col double,
date_string_col string,
string_col string)
PARTITIONED BY (year int, month int, day int)
STORED AS PARQUET;
{noformat}
The following queries throws ClassCastException:
{noformat}
select count(tinyint_col), min(tinyint_col), max(tinyint_col),
sum(tinyint_col) from commontypesagg;
select count(smallint_col), min(smallint_col), max(smallint_col),
sum(smallint_col) from commontypesagg;
{noformat}
Exception is the following:
{noformat}
2014-01-29 14:02:11,381 INFO org.apache.hadoop.mapred.TaskStatus:
task-diagnostic-info for task attempt_201401290934_0006_m_01_1 :
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
Hive Runtime Error while processing row
{id:null,bool_col:null,tinyint_col:1,smallint_col:null,int_col:null,bigint_col:null,float_col:null,double_col:null,date_string_col:null,string_col:null,year:2009,month:1}
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:175)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:417)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing row
{id:null,bool_col:null,tinyint_col:1,smallint_col:null,int_col:null,bigint_col:null,float_col:null,double_col:null,date_string_col:null,string_col:null,year:2009,month:1}
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:529)
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:157)
... 8 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.ClassCastException: org.apache.hadoop.io.IntWritable cannot be cast
to java.lang.Byte
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:796)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:844)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:87)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:844)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:91)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:844)
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:519)
... 9 more
Caused by: java.lang.ClassCastException: org.apache.hadoop.io.IntWritable
cannot be cast to java.lang.Byte
at
org.apache.hadoop.hive.serde2.objectinspector.primitive.JavaByteObjectInspector.get(JavaByteObjectInspector.java:40)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.compare(ObjectInspectorUtils.java:666)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.compare(ObjectInspectorUtils.java:631)
at
[jira] [Commented] (HIVE-6477) Aggregation functions for tiny/smallint broken with parquet
[
https://issues.apache.org/jira/browse/HIVE-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13909761#comment-13909761
]
Hive QA commented on HIVE-6477:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12630191/HIVE-6477.patch
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 5176 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_agg
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1451/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1451/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12630191
Aggregation functions for tiny/smallint broken with parquet
---
Key: HIVE-6477
URL: https://issues.apache.org/jira/browse/HIVE-6477
Project: Hive
Issue Type: Bug
Components: Serializers/Deserializers
Affects Versions: 0.13.0
Reporter: Szehon Ho
Assignee: Szehon Ho
Labels: parquet
Attachments: HIVE-6477.patch
Given the following table:
{noformat}
CREATE TABLE IF NOT EXISTS commontypesagg (
id int,
bool_col boolean,
tinyint_col tinyint,
smallint_col smallint,
int_col int,
bigint_col bigint,
float_col float,
double_col double,
date_string_col string,
string_col string)
PARTITIONED BY (year int, month int, day int)
STORED AS PARQUET;
{noformat}
The following queries throws ClassCastException:
{noformat}
select count(tinyint_col), min(tinyint_col), max(tinyint_col),
sum(tinyint_col) from commontypesagg;
select count(smallint_col), min(smallint_col), max(smallint_col),
sum(smallint_col) from commontypesagg;
{noformat}
Exception is the following:
{noformat}
2014-01-29 14:02:11,381 INFO org.apache.hadoop.mapred.TaskStatus:
task-diagnostic-info for task attempt_201401290934_0006_m_01_1 :
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
Hive Runtime Error while processing row
{id:null,bool_col:null,tinyint_col:1,smallint_col:null,int_col:null,bigint_col:null,float_col:null,double_col:null,date_string_col:null,string_col:null,year:2009,month:1}
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:175)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:417)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing row
{id:null,bool_col:null,tinyint_col:1,smallint_col:null,int_col:null,bigint_col:null,float_col:null,double_col:null,date_string_col:null,string_col:null,year:2009,month:1}
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:529)
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:157)
... 8 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.ClassCastException: org.apache.hadoop.io.IntWritable cannot be cast
to java.lang.Byte
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:796)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:844)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:87)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:844)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:91)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:844)
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:519)
... 9 more
Caused by: java.lang.ClassCastException: org.apache.hadoop.io.IntWritable
cannot be cast to java.lang.Byte
at
org.apache.hadoop.hive.serde2.objectinspector.primitive.JavaByteObjectInspector.get(JavaByteObjectInspector.java:40)
