[
https://issues.apache.org/jira/browse/HIVE-18413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-18413:
------------------------------------
Description:
exposed by: HIVE-18359
in case of vectorization, the summary row object was left as is (presumed null
earlier); which may cause it to be inconsistent isNull conditions in
.VectorHashKeyWrapperBatch
issue happens only if:
* vectorizable groupby
* groupping set contains empty
* non-trivial empty; mapper is run
* groupping key is select ; with a type which is backed by a bytea; ex:string
{code}
set hive.vectorized.execution.enabled=true;
create table tx2 (a integer,b integer,c integer,d double,u string,bi binary)
stored as orc;
insert into tx2 values
(1,2,3,1.1,'x','b'),
(3,2,3,1.1,'y','b');
select sum(a),
u,
bi,
'asd',
grouping(bi),
'NULL,1' as expected
from tx2
where a=2
group by a,u,bi grouping sets ( u, (), bi);
{code}
causes:
{code}
Caused by: java.lang.NullPointerException
at java.lang.System.arraycopy(Native Method)
at
org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:173)
at
org.apache.hadoop.hive.ql.exec.vector.VectorHashKeyWrapperBatch.assignRowColumn(VectorHashKeyWrapperBatch.java:1065)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.writeSingleRow(VectorGroupByOperator.java:1134)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.access$800(VectorGroupByOperator.java:74)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeReduceMergePartial.close(VectorGroupByOperator.java:862)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.closeOp(VectorGroupByOperator.java:1176)
at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:705)
at
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:383)
... 16 more
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1
killedTasks:0, Vertex vertex_1515531021543_0001_12_01 [Reducer 2] killed/failed
due to:OWN_TASK_FAILURE]DAG did not succeed due to VERTEX_FAILURE.
failedVertices:1 killedVertices:0
2018-01-09T12:50:30,611 DEBUG [01fdcefd-40b0-45a6-8e5b-b1cd14241088 main]
ql.Driver: Shutting down query
{code}
was:
exposed by: HIVE-18359
in case of vectorization the summary row object was left as is; which may cause
it to be inconsistent with the column isNull conditions
> Grouping of an empty result set may only contain null values
> ------------------------------------------------------------
>
> Key: HIVE-18413
> URL: https://issues.apache.org/jira/browse/HIVE-18413
> Project: Hive
> Issue Type: Bug
> Reporter: Zoltan Haindrich
> Assignee: Zoltan Haindrich
> Attachments: HIVE-18413.01.patch
>
>
> exposed by: HIVE-18359
> in case of vectorization, the summary row object was left as is (presumed
> null earlier); which may cause it to be inconsistent isNull conditions in
> .VectorHashKeyWrapperBatch
> issue happens only if:
> * vectorizable groupby
> * groupping set contains empty
> * non-trivial empty; mapper is run
> * groupping key is select ; with a type which is backed by a bytea; ex:string
> {code}
> set hive.vectorized.execution.enabled=true;
> create table tx2 (a integer,b integer,c integer,d double,u string,bi binary)
> stored as orc;
> insert into tx2 values
> (1,2,3,1.1,'x','b'),
> (3,2,3,1.1,'y','b');
> select sum(a),
> u,
> bi,
> 'asd',
> grouping(bi),
> 'NULL,1' as expected
> from tx2
> where a=2
> group by a,u,bi grouping sets ( u, (), bi);
> {code}
> causes:
> {code}
> Caused by: java.lang.NullPointerException
> at java.lang.System.arraycopy(Native Method)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:173)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorHashKeyWrapperBatch.assignRowColumn(VectorHashKeyWrapperBatch.java:1065)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.writeSingleRow(VectorGroupByOperator.java:1134)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.access$800(VectorGroupByOperator.java:74)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeReduceMergePartial.close(VectorGroupByOperator.java:862)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.closeOp(VectorGroupByOperator.java:1176)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:705)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:383)
> ... 16 more
> ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1
> killedTasks:0, Vertex vertex_1515531021543_0001_12_01 [Reducer 2]
> killed/failed due to:OWN_TASK_FAILURE]DAG did not succeed due to
> VERTEX_FAILURE. failedVertices:1 killedVertices:0
> 2018-01-09T12:50:30,611 DEBUG [01fdcefd-40b0-45a6-8e5b-b1cd14241088 main]
> ql.Driver: Shutting down query
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
