[
https://issues.apache.org/jira/browse/HIVE-18593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amruth S updated HIVE-18593:
----------------------------
Description:
Vectorisation with some queries seem to be failing with null pointer
exceptions. This happens only with 2.3.2 release and not the older ones.
It seems in this method (BytesColumnVector.java, vector[0] is null, isRepeating
is true, length is 0)
{code:java}
public void copySelected(
boolean selectedInUse, int[] sel, int size, BytesColumnVector output) {
// Output has nulls if and only if input has nulls.
output.noNulls = noNulls;
output.isRepeating = false;
// Handle repeating case
if (isRepeating) {
output.setVal(0, vector[0], start[0], length[0]);
output.isNull[0] = isNull[0];
output.isRepeating = true;
return;
}
{code}
Exception trace below
{code:java}
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error
while processing row
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:883)
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:86)
... 17 more
Caused by: java.lang.NullPointerException
at java.lang.System.arraycopy(Native Method)
at
org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:173)
at
org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.copySelected(BytesColumnVector.java:321)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprStringGroupColumnStringGroupColumn.evaluate(IfExprStringGroupColumnStringGroupColumn.java:85)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.gen.VectorUDAFMaxString.aggregateInputSelection(VectorUDAFMaxString.java:135)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeBase.processAggregators(VectorGroupByOperator.java:218)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeHashAggregate.doProcessBatch(VectorGroupByOperator.java:408)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeBase.processBatch(VectorGroupByOperator.java:179)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.process(VectorGroupByOperator.java:1021)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
at
org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:137)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
at
org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator.process(VectorFilterOperator.java:123)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:130)
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:783)
... 18 more
{code}
was:
Vectorisation with some queries seem to be failing with null pointer
exceptions. This happens only with 2.3.2 release and not the older ones.
It seems in this method (BytesColumnVector.java, vector[0] is null, if the
complete batch is null)
{code:java}
public void copySelected(
boolean selectedInUse, int[] sel, int size, BytesColumnVector output) {
// Output has nulls if and only if input has nulls.
output.noNulls = noNulls;
output.isRepeating = false;
// Handle repeating case
if (isRepeating) {
output.setVal(0, vector[0], start[0], length[0]);
output.isNull[0] = isNull[0];
output.isRepeating = true;
return;
}
{code}
Exception trace below
{code:java}
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error
while processing row
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:883)
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:86)
... 17 more
Caused by: java.lang.NullPointerException
at java.lang.System.arraycopy(Native Method)
at
org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:173)
at
org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.copySelected(BytesColumnVector.java:321)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprStringGroupColumnStringGroupColumn.evaluate(IfExprStringGroupColumnStringGroupColumn.java:85)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.gen.VectorUDAFMaxString.aggregateInputSelection(VectorUDAFMaxString.java:135)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeBase.processAggregators(VectorGroupByOperator.java:218)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeHashAggregate.doProcessBatch(VectorGroupByOperator.java:408)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeBase.processBatch(VectorGroupByOperator.java:179)
at
org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.process(VectorGroupByOperator.java:1021)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
at
org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:137)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
at
org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator.process(VectorFilterOperator.java:123)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:130)
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:783)
... 18 more
{code}
> NPE on vectorization group by
> -----------------------------
>
> Key: HIVE-18593
> URL: https://issues.apache.org/jira/browse/HIVE-18593
> Project: Hive
> Issue Type: Bug
> Components: Hive, HiveServer2
> Affects Versions: 2.3.2
> Reporter: Amruth S
> Priority: Major
>
> Vectorisation with some queries seem to be failing with null pointer
> exceptions. This happens only with 2.3.2 release and not the older ones.
> It seems in this method (BytesColumnVector.java, vector[0] is null,
> isRepeating is true, length is 0)
> {code:java}
> public void copySelected(
> boolean selectedInUse, int[] sel, int size, BytesColumnVector output) {
> // Output has nulls if and only if input has nulls.
> output.noNulls = noNulls;
> output.isRepeating = false;
> // Handle repeating case
> if (isRepeating) {
> output.setVal(0, vector[0], start[0], length[0]);
> output.isNull[0] = isNull[0];
> output.isRepeating = true;
> return;
> }
> {code}
> Exception trace below
> {code:java}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:883)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:86)
> ... 17 more
> Caused by: java.lang.NullPointerException
> at java.lang.System.arraycopy(Native Method)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:173)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.copySelected(BytesColumnVector.java:321)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprStringGroupColumnStringGroupColumn.evaluate(IfExprStringGroupColumnStringGroupColumn.java:85)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.gen.VectorUDAFMaxString.aggregateInputSelection(VectorUDAFMaxString.java:135)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeBase.processAggregators(VectorGroupByOperator.java:218)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeHashAggregate.doProcessBatch(VectorGroupByOperator.java:408)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeBase.processBatch(VectorGroupByOperator.java:179)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.process(VectorGroupByOperator.java:1021)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:137)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator.process(VectorFilterOperator.java:123)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:130)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:783)
> ... 18 more
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
