[
https://issues.apache.org/jira/browse/HIVE-8272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14154598#comment-14154598
]
Jason Dere commented on HIVE-8272:
----------------------------------
Here's the plan with the patch
{noformat}
LOGICAL PLAN:
src
TableScan (TS_0)
alias: src
Statistics: Num rows: 58 Data size: 5812 Basic stats: COMPLETE Column
stats: NONE
Select Operator (SEL_1)
expressions: key (type: string)
outputColumnNames: key
Statistics: Num rows: 58 Data size: 5812 Basic stats: COMPLETE Column
stats: NONE
Group By Operator (GBY_2)
aggregations: sum(key)
mode: hash
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column
stats: NONE
Reduce Output Operator (RS_3)
sort order:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column
stats: NONE
value expressions: _col0 (type: double)
Group By Operator (GBY_4)
aggregations: sum(VALUE._col0)
mode: mergepartial
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column
stats: NONE
Select Operator (SEL_5)
expressions: CAST( (_col0 * 100) AS decimal(15,3)) (type:
decimal(15,3))
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column
stats: NONE
Reduce Output Operator (RS_6)
key expressions: _col0 (type: decimal(15,3))
sort order: +
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE
Column stats: NONE
Select Operator (SEL_7)
expressions: KEY.reducesinkkey0 (type: decimal(15,3))
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE
Column stats: NONE
File Output Operator (FS_8)
compressed: false
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE
Column stats: NONE
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde:
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
{noformat}
Without the patch, ReduceSinkDeDuplication.merge() was incorrectly merging RS6
and RS3, even though the sort key in R6 could not be resolved by R3. The patch
fixes backtrack() to return null if the ExprNode children of a function resolve
to null, which causes the merge to fail.
> Query with particular decimal expression causes NPE during execution
> initialization
> -----------------------------------------------------------------------------------
>
> Key: HIVE-8272
> URL: https://issues.apache.org/jira/browse/HIVE-8272
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer, Physical Optimizer
> Reporter: Matt McCline
> Assignee: Jason Dere
> Priority: Critical
> Fix For: 0.14.0
>
> Attachments: HIVE-8272.1.patch
>
>
> Query:
> {code}
> select
> cast(sum(dc)*100 as decimal(11,3)) as c1
> from somedecimaltable
> order by c1
> limit 100;
> {code}
> Fails during execution initialization due to *null* ExprNodeDesc.
> Noticed while trying to simplify a Vectorization issue and realized it was a
> more general issue.
> {code}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:154)
> ... 22 more
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initializeOp(ReduceSinkOperator.java:215)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:464)
> at
> org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:420)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:427)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:464)
> at
> org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:420)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:65)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:464)
> at
> org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:420)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:193)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:425)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:380)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:133)
> ... 22 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc.getExprString(ExprNodeGenericFuncDesc.java:154)
> at
> org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc.getExprString(ExprNodeGenericFuncDesc.java:154)
> at
> org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initializeOp(ReduceSinkOperator.java:148)
> ... 38 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
