[
https://issues.apache.org/jira/browse/HIVE-26436?focusedWorklogId=800268&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-800268
]
ASF GitHub Bot logged work on HIVE-26436:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 12/Aug/22 10:30
Start Date: 12/Aug/22 10:30
Worklog Time Spent: 10m
Work Description: ganlute commented on code in PR #3485:
URL: https://github.com/apache/hive/pull/3485#discussion_r944327963
##########
ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java:
##########
@@ -675,6 +678,13 @@ public void close(boolean abort) throws HiveException {
return;
}
+ // if the operator has not been initialized, there is no need to close
+ if (!this.rootInitializeCalled) {
+ LOG.debug("No need to close operator {}", this);
+ return;
+ }
+
+
Review Comment:
> why this is specific to MR and doesn't happen with Tez? And what is the
actual scenario/query where this happens, the extended test doesn't seems to be
very indicative
@anishek I am so sorry for the slow reply.
In fact, I can not reproduce the problem. It happened in one of my online
clusters and I can not reproduce it in my new cluster. Some of configs may be
different but I am not sure which one cause the problem.
Here is the map file printed when exception happened, It does not look like
a good format file:
```text
<MAP>Id =0
<Children>
<TS>Id =156
<Children>
<UNION>Id =26
<Children>
<SEL>Id =40
<Children>
<FS>Id =146
<Children>
<\Children>
<Parent>Id = 40 null<\Parent>
<\FS>
<\Children>
<Parent>Id = 26 null<\Parent>
<\SEL>
<\Children>
<Parent>Id = 156 nullId = 162
<TS>Id =162
<Children>null
<\Children>
<Parent><\Parent>
<\TS>Id = 166
<TS>Id =166
<Children>null
<\Children>
<Parent><\Parent>
<\TS><\Parent>
<\UNION>
<\Children>
<Parent>Id = 0 null<\Parent>
<\TS>
<\Children>
<Parent><\Parent>
<\MAP>
```
However, I think for whatever reason the operator is not initialized, close
should not throw an exception.
Issue Time Tracking
-------------------
Worklog Id: (was: 800268)
Time Spent: 1.5h (was: 1h 20m)
> Hive on MR NullPointerException
> -------------------------------
>
> Key: HIVE-26436
> URL: https://issues.apache.org/jira/browse/HIVE-26436
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
> Affects Versions: 3.1.2
> Reporter: TE
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> 2022-07-27 11:23:26,337 INFO [AsyncDispatcher event handler]
> org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics
> report from attempt_1658508427196_17228_m_000000_3: Error:
> java.lang.RuntimeException: Hive Runtime Error while closing operators
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:211)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapRunner.run(ExecMapRunner.java:37)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:465)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:349)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.Operator.getCounterName(Operator.java:502)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:742)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:757)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:757)
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:193)
> ... 9 more
>
> Hive on MR run failed and the error details is above.
> Hive version: 3.1.2
>
> When initializeOp has not been called and close called, this exception will
> happen.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
