[
https://issues.apache.org/jira/browse/HIVE-10781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14553969#comment-14553969
]
Chinna Rao Lalam commented on HIVE-10781:
-----------------------------------------
Hi [~nemon],
org.apache.hadoop.hive.ql.exec.mr.HadoopJobExecHelper.runningJobs holds running
jobs, if any abrupt shutdown happens shutdown hook uses this running jobs to
kill. But in a normal flow after executing the running job it should kill and
remove from the list.
In normal flow i suspect before removing from the list, if some exception was
thrown (while doing rj.killJob()) it won't remove from the list. I have
uploaded a patch to fix this..
Please check this if it helps
> HadoopJobExecHelper Leaks RunningJobs
> -------------------------------------
>
> Key: HIVE-10781
> URL: https://issues.apache.org/jira/browse/HIVE-10781
> Project: Hive
> Issue Type: Bug
> Components: Hive, HiveServer2
> Affects Versions: 0.13.1, 1.2.0
> Reporter: Nemon Lou
> Attachments: HIVE-10781.patch
>
>
> On one of our busy hadoop cluster, hiveServer2 holds more than 4000
> org.apache.hadoop.mapred.JobClient$NetworkedJob instances,while only has less
> than 3 backgroud handler thread at the same time.
> All these instances are hold in one LinkedList from
> org.apache.hadoop.hive.ql.exec.mr.HadoopJobExecHelper's runningJobs
> property,which is static.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
