[
https://issues.apache.org/jira/browse/HIVE-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13755999#comment-13755999
]
Sivaramakrishnan Narayanan commented on HIVE-3969:
--------------------------------------------------
[~navis] please see my blog on the subject
[here|http://www.qubole.com/blog/qubole-hive-server/]. It is insufficient to
reset the classloader as this doesn't close connections to jar files. I've
pasted the relevant part of the blog here:
There was one issue that was a little perplexing. After running for a week or
so, QHS (Qubole Hive Server) starting throwing “too many files open”
exceptions. A quick lsof call confirmed that there were numerous open file
handles. Surprisingly, though, these all pointed to jar files. After some
investigation, we found that the URLClassLoader leaks file handles to jars it
opens (see this
[link|http://management-platform.blogspot.in/2009/01/classloaders-keeping-jar-files-open.html]
for some dirty details). These are never garbage collected. We ended up using
the non-standard ClassLoaderUtil.releaseLoader to free up resources. Java 7 has
a nicer solution for this where URLClassLoader has a close method that performs
the necessary cleanup.
> Session state for hive server should be cleanup
> -----------------------------------------------
>
> Key: HIVE-3969
> URL: https://issues.apache.org/jira/browse/HIVE-3969
> Project: Hive
> Issue Type: Bug
> Components: Server Infrastructure
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3969.D8325.1.patch
>
>
> Currently "add jar" command by clients are adding child ClassLoader to worker
> thread cumulatively, causing various problems.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
