[
https://issues.apache.org/jira/browse/HIVE-10453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14517324#comment-14517324
]
Yongzhi Chen commented on HIVE-10453:
-------------------------------------
The file is opened when:
Resource res = ucp.getResource(path, false);
{noformat}
URLClassLoader$1.run() line: 358 [local variables unavailable]
URLClassLoader$1.run() line: 355
AccessController.doPrivileged(PrivilegedExceptionAction<T>,
AccessControlContext) line: not available [native method]
URLClassLoader.findClass(String) line: 354
URLClassLoader(ClassLoader).loadClass(String, boolean) line: 425
URLClassLoader(ClassLoader).loadClass(String) line: 358
Class<T>.forName0(String, boolean, ClassLoader) line: not available [native
method]
Class<T>.forName(String, boolean, ClassLoader) line: 270
Registry.registerToSessionRegistry(String, FunctionInfo) line: 460
Registry.getQualifiedFunctionInfo(String) line: 438
Registry.getFunctionInfo(String) line: 250
FunctionRegistry.getFunctionInfo(String) line: 465
FunctionRegistry.impliesOrder(String) line: 1523
CalcitePlanner(SemanticAnalyzer).doPhase1GetAllAggregations(ASTNode,
HashMap<String,ASTNode>, List<ASTNode>) line: 526
{noformat}
And closed by:
List<IOException> errors = ucp.closeLoaders();
{noformat}
URLClassLoader.close() line: 286 [local variables unavailable]
JavaUtils.closeClassLoader(ClassLoader) line: 110
JavaUtils.closeClassLoadersTo(ClassLoader, ClassLoader) line: 87
SessionState.close() line: 1450
HiveSessionImplwithUGI(HiveSessionImpl).close() line: 566
HiveSessionImplwithUGI.close() line: 110
NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available
[native method]
NativeMethodAccessorImpl.invoke(Object, Object[]) line: 57
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43
Method.invoke(Object, Object...) line: 606
HiveSessionProxy.invoke(Method, Object[]) line: 78
HiveSessionProxy.access$000(HiveSessionProxy, Method, Object[]) line: 36
HiveSessionProxy$1.run() line: 63
AccessController.doPrivileged(PrivilegedExceptionAction<T>,
AccessControlContext) line: not available [native method]
Subject.doAs(Subject, PrivilegedExceptionAction<T>) line: 415
UserGroupInformation.doAs(PrivilegedExceptionAction<T>) line: 1628
HiveSessionProxy.invoke(Object, Method, Object[]) line: 59
$Proxy23.close() line: not available
SessionManager.closeSession(SessionHandle) line: 279
CLIService.closeSession(SessionHandle) line: 237
{noformat}
The test sometimes fails sometimes succeeds, these are java API, not sure why
it has random behaviors.
> HS2 leaking open file descriptors when using UDFs
> -------------------------------------------------
>
> Key: HIVE-10453
> URL: https://issues.apache.org/jira/browse/HIVE-10453
> Project: Hive
> Issue Type: Bug
> Reporter: Yongzhi Chen
> Assignee: Yongzhi Chen
>
> 1. create a custom function by
> CREATE FUNCTION myfunc AS 'someudfclass' using jar 'hdfs:///tmp/myudf.jar';
> 2. Create a simple jdbc client, just do
> connect,
> run simple query which using the function such as:
> select myfunc(col1) from sometable
> 3. Disconnect.
> Check open file for HiveServer2 by:
> lsof -p HSProcID | grep myudf.jar
> You will see the leak as:
> {noformat}
> java 28718 ychen txt REG 1,4 741 212977666
> /private/var/folders/6p/7_njf13d6h144wldzbbsfpz80000gp/T/1bfe3de0-ac63-4eba-a725-6a9840f1f8d5_resources/myudf.jar
> java 28718 ychen 330r REG 1,4 741 212977666
> /private/var/folders/6p/7_njf13d6h144wldzbbsfpz80000gp/T/1bfe3de0-ac63-4eba-a725-6a9840f1f8d5_resources/myudf.jar
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
