[
https://issues.apache.org/jira/browse/HADOOP-1130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12484500
]
Owen O'Malley commented on HADOOP-1130:
---------------------------------------
Jira keeps eating my comments on this issue. *sigh*
> As hadoop may be used as a third party library, it shouldn't install shutdown
> hook by itself.
Actually, I think you have this backwards. Since Hadoop may be used as a third
party library, it must use finalizers and shutdown hooks to clean up properly.
The problems with have an explicit close method that the user must call are:
1. Forcing users to remember to cleanup your library is error-prone.
2. Just because library Foo is done does not mean library Bar is done and Bar
will lose when Foo calls close.
That said, I think that with some relatively minor changes, we can get the
effect that you want (minimal garbage after the usage of the FileSystem is
done):
1. Fix HADOOP-1160 so that the FileSystem.closeAll actually closes the client.
2. Have FileSystem.closeAll clear the CACHE map via reset.
3. Have the DFSClient.close method remove the client from the ClientFinalizer.
4. Have the constructor add "this" to the ClientFinalizer.
So after the closeAll call, you'll still have the DFSClient class loaded, but
not much else. In particular, you'll have no instances of DFSClient left.
> Remove unused ClientFinalizer in DFSClient
> ------------------------------------------
>
> Key: HADOOP-1130
> URL: https://issues.apache.org/jira/browse/HADOOP-1130
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.10.1, 0.11.0, 0.11.1, 0.11.2, 0.12.0
> Reporter: Philippe Gassmann
> Priority: Critical
> Attachments: HADOOP-1130.diff
>
>
> The ClientFinalizer shutdown hook is not used.
> This can lead to severe memory leaks if you use the DFSClient in a dynamic
> class loading context (such as in a webapp) as the DFSClient.ClientFinalizer
> class is retained in the memory by the shutdown hook. It retains also the
> current ClassLoader and thus all classes loaded by the ClassLoader. (Threads
> put in the shutdown hook are never garbage collected).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
