[
https://issues.apache.org/jira/browse/HDFS-3359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269740#comment-13269740
]
Kihwal Lee commented on HDFS-3359:
----------------------------------
bq. If the client isn't explicitly closed, it appears the lease renewer holds a
reference to it so the client will never be closed until exit.
If there is nothing left to renew (i.e. all DFSOutputStreams are closed), the
renewer thread will terminate after a delay. So, if user close all output
streams, the renewer won't be the one holding the reference. DFSClient staying
open likely means DistributedFileSystem isn't closed and has a reference to the
client.
bq. Will this cause the client's leases to be renewed until exit? The renewer
will shutdown if all clients are closed, but due to the aforementioned, this
will never happen. Will this cause the process or a new thread to encounter
lease problems when re-opening the file?
The newer will shutdown if all DFSOutputStreams are closed, not necessarily
DFSClients. Even if the renewer is active due to one forgotten stream, it won't
affect others that are closed (NN has already removed the lease for those, so
won't get renewed). So, the problem you mentioned will be confined to the files
associated with the forgotten output streams.
> DFSClient.close should close cached sockets
> -------------------------------------------
>
> Key: HDFS-3359
> URL: https://issues.apache.org/jira/browse/HDFS-3359
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
> Affects Versions: 0.22.0, 2.0.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Critical
> Fix For: 0.23.3, 2.0.0
>
> Attachments: hdfs-3359-branch-0.23.txt, hdfs-3359.txt, hdfs-3359.txt
>
>
> Some applications like the TT/JT (pre-2.0) and probably the RM/NM cycle
> through DistributedFileSystem objects reasonably frequently. So long as they
> call close() it isn't a big problem, except that currently DFSClient.close()
> doesn't explicitly close the SocketCache. So unless a full GC runs (causing
> the references to get finalized), many SocketCaches can get orphaned, each
> with many open sockets inside. We should fix the close() function to close
> all cached sockets.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
