[
https://issues.apache.org/jira/browse/HADOOP-7973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13185769#comment-13185769
]
Daryn Sharp commented on HADOOP-7973:
-------------------------------------
I don't think it's related to use_ip, but I'll double check. Even if it is
related, it's still a bug. I think using a finalizer avoids the problem you
describe where something gets a fs, passes it to others, then closes it. Or
will a finalizer introduce other problems?
bq. App is expected to write the correct code - that is if the file system is
closed, do not use it post that. However I am not sure if apps do handle this
correctly.
Let's say I'm using DFS. I use some other package that opens the same default
DFS, does something, and then closes it. Whatever I was doing before I called
the external routine is now invalidated. What if I was writing to an output
stream? How would apps be able to reasonably recover from their fs being
unexpectedly closed when there's not really an error? Or am I misunderstanding
your intent?
bq. Other alternative solution is to disable cache
Isn't this going to cause multiple sockets to the same namenode? For code that
doesn't explicitly call close (perhaps because of this bug), that results in
leaked fds that tie up resources. I vaguely seem to recall that disabling the
cache for a job would cause the JT(?) to overwhelm the namenode with
connections.
> DistributedFileSystem close has severe consequences
> ---------------------------------------------------
>
> Key: HADOOP-7973
> URL: https://issues.apache.org/jira/browse/HADOOP-7973
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 1.0.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Priority: Blocker
> Attachments: HADOOP-7973.patch
>
>
> The way {{FileSystem#close}} works is very problematic. Since the
> {{FileSystems}} are cached, any {{close}} by any caller will cause problems
> for every other reference to it. Will add more detail in the comments.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
