[
https://issues.apache.org/jira/browse/HADOOP-8490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13395977#comment-13395977
]
Robert Joseph Evans commented on HADOOP-8490:
---------------------------------------------
I think that it would be nice to actually fix the cache if at all possible. I
have heard that most multi-tenet long running systems that use HDFS have to
disable the cache and then write their own, i.e. Oozie and Hive. The
JobTracker is the only one I know of that does not do this. Most client code I
have seen really relies on the fact that getting a "new" FileSystem is cheap,
so completely removing the cache is not a feasible option. Look at the MR/Yarn
code. Just uploading aggregated log files to HDFS creates about 5 FileSystems
if the cache is disabled.
If our users have to disable and work around a "feature" that we cannot turn
off we should take that as a bad sign and try to provide a better solution for
them. Now if the "fix" makes the performance horrible, or there are other
problems we may need to rethink things, but I am +1 on trying to fix the cache.
> Add Configuration to FileSystem cache key
> -----------------------------------------
>
> Key: HADOOP-8490
> URL: https://issues.apache.org/jira/browse/HADOOP-8490
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.0, 0.24.0, 2.0.0-alpha
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
>
> The {{FileSystem#get(URI, Configuration}} does not take the given
> {{Configuration}} into consideration before returning an existing fs instance
> from the cache with a possibly different conf.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
