Timothy Potter created SOLR-11335:
-------------------------------------
Summary: HdfsDirectory & Factory should not close the FileSystem
object retrieved with get
Key: SOLR-11335
URL: https://issues.apache.org/jira/browse/SOLR-11335
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Components: Hadoop Integration
Reporter: Timothy Potter
I'm seeing issues where the Hadoop FileSystem instance is closed out from under
other objects. From what I understand, the Hadoop FileSystem object
(org.apache.hadoop.fs.FileSystem) retrieved with {{FileSystem.get}} as is done
in HdfsDirectory's ctor is a shared object that if closed, can affect other
code using that same shared instance. You can see this is a cached, shared
object here ->
https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java#L455
Thus, I suspect Solr should not be closing any FileSystem instance retrieved
with get. It's important to mention that if I set the
{{fs.$SCHEME.impl.disable.cache}} to true, then my problems go away, which
seems to confirm that Solr is using the API incorrectly. That being said, I'm
surprised this hasn't been raised before, so maybe I've missed something basic
in Solr's use of HDFS?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
