[
https://issues.apache.org/jira/browse/HADOOP-12614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15041875#comment-15041875
]
Chris Nauroth commented on HADOOP-12614:
----------------------------------------
bq. ...reflection games to check whether HDFS is in safe mode or not...
Hi [[email protected]]. For my curiosity, would you mind giving a link to the
Spark code that does this?
I'm wondering how useful this proposal would really be. It seems inherently
subject to a time of check/time of use race condition. A caller could write
this...
{code}
if (!fs.isOffline()) {
FSDataOutputStream os = fs.create(...);
}
{code}
...and it still wouldn't guarantee that the application never hits a
{{SafeModeException}}.
Maybe seeing that Spark code in context would help clarify it for me.
> Add a generic .isOffline() method to filesystems to probe availability
> ----------------------------------------------------------------------
>
> Key: HADOOP-12614
> URL: https://issues.apache.org/jira/browse/HADOOP-12614
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 2.7.1
> Reporter: Steve Loughran
> Priority: Minor
>
> Looking at some of the spark `HistoryServer` code, they do reflection games
> to check whether HDFS is in safe mode or not, games which vary with version
> and could be at risk of failing with the client/server split (fortunately,
> it's all client-side). Nor do the checks apply to other filesystems, which
> could have their own online/offline state.
> I propose adding the new methods {{FileSystem.isOffline()}},
> {{FileContext.isOffline()}}, to return true if an FS knows that it is
> offline. For HDFS: Safe mode. For other filesystems? Maybe network state,
> disk being r/W, etc. Their choice. The default would be false: an FS is not
> offline,
> obvously, {{!isOffline()}} doesn't guarantee the FS is fully functional;
> that's why I propose {{isOffline()}}; less dangerous than the opposite values
> of {{isLive()}} or {{isAvailable()}}, which may be making promises which
> cannot hold
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
