[
https://issues.apache.org/jira/browse/HADOOP-15691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596377#comment-16596377
]
Steve Loughran commented on HADOOP-15691:
-----------------------------------------
BTW, I've realised that I could expose this not just through a new API, but
simply as a set of config options which you can query through
FileSystem.getConf().getBoolean("fs.capabilities.fs.atomic-rename"). FS
instances would add their list of capabilities in init(), so existing code can
see if a feature is there without needing to call the API.
This'd make it easier for apps like HBase, Tez & Spark to be able to probe for
specific features, e.g. whether they could skip the use of a temp file +
rename. It would complicate subclassed filesystems which changed the
capabilites though, especially those which removed them, and I'd also need to
add some property declaring that the capabilities had been registered, e.g.
"fs.capabilities.registered"
> Add PathCapabilities to FS and FC to complement StreamCapabilities
> ------------------------------------------------------------------
>
> Key: HADOOP-15691
> URL: https://issues.apache.org/jira/browse/HADOOP-15691
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Attachments: HADOOP-15691-001.patch, HADOOP-15691-002.patch
>
>
> Add a {{PathCapabilities}} interface to both FileSystem and FileContext to
> declare the capabilities under the path of an FS
> This is needed for
> * HADOOP-15407: declare that a dest FS supports permissions
> * object stores to declare that they offer PUT-in-place alongside
> (slow-rename)
> * Anything else where the implementation semantics of an FS is so different
> caller apps would benefit from probing for the underlying semantics
> I know, we want all filesystem to work *exactly* the same. But it doesn't
> hold, especially for object stores —and to efficiently use them, callers need
> to be able to ask for specific features.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
