[
https://issues.apache.org/jira/browse/HDFS-5902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13895088#comment-13895088
]
Steve Loughran commented on HDFS-5902:
--------------------------------------
comments are meaningless if client code uses a method and assumes it is atomic
-and see that it is on one FS, so assumes it is everywhere. There's already the
issue that some code may assume that {{Filesystem.mkdir()}} is atomic all the
way up the tree. It is in HDFS, but only because we grab the lock on the
metadata -for performance-. Most native unix filesystems do this in user-space,
non-atomically
# HADOOP-9565 proposes a special interface {{Blobstore}} which could add some
pure blobstore operations (PUT), as well as tagging
# In HADOOP-9361 I'm adding declarative data stating what features an FS
supports, example
[[LocalFS|https://github.com/steveloughran/hadoop-trunk/blob/stevel/HADOOP-9361-filesystem-contract/hadoop-common-project/hadoop-common/src/test/resources/contract/localfs.xml].
This for testing (and hence in the test JAR), but it shows how it is useful
for some things to care about features. And the way to declare feature support
in Java code is to implement interfaces.
> Atomic Directory Swapping Operation
> -----------------------------------
>
> Key: HDFS-5902
> URL: https://issues.apache.org/jira/browse/HDFS-5902
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: namenode
> Affects Versions: 2.2.0
> Reporter: Benoit Perroud
> Priority: Minor
>
> With snapshots, we can already start to have a consistent view of the data
> over time.
> One missing primitive to do one more step in the direction of a transactional
> FS is the ability to swap atomically two folders:
> {code}
> swapFolders(src, dst)
> {code}
> As of today, we need to:
> 1. move the destination folder to a temporary name,
> 2. rename the source folder in destination's name
> 3. rename the dst folder with the temporary name in the source name.
> This can leads to race conditions where the data is not found.
> An operation inside the namenode achieving folders swapping, with the
> appropriate checks and locking to ensure atomicity, would be really useful to
> perform maintenance tasks like folder compation, format conversion, etc...
> (and lot of other use cases).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
