[
https://issues.apache.org/jira/browse/HADOOP-6240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754283#action_12754283
]
Doug Cutting commented on HADOOP-6240:
--------------------------------------
> We should not provide atomic renames [ ... ]
A middle ground might be to be very clear about where they can be relied on and
where they cannot, so that applications are forewarned. They might be only
supported, e.g., within a single volume of unix local filesystems, or within a
single hdfs filesystem.
We might even add an atomicRename method that fails in other cases. Then
applications that require atomic renames would use that method instead.
However, if mapreduce job output promotion is required to be atomic, and S3
does not implement atomic rename, then one could not, e.g., use S3 as a
mapreduce output directory.
> LocalFileSystem can do necessary things (such as deleting an existing dst)
> before calling java File.renameTo() to make is consistent across different
> operating systems.
But that's not atomic, right? If we're willing to tolerate that, then we might
implement a non-atomic rename directly in FileContext, in terms of the existing
FileSystem#rename(), and skip adding AbstractFileSystem for now. Things like
job promotion, etc. will not be switched to use FileContext in 0.21, so, if
atomicity were optional, making FileContext#rename() atomic might wait until
0.22. But, from what I've heard thus far, that's not an acceptable option.
> Rename operation is not consistent between different implementations of
> FileSystem
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-6240
> URL: https://issues.apache.org/jira/browse/HADOOP-6240
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Fix For: 0.21.0
>
>
> The rename operation has many scenarios that are not consistently implemented
> across file systems.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
