[
https://issues.apache.org/jira/browse/HADOOP-6240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754255#action_12754255
]
Konstantin Shvachko commented on HADOOP-6240:
---------------------------------------------
I just want to comment on atomicity of rename() and whether we should be posix
complaint with it. See also my earlier comment
[HDFS-303|https://issues.apache.org/jira/browse/HDFS-303?focusedCommentId=12750046&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12750046]
1. We should not provide atomic renames, mostly because it will extremely hard
to support if we ever build a distributed namespace service. Atomic rename is
relatively easy within a single name-node, but supporting it down the road for
renames across two or more name-servers is really hard.
2. We should not support posix definition for renames. Mostly because of [1]
above. But also because there are other (local) file systems which don't
support it either, which means that we cannot guarantee posix behavior for
hadoop {{LocalFileSystem}}. E.g., if you run {{LocalFileSystem}} on a local
WinNT drive then renaming of a file to an existing file and a directory to an
existing empty directory (posix 2 and 4) will fail, but if you run it on an
ext3 volume it will succeed.
To support the legal basis for these inconsistencies between different file
systems I cited Java definition of rename, which says:
"Many aspects of the behavior of this method are inherently platform-dependent:
The rename operation might not be able to move a file from one filesystem to
another, it might not be atomic, and it might not succeed if a file with the
destination abstract pathname already exists."
Imo, rename should always be treated as a convenience method. Posix semantics
for rename has been problematic for traditional file systems, for distributed
ones it should be avoided by all means.
For example, rename is the only operation in HDFS, which makes edits logs
non-idempotent, and we could have done so many things so much better with
idempotent journal records.
> Rename operation is not consistent between different implementations of
> FileSystem
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-6240
> URL: https://issues.apache.org/jira/browse/HADOOP-6240
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Fix For: 0.21.0
>
>
> The rename operation has many scenarios that are not consistently implemented
> across file systems.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
