[
https://issues.apache.org/jira/browse/HADOOP-8842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463185#comment-13463185
]
Julien Le Dem commented on HADOOP-8842:
---------------------------------------
this comes from the behavior of java.io.File.renameTo(File)
to see it:
{noformat}
System.out.println("move to empty dir");
File tmp = File.createTempFile("tmp_", "_dir");
tmp.delete();
tmp.mkdirs();
File a = new File(tmp, "a");
File c = new File(tmp, "b/c");
a.mkdirs();
new FileWriter(new File(a, "part")).close();
c.mkdirs();
System.out.println(a.renameTo(c)); // true
System.out.println(Arrays.toString(c.listFiles())); // b/c/part
System.out.println("move to non empty dir");
File a2 = new File(tmp, "a2");
File c2 = new File(tmp, "b2/c2");
a2.mkdirs();
new FileWriter(new File(a2, "part")).close();
c2.mkdirs();
new FileWriter(new File(c2, "dummy")).close();
System.out.println(a2.renameTo(c2)); // false
{noformat}
Which RawLocalFileSystem is using:
http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.20/src/core/org/apache/hadoop/fs/RawLocalFileSystem.java?view=markup
{noformat}
249 public boolean rename(Path src, Path dst) throws IOException {
250 if (pathToFile(src).renameTo(pathToFile(dst))) {
251 return true;
252 }
253 return FileUtil.copy(this, src, this, dst, true, getConf());
254 }
{noformat}
I would suggest something like:
{noformat}
public boolean rename(Path src, Path dst) throws IOException {
File srcFile = pathToFile(src);
File dstFile = pathToFile(dst);
if (dstFile.exists() && dstFile.isDirectory()) {
dstFile = new File(destFile, srcFile.getName());
}
if (srcFile.renameTo(dstFile)) {
return true;
}
return FileUtil.copy(this, src, this, new Path(dstFile.toURI()), true,
getConf());
}
{noformat}
Or possibly move this logic up in the mv command to pass the final path as dst
instead of just what was passed to the mv command
> local file system behavior of mv into an empty directory is inconsistent with
> HDFS
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-8842
> URL: https://issues.apache.org/jira/browse/HADOOP-8842
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.20.2
> Reporter: Julien Le Dem
>
> moving into an empty directory replaces the directory instead.
> See output of attached script to reproduce :
> repro.sh
> {noformat}
> rm -rf local_fs_bug
> mkdir local_fs_bug
> hdfs -rmr local_fs_bug
> hdfs -mkdir local_fs_bug
> echo ">>> HDFS: normal behavior"
> touch part-0000
> hdfs -mkdir local_fs_bug/a
> hdfs -copyFromLocal part-0000 local_fs_bug/a
> hdfs -mkdir local_fs_bug/b
> hdfs -mkdir local_fs_bug/b/c
> echo "content of a: 1 part"
> hdfs -ls local_fs_bug/a
> echo "content of b/c: empty"
> hdfs -ls local_fs_bug/b/c
> echo "mv a b/c"
> hdfs -mv local_fs_bug/a local_fs_bug/b/c
> echo "resulting content of b/c"
> hdfs -ls local_fs_bug/b/c
> echo "a is moved inside of c"
> echo
> echo ">>> local fs: bug"
> mkdir -p local_fs_bug/a
> touch local_fs_bug/a/part-0000
> mkdir -p local_fs_bug/b/c
> echo "content of a: 1 part"
> hdfs -fs local -ls local_fs_bug/a
> echo "content of b/c: empty"
> hdfs -fs local -ls local_fs_bug/b/c
> echo "mv a b/c"
> hdfs -fs local -mv local_fs_bug/a local_fs_bug/b/c
> echo "resulting content of b/c"
> hdfs -fs local -ls local_fs_bug/b/c
> echo "bug: a replaces c"
> echo
> echo ">>> but it works if the destination is not empty"
> mkdir local_fs_bug/a2
> touch local_fs_bug/a2/part-0000
> mkdir -p local_fs_bug/b2/c2
> touch local_fs_bug/b2/c2/dummy
> echo "content of a2: 1 part"
> hdfs -fs local -ls local_fs_bug/a2
> echo "content of b2/c2: 1 dummy file"
> hdfs -fs local -ls local_fs_bug/b2/c2
> echo "mv a2 b2/c2"
> hdfs -fs local -mv local_fs_bug/a2 local_fs_bug/b2/c2
> echo "resulting content of b/c"
> hdfs -fs local -ls local_fs_bug/b2/c2
> echo "a2 is moved inside of c2"
> {noformat}
> Output:
> {noformat}
> >>> HDFS: normal behavior
> content of a: 1 part
> Found 1 items
> -rw-r--r-- 3 julien g 0 2012-09-25 17:16
> /user/julien/local_fs_bug/a/part-0000
> content of b/c: empty
> mv a b/c
> resulting content of b/c
> Found 1 items
> drwxr-xr-x - julien g 0 2012-09-25 17:16
> /user/julien/local_fs_bug/b/c/a
> a is moved inside of c
> >>> local fs: bug
> content of a: 1 part
> 12/09/25 17:16:34 WARN fs.FileSystem: "local" is a deprecated filesystem
> name. Use "file:///" instead.
> Found 1 items
> -rw-r--r-- 1 julien g 0 2012-09-25 17:16
> /home/julien/local_fs_bug/a/part-0000
> content of b/c: empty
> 12/09/25 17:16:34 WARN fs.FileSystem: "local" is a deprecated filesystem
> name. Use "file:///" instead.
> mv a b/c
> 12/09/25 17:16:35 WARN fs.FileSystem: "local" is a deprecated filesystem
> name. Use "file:///" instead.
> resulting content of b/c
> 12/09/25 17:16:35 WARN fs.FileSystem: "local" is a deprecated filesystem
> name. Use "file:///" instead.
> Found 1 items
> -rw-r--r-- 1 julien g 0 2012-09-25 17:16
> /home/julien/local_fs_bug/b/c/part-0000
> bug: a replaces c
> >>> but it works if the destination is not empty
> content of a2: 1 part
> 12/09/25 17:16:36 WARN fs.FileSystem: "local" is a deprecated filesystem
> name. Use "file:///" instead.
> Found 1 items
> -rw-r--r-- 1 julien g 0 2012-09-25 17:16
> /home/julien/local_fs_bug/a2/part-0000
> content of b2/c2: 1 dummy file
> 12/09/25 17:16:37 WARN fs.FileSystem: "local" is a deprecated filesystem
> name. Use "file:///" instead.
> Found 1 items
> -rw-r--r-- 1 julien g 0 2012-09-25 17:16
> /home/julien/local_fs_bug/b2/c2/dummy
> mv a2 b2/c2
> 12/09/25 17:16:37 WARN fs.FileSystem: "local" is a deprecated filesystem
> name. Use "file:///" instead.
> resulting content of b/c
> 12/09/25 17:16:38 WARN fs.FileSystem: "local" is a deprecated filesystem
> name. Use "file:///" instead.
> Found 2 items
> drwxr-xr-x - julien g 4096 2012-09-25 17:16
> /home/julien/local_fs_bug/b2/c2/a2
> -rw-r--r-- 1 julien g 0 2012-09-25 17:16
> /home/julien/local_fs_bug/b2/c2/dummy
> a2 is moved inside of c2
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
