[
https://issues.apache.org/jira/browse/HADOOP-901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12466161
]
Tom White commented on HADOOP-901:
----------------------------------
There are a couple of things that I would change:
1. There are a number of additional checks that need performing before doing
the rename. For example, checking if src exists.
2. It would be simpler and more efficient to use S3's ability to match all
paths with a given prefix, rather than using listPathsRaw recursively. This
entails adding a listDeepSubPaths method to FileSystemStore.
I'll create a patch to do this.
> Make S3FileSystem do recursive renames
> --------------------------------------
>
> Key: HADOOP-901
> URL: https://issues.apache.org/jira/browse/HADOOP-901
> Project: Hadoop
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.10.1
> Reporter: Tom White
>
> From Mike Smith:
> I went through the S3FileSystem.java codes and fixed the renameRaw() method.
> Now, it iterates through the folders recursively and rename those. Also, in
> the case of existing destination folder, it moves the src folder under the
> dst folder.
> Here is the piece code that should be replaced in S3FileSystem.java.
> renameRaw() method should be replaced by the following methods:
> @Override
> public boolean renameRaw(Path src, Path dst) throws IOException {
> Path absoluteDst = makeAbsolute(dst);
> Path absoluteSrc = makeAbsolute(src);
> INode inode = store.getINode(absoluteDst);
> // checking to see of dst folder exist. In this case moves the
> // src folder under the existing path.
> if (inode != null && inode.isDirectory()) {
> Path newDst = new Path(absoluteDst.toString
> ()+"/"+absoluteSrc.getName());
> return renameRaw(src,newDst,src);
> } else {
> // if the dst folder does not exist, then the dst folder will be created.
> return renameRaw(src,dst,src);
> }
> }
> // recursively goes through all the subfolders and rename those.
> public boolean renameRaw(Path src, Path dst,Path orgSrc) throws
> IOException {
> Path absoluteSrc = makeAbsolute(src);
> Path newDst = new Path(src.toString().replaceFirst(orgSrc.toString(),
> dst.toString()));
> Path absoluteDst = makeAbsolute(newDst);
> LOG.info(absoluteSrc.toString());
> INode inode = store.getINode (absoluteSrc);
> if (inode == null) {
> return false;
> }
> if (inode.isFile()) {
> store.storeINode(makeAbsolute(absoluteDst), inode);
> } else {
> store.storeINode (makeAbsolute(absoluteDst), inode);
> Path[] contents = listPathsRaw(absoluteSrc);
> if (contents == null) {
> return false;
> }
> for (Path p : contents) {
> if (! renameRaw(p,dst,orgSrc)) {
> return false;
> }
> }
> }
> store.deleteINode(absoluteSrc);
> return true;
> }
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
