[
https://issues.apache.org/jira/browse/HDFS-4248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507853#comment-13507853
]
Tsz Wo (Nicholas), SZE commented on HDFS-4248:
----------------------------------------------
Hi Daryn,
I think the bug of dangling lease may not exist after tried some tests,
although there is a bug that a lease is removed but not renamed. In my tests,
a file is created in src directory but not close, then rename and then check
lease. We have 6 cases: whether the old/new rename is used, whether dst
exists, and whether overwrite is set for the new API.
- Case 1) src=/rename_new/src, dst=/rename_new/dst_exist, overwrite=true
Before {/rename_new/src/file=[Lease. Holder:
DFSClient_NONMAPREDUCE_790025804_1, pendingcreates: 1]}
After {}
{noformat}
drwxr-xr-x - szetszwo supergroup 0 2012-11-30 19:22
/rename_new/dst_exist
-rw-r--r-- 3 szetszwo supergroup 0 2012-11-30 19:22
/rename_new/dst_exist/file
{noformat}
FAILED sortedLeases.size() = 0 != 1
- Case 2) src=/rename_new/src, dst=/rename_new/dst_exist, overwrite=false
Before {/rename_new/src/file=[Lease. Holder:
DFSClient_NONMAPREDUCE_-1062494203_1, pendingcreates: 1]}
2012-11-30 19:22:48,393 INFO namenode.TestFSDirectory
(TestFSDirectory.java:runRename(220)) - Case 2) src=/rename_new/src,
dst=/rename_new/dst_exist, overwrite=false
{noformat}
org.apache.hadoop.fs.FileAlreadyExistsException: rename destination
/rename_new/dst_exist already exists
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.unprotectedRenameTo(FSDirectory.java:676)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.renameTo(FSDirectory.java:469)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.renameToInternal(FSNamesystem.java:2646)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.renameTo(FSNamesystem.java:2613)
...
{noformat}
After {/rename_new/src/file=[Lease. Holder:
DFSClient_NONMAPREDUCE_-1062494203_1, pendingcreates: 1]}
{noformat}
drwxr-xr-x - szetszwo supergroup 0 2012-11-30 19:22
/rename_new/dst_exist
drwxr-xr-x - szetszwo supergroup 0 2012-11-30 19:22 /rename_new/src
-rw-r--r-- 3 szetszwo supergroup 0 2012-11-30 19:22
/rename_new/src/file
{noformat}
SUCCESS
- Case 3) src=/rename_new/src, dst=/rename_new/dst_nonexist, overwrite=true
Before {/rename_new/src/file=[Lease. Holder:
DFSClient_NONMAPREDUCE_-1012508552_1, pendingcreates: 1]}
After {/rename_new/dst_nonexist/file=[Lease. Holder:
DFSClient_NONMAPREDUCE_-1012508552_1, pendingcreates: 1]}
{noformat}
drwxr-xr-x - szetszwo supergroup 0 2012-11-30 19:22
/rename_new/dst_nonexist
-rw-r--r-- 3 szetszwo supergroup 0 2012-11-30 19:22
/rename_new/dst_nonexist/file
{noformat}
SUCCESS
- Case 4) src=/rename_new/src, dst=/rename_new/dst_nonexist, overwrite=false
Before {/rename_new/src/file=[Lease. Holder:
DFSClient_NONMAPREDUCE_-1530262891_1, pendingcreates: 1]}
After {/rename_new/dst_nonexist/file=[Lease. Holder:
DFSClient_NONMAPREDUCE_-1530262891_1, pendingcreates: 1]}
{noformat}
drwxr-xr-x - szetszwo supergroup 0 2012-11-30 19:22
/rename_new/dst_nonexist
-rw-r--r-- 3 szetszwo supergroup 0 2012-11-30 19:22
/rename_new/dst_nonexist/file
{noformat}
SUCCESS
- Case 5) src=/rename_depreacted/src, dst=/rename_depreacted/dst_exist,
overwrite=true
Before {/rename_depreacted/src/file=[Lease. Holder:
DFSClient_NONMAPREDUCE_925124308_1, pendingcreates: 1]}
After {/rename_depreacted/dst_exist/src/file=[Lease. Holder:
DFSClient_NONMAPREDUCE_925124308_1, pendingcreates: 1]}
{noformat}
drwxr-xr-x - szetszwo supergroup 0 2012-11-30 19:22
/rename_depreacted/dst_exist
drwxr-xr-x - szetszwo supergroup 0 2012-11-30 19:22
/rename_depreacted/dst_exist/src
-rw-r--r-- 3 szetszwo supergroup 0 2012-11-30 19:22
/rename_depreacted/dst_exist/src/file
{noformat}
SUCCESS
- Case 6) src=/rename_depreacted/src, dst=/rename_depreacted/dst_nonexist,
overwrite=true
Before {/rename_depreacted/src/file=[Lease. Holder:
DFSClient_NONMAPREDUCE_-1211709026_1, pendingcreates: 1]}
After {/rename_depreacted/dst_nonexist/file=[Lease. Holder:
DFSClient_NONMAPREDUCE_-1211709026_1, pendingcreates: 1]}
{noformat}
drwxr-xr-x - szetszwo supergroup 0 2012-11-30 19:22
/rename_depreacted/dst_nonexist
-rw-r--r-- 3 szetszwo supergroup 0 2012-11-30 19:22
/rename_depreacted/dst_nonexist/file
{noformat}
SUCCESS
Only Case 1 fails but it does not generate a dangling lease.
> Renames may cause dangling leases
> ---------------------------------
>
> Key: HDFS-4248
> URL: https://issues.apache.org/jira/browse/HDFS-4248
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Priority: Blocker
> Attachments: HDFS-4248.branch-0.23.patch, HDFS-4248.patch
>
>
> Renames of directories may incorrectly re-write the paths in leases under the
> tree being moved.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
