[
https://issues.apache.org/jira/browse/HDFS-4258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13583638#comment-13583638
]
Aaron T. Myers commented on HDFS-4258:
--------------------------------------
bq. I have hard time understanding the relationship of that with rename in this
jira. In fact when recoverLease is done an active writer will not be able to
write whether rename has occurred or not. That behavior is not being changed
with the changes done as a part of this.
I wasn't implying that this proposed change has anything to do with the
recoverLease API. I was just using it as a counterpoint to Brandon's claim that
clients would be overpowered since "for any client wants to gain a lease of any
file, it could just rename it and then open it. Most likely it could get the
lease immediately as long as it has write permission." i.e. that should not be
a concern, since a client can already do that today with the recoverLease API.
{quote}
There are other ideas that have been proposed about revoking the lease on
rename. I am -1 on it for the following reasons:
# Current behavior is when a rename occurs the current writer continues to
write to the block that is currently allocated but fails to allocate new blocks.
# New rename behavior will be incompatible where the current writer is fenced
from writing. Given that a lot of files in HDFS are less than a block in
length, this could result in strange behaviors for some applications.
# I agree with the point Brandon raised above. Renaming a directory means
walking through all the files open under it and revoking the leases. Rename
already is a complicated operation. Doing this additional work during rename
makes it even more heavier and the operation unpredictably large.
I actually like the direction Brandon and Nicholas have taken. We can continue
the existing behavior. In fact with this change, we can allow current writer to
continue to allocated new blocks (based on file ID) and continue to write, if
we want. But that could be done in another jira.
{quote}
I agree with Daryn's reasoning as stated in HDFS-4437:
{quote}
I think supporting file descriptor behavior is a great idea (we've internally
talked about this). Until we do, I think the lease should be revoked. My
concerns with fd behavior would be the ever pervasive "two wrongs make a right"
where users are relying unintentionally on renames breaking writers, and
ensuring we get the security right to avoid attacks probing for fileids.
{quote}
Regardless of whether or not we implement HDFS-4437 as Daryn proposed, I still
think we should move the INode ID stuff to a branch. It's a fairly involved
change with several sub-task JIRAs, which indicates to me that it would be
better done incrementally on a branch and then merged to trunk once it's a
completely functional whole.
> Rename of Being Written Files
> -----------------------------
>
> Key: HDFS-4258
> URL: https://issues.apache.org/jira/browse/HDFS-4258
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client, namenode
> Affects Versions: 3.0.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Brandon Li
> Attachments: HDFS-4258.patch, HDFS-4258.patch, HDFS-4258.patch,
> HDFS-4258.patch
>
>
> When a being written file or it's ancestor directories is renamed, the path
> in the file lease is also renamed. Then the writer of the file usually will
> fail since the file path in the writer is not updated.
> Moreover, I think there is a bug as follow:
> # Client writes 0's to F_0="/foo/file" and writes 1's to F_1="/bar/file" at
> the same time.
> # Rename /bar to /baz
> # Rename /foo to /bar
> Then, writing to F_0 will fail since /foo/file does not exist anymore but
> writing to F_1 may succeed since /bar/file exits as a different file. In
> such case, the content of /bar/file could be partly 0's and partly 1's.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
