[
https://issues.apache.org/jira/browse/HDFS-16748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17598795#comment-17598795
]
ASF GitHub Bot commented on HDFS-16748:
---------------------------------------
ZanderXu commented on PR #4813:
URL: https://github.com/apache/hadoop/pull/4813#issuecomment-1233896312
@Hexiaoqiao Thanks for your review. This improvement is not used to help
Router to forward renewLease to only one or certain Namenode.
One DFSClient may be writing some files from different namespace but with
the same file id via RBF. The current `filesBeingWritten` is only use iNodeId
to identify the written file, it cannot cover the case that two written files
from different namespace with the same iNodeId.
For example:
- RBF contains two mount point, /ns0 -> ns0 and /ns1 -> ns1
- One DFSClient create one file /ns0/file0 via RBF, the RBF will forward
this create rpc to ns0, and suppose the file with iNodeId 1000
- DFSClient will put this outputStream into `filesBeingWritten` with iNodeId
1000
- Then this DFSClient create one new file /ns1/file1 via RBF, the RBF will
forward this create RPC to NS1, and the response of this RPC may with one 1000
iNodeId, because they are from different namespace
- Then this DFSClient will put the new outputStream into `filesBeingWritten`
with iNodeId 1000 again. It will overwrite the previous outputStream with path
/ns0/file0 and iNodeId 1000 from NS0
It may caused this DFSClient cannot renew the lease of /ns0/file0
> DFSClient should uniquely identify writing files by namespace id and iNodeId
> ----------------------------------------------------------------------------
>
> Key: HDFS-16748
> URL: https://issues.apache.org/jira/browse/HDFS-16748
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: ZanderXu
> Assignee: ZanderXu
> Priority: Critical
> Labels: pull-request-available
>
> DFSClient should diff the writing files with namespaceId and iNodeId, because
> the writing files may belongs to different namespace with the same iNodeId.
> And the related code as bellows:
> {code:java}
> public void putFileBeingWritten(final long inodeId,
> final DFSOutputStream out) {
> synchronized(filesBeingWritten) {
> filesBeingWritten.put(inodeId, out);
> // update the last lease renewal time only when there was no
> // writes. once there is one write stream open, the lease renewer
> // thread keeps it updated well with in anyone's expiration time.
> if (lastLeaseRenewal == 0) {
> updateLastLeaseRenewal();
> }
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
