[
https://issues.apache.org/jira/browse/HDFS-12914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16845430#comment-16845430
]
Santosh Marella commented on HDFS-12914:
----------------------------------------
[~starphin] - thanks a lot for the feedback.
1) Good catch. I had to modify the pom.xml to compile locally, but missed out
to remove it before uploading the diff. Removed it now and uploaded a new
patch.
2) Tests are certainly great to have. I didn't find any existing tests related
to FBR leases to add/modify further.
3) {{BlockReportLeaseManager}} has good amount of logging at the DEBUG level to
track things currently - for e.g. when the lease is granted (or cannot be
granted), when the lease is removed etc. Would like to hear from others if
this is proving to be sufficient or we need some more logging.
> Block report leases cause missing blocks until next report
> ----------------------------------------------------------
>
> Key: HDFS-12914
> URL: https://issues.apache.org/jira/browse/HDFS-12914
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 2.8.0, 2.9.2
> Reporter: Daryn Sharp
> Assignee: Santosh Marella
> Priority: Critical
> Attachments: HDFS-12914-branch-2.001.patch,
> HDFS-12914-trunk.00.patch, HDFS-12914-trunk.01.patch
>
>
> {{BlockReportLeaseManager#checkLease}} will reject FBRs from DNs for
> conditions such as "unknown datanode", "not in pending set", "lease has
> expired", wrong lease id, etc. Lease rejection does not throw an exception.
> It returns false which bubbles up to {{NameNodeRpcServer#blockReport}} and
> interpreted as {{noStaleStorages}}.
> A re-registering node whose FBR is rejected from an invalid lease becomes
> active with _no blocks_. A replication storm ensues possibly causing DNs to
> temporarily go dead (HDFS-12645), leading to more FBR lease rejections on
> re-registration. The cluster will have many "missing blocks" until the DNs
> next FBR is sent and/or forced.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
