[
https://issues.apache.org/jira/browse/HDFS-7923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14575058#comment-14575058
]
Colin Patrick McCabe commented on HDFS-7923:
--------------------------------------------
bq. Should the checkLease logs be done to the blockLog? We log the startup
error log there in processReport
I like having the ability to turn on and off TRACE logging for various
subsystems. Putting everything in the blockLog would make that harder, right?
bq. Update javadoc in BlockReportContext with what leaseID is for.
added
bq. Add something to the log message about overwriting the old leaseID in
offerService. Agree that this shouldn't really trigger, but good defensive
coding practice
ok
bq. DatanodeManager, there's still a register/unregister in registerDatanode I
think we could skip. This is the node restart case where it's registered
previously.
Good catch. The calls in {{removeDatanode}} and {{addDatanode}} should take
care of this, so there's no need to have it here.
bq. BRLManager requestLease, we auto-register the node on requestLease. This
shouldn't happen since DNs need to register before doing anything else. We can
keep this here
I added a warn message since this shouldn't happen.
bq. Still need documentation of new config keys in hdfs-default.xml
added
bq. Extra import in TestBPSAScheduler and BPSA
removed
bq. If you want to pursue \[the block report timing\] logic change more, let's
split it out into a follow-on JIRA. The rest LGTM, +1 pending above comments.
OK, I will restore the old behavior for now, and we can do this in a follow-on
change.
> The DataNodes should rate-limit their full block reports by asking the NN on
> heartbeat messages
> -----------------------------------------------------------------------------------------------
>
> Key: HDFS-7923
> URL: https://issues.apache.org/jira/browse/HDFS-7923
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: 2.8.0
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Attachments: HDFS-7923.000.patch, HDFS-7923.001.patch,
> HDFS-7923.002.patch, HDFS-7923.003.patch, HDFS-7923.004.patch
>
>
> The DataNodes should rate-limit their full block reports. They can do this
> by first sending a heartbeat message to the NN with an optional boolean set
> which requests permission to send a full block report. If the NN responds
> with another optional boolean set, the DN will send an FBR... if not, it will
> wait until later. This can be done compatibly with optional fields.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
