[
https://issues.apache.org/jira/browse/HDFS-7923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14565672#comment-14565672
]
Andrew Wang commented on HDFS-7923:
-----------------------------------
Nits:
* Should the checkLease logs be done to the blockLog? We log the startup error
log there in processReport
* Update javadoc in BlockReportContext with what leaseID is for.
* Add something to the log message about overwriting the old leaseID in
offerService. Agree that this shouldn't really trigger, but good defensive
coding practice :)
* DatanodeManager, there's still a register/unregister in registerDatanode I
think we could skip. This is the node restart case where it's registered
previously.
* BRLManager requestLease, we auto-register the node on requestLease. This
shouldn't happen since DNs need to register before doing anything else. We can
keep this here
* Still need documentation of new config keys in hdfs-default.xml
Block report scheduling:
* We removed TestBPSAScheduler#testScheduleBlockReportImmediate, should this
swap over to testing forceFullBlockReport?
* Extra import in TestBPSAScheduler and BPSA
* I'm worried about convoy effects if we don't stick to the stride system of
the old code. I think of the old code as follows:
# Choose a random time within the "initialDelay" interval to jitter
# Attempt to block report at that same time every hour.
This keeps the BRs from all the DNs spread out, even if the NN gets temporarily
backed up. Once the NN catches up and flushes its backlog of FBRs, future BRs
will still be nicely spread out.
My understanding of your new scheme is that after a DN successfully BRs, it'll
BR again an hour afterwards. So, if all the BRs piled up and then are processed
in quick succession, all the DNs will BR at about the same time next hour.
Since we want to spread the BRs out across the hour, this is not good.
Other ideas are to round up to the next stride. Or, wait an interval plus a
random delay. We might consider some congestion control too, where the DNs
backoff linearly or exponentially. All these schemes delay the FBRs, but maybe
we trust IBRs enough now.
If you want to pursue this logic change more, let's split it out into a
follow-on JIRA. The rest LGTM, +1 pending above comments.
> The DataNodes should rate-limit their full block reports by asking the NN on
> heartbeat messages
> -----------------------------------------------------------------------------------------------
>
> Key: HDFS-7923
> URL: https://issues.apache.org/jira/browse/HDFS-7923
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: 2.8.0
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Attachments: HDFS-7923.000.patch, HDFS-7923.001.patch,
> HDFS-7923.002.patch, HDFS-7923.003.patch, HDFS-7923.004.patch
>
>
> The DataNodes should rate-limit their full block reports. They can do this
> by first sending a heartbeat message to the NN with an optional boolean set
> which requests permission to send a full block report. If the NN responds
> with another optional boolean set, the DN will send an FBR... if not, it will
> wait until later. This can be done compatibly with optional fields.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
