sadanand48 commented on code in PR #11962:
URL: https://github.com/apache/ozone/pull/11962#discussion_r4509924491
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/states/endpoint/HeartbeatEndpointTask.java:
##########
@@ -148,25 +170,34 @@ public EndpointStateMachine.EndPointStates call() throws
Exception {
requestBuilder = SCMHeartbeatRequestProto.newBuilder()
.setDatanodeDetails(datanodeDetailsProto)
.setDatanodeVersion(versionInfo);
- addReports(requestBuilder);
+ int reportCount = addNonCommandStatusReports(requestBuilder);
addContainerActions(requestBuilder);
addPipelineActions(requestBuilder);
addQueuedCommandCounts(requestBuilder);
+ commandStatusSelection = addCommandStatusReports(
+ requestBuilder, maxReportsPerHB - reportCount);
SCMHeartbeatRequestProto request = requestBuilder.build();
LOG.debug("Sending heartbeat message : {}", request);
SCMHeartbeatResponseProto response = rpcEndpoint.getEndPoint()
.sendHeartbeat(request);
processResponse(response, datanodeDetailsProto);
rpcEndpoint.setLastSuccessfulHeartbeat(ZonedDateTime.now());
rpcEndpoint.zeroMissedCount();
- if (context.hasPendingReports(rpcEndpoint.getAddress())) {
- // addReports was capped at maxReportsPerHB and left reports queued.
Trigger immediately.
- context.getParent().setNextHB(Time.monotonicNow());
+ if (context.getQueueMetrics() != null) {
Review Comment:
Thanks, this behavior change is intentional. The byte limit bounds each
heartbeat payload, but immediate follow-ups could still send repeated 64 MB
heartbeats in a tight loop while draining a large backlog, potentially
recreating RPC and heap pressure on a recovering SCM. Waiting for the normal
heartbeat interval bounds both payload size and delivery rate.
The default count limit is 10,000 top-level reports per datanode per
heartbeat, while an FCR counts as only one report, so the expected backlog
recovery rate should remain reasonable.
I would prefer to keep this change in the same JIRA because pacing backlog
recovery is part of the mitigation. I will update the stale ozone-default.xml
description
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]