sodonnel opened a new pull request, #4341: URL: https://github.com/apache/ozone/pull/4341
## What changes were proposed in this pull request? The total commands pending for a datanode is the sum of the commands on the NodeManager CommandQueue and the number of commands the DN reported it has in the previous heartbeat. As things stand, these two piece of information come from two different methods, each with their own locking, the result is potentially inconsistent. To allow a consistent view of the commands queued on a data, this PR: 1. Adds a read write lock into the SCMNodeManager so it can lock around updates to the command queue, updating the DN queue count in heartbeat processing and querying the counts. 2. Moves the CommandQueueReportProcessing from being asynchronous to being processed as part of the heartbeat in SCM. This avoids a problem were the command queue has been emptied, but the pending count has not been updated inside DatanodeInfo. 3. In an earlier PR, a low priority flag was added to ReplicateContainer commands, so that the balancer can send commands with a lower priority. The DN does not report these low priority commands in its counts, so the command queue has been adjusted to not count them either. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-8074 ## How was this patch tested? Existing tests modified as needed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
