errose28 opened a new pull request, #7076: URL: https://github.com/apache/ozone/pull/7076
## What changes were proposed in this pull request? ### Summary In the placeholder reconciliation implementation, handling of reconcile container commands from SCM to datanodes are done in the `KeyValueHandler`. In the final implementation they should go through the `ReplicationSupervisor` like replication and reconstruction commands for proper scheduling. This change also makes sure the same `ContainerChecksumTreeManager` instance is available in all the places it is needed within the datanode, and closes its metrics properly instead of the workaround added in [HDDS-10373](https://issues.apache.org/jira/browse/HDDS-10373). ### Specific Changes - Usage of `ContainerChecksumTreeManager` is unified to one instance for the whole datanode. - Merkle tree metrics object is properly closed. - Reconcile commands are deduplicated by container ID and peer list. - The `ReplicationSupervisor` should drop duplicate commands. - `ReconcileContainerCommandHandler` uses `ReplicationSupervisor` for queueing. It does not implement it's own thread pool and queue. - With this change, it also uses `ReplicationSupervisor` for metrics. - This is similar to how `ReconstructECContainersCommandHandler` and `ReplicateContainerCommandHandler` are implemented (total time and invocation count are unused in that class) - It would be good to have more fine grained metrics for each of these task types instead of the shared `ReplicationSupervisorMetrics`, but that would be a broader change outside the scope of this PR. - New test helper methods for creating `KeyValueHandler` and `HddsDispatcher` instances when only a susbet of the functionality is required were added and existing tests were updated. - This greatly reduced test dependence on the `KeyValueHandler` constructor allowing us to make changes here to get the `ContainerChecksumTreeManager` where we need it. ## What is the link to the Apache JIRA HDDS-11254 ## How was this patch tested? - Metrics unregistration has been seen to fail existing tests if not done properly. No failures observed here. - `TestReconcileCommandHandler` tests for metrics were removed since those are now covered by `TestReplicationSupervisor`. - Unit test added for `TestReconcileContainerTask` that it properly detects task failure and equality. - Existing e2e acceptance tests for reconciliation pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
