QI, Jiale created HDFS-17949:
--------------------------------
Summary: Add optional FedBalance content summary verification
Key: HDFS-17949
URL: https://issues.apache.org/jira/browse/HDFS-17949
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: QI, Jiale
FedBalance currently finishes after the final DistCp succeeds, but it does not
provide an optional built-in verification step to compare the source and
destination tree at a coarse content-summary level.
This issue proposes adding an optional FedBalance verification phase after the
final DistCp succeeds. The verification is disabled by default and can be
enabled through CLI or configuration. When enabled, it compares the source and
destination ContentSummary directory count, file count, and length after
excluding snapshot content.
This gives operators a lightweight built-in correctness check without changing
the default FedBalance behavior.
Proposed changes:
- Add an optional VERIFY stage to DistCpProcedure.
- Add a FedBalanceVerifier helper based on ContentSummary.
- Add a CLI option and configuration key to enable verification.
- Add tests for successful verification and mismatch detection.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
