Ritesh Shukla created HDDS-16012:
------------------------------------
Summary: SCM should flag closed-container replicas reporting equal
BCSID with mismatched data checksums
Key: HDDS-16012
URL: https://issues.apache.org/jira/browse/HDDS-16012
Project: Apache Ozone
Issue Type: Improvement
Components: SCM
Reporter: Ritesh Shukla
h3. Problem
Two replicas of a CLOSED container claiming the same BCSID assert the same
committed content, so their container data checksums must match once both have
reported trees. SCM already receives both {{sequenceId}} and {{dataChecksum}}
in container replica reports, but nothing consumes {{dataChecksum}} in the
replication path today. The BCSID-inflation bug class (HDDS-15542, HDDS-12986)
produces exactly this signature -- equal sequenceId, different content -- and
is invisible to datanode scanners because the inflated replica is internally
consistent; only a cross-replica comparison can detect it.
h3. Proposal
A cheap comparison in SCM (or Recon) over data already reported:
* For CLOSED containers where all replicas have reported non-empty data
checksums: if any two replicas report equal {{sequenceId}} but different
{{dataChecksum}}, emit a metric and a rate-limited WARN, and surface it in
{{ozone admin container info}}.
* No automated corrective action, consistent with the reconciliation design's
System Safety principle; operators can run {{ozone admin container reconcile}}.
h3. Constraints
* Only compare when both replicas have reported checksums; suppress during
known transient windows (tree not yet generated post-close/scan,
mid-reconciliation) and debounce.
* Persistent mismatch with equal BCSIDs after reconciliation indicates either
BCSID inflation (a bug) or unrecoverable divergence -- both operator-worthy.
* Equal checksums with unequal BCSIDs is NOT an anomaly (under-claiming is the
safe direction); do not flag it.
h3. References
HDDS-10239, HDDS-15542, HDDS-12986, and the BCSID heuristics umbrella
(HDDS-16011).
Generated-by: Claude Code (Claude Fable 5)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
