Paulo Motta created CASSANDRA-21485:
---------------------------------------
Summary: AutoRepair metric BytesAlreadyRepaired can exceed
TotalBytesToRepair due to memtable data not included in total estimation
Key: CASSANDRA-21485
URL: https://issues.apache.org/jira/browse/CASSANDRA-21485
Project: Apache Cassandra
Issue Type: Task
Reporter: Paulo Motta
h2. Summary
The {{TotalBytesToRepair}} metric does not account for memtable data, causing
{{BytesAlreadyRepaired}} to exceed {{TotalBytesToRepair}} (progress > 100%)
after a repair turn completes.
h2. Root Cause
{{TotalBytesToRepair}} is computed in
{{AutoRepairState.updateRepairScheduleStatistics()}} by summing
{{KeyspaceRepairPlan.getEstimatedBytes()}}, which only considers on-disk
SSTable sizes via {{SizeEstimate.sizeForRepair}}.
However, during repair assignment generation,
{{RepairTokenRangeSplitter.getRepairAssignments()}} (line 522-527) discovers
tables with no SSTables but active memtable data and creates assignments with
{{estimatedBytes = memtableSize}}. These bytes are then accumulated in
{{bytesAlreadyRepaired}} but were never counted in the total.
h2. Code References
* {{AutoRepairState.java:117-119}} - computes {{totalBytesToRepair}} from plan
(SSTable-only)
* {{RepairTokenRangeSplitter.java:522-527}} - discovers memtable data and
assigns {{memtableSize}} as estimated bytes
* {{AutoRepair.java:401}} - accumulates {{bytesAlreadyRepaired}} including
memtable-based assignments
* {{AutoRepairUtils.java:1391}} - {{sizeForRepair}} is derived from SSTable
sizes only
h2. Impact
* The {{BytesAlreadyRepaired}} and {{TotalBytesToRepair}} JMX metrics report
inconsistent values (repaired > total)
* Progress monitoring dashboards show values above 100% which is confusing for
operators
* The repair itself is not affected - all data (including memtable) is
correctly repaired
h2. Suggested Fix
Include memtable sizes in the total estimation. When computing
{{totalBytesToRepair}}, also check for live memtable data on tables that have
{{sizeForRepair == 0}}, mirroring the logic in the splitter. This could be done
in {{calcTotalBytesToBeRepaired()}} or {{updateRepairScheduleStatistics()}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
