Stephen O'Donnell created HDDS-6384:
---------------------------------------
Summary: EC: Ensure EC container usage is updated correctly when
handling reports
Key: HDDS-6384
URL: https://issues.apache.org/jira/browse/HDDS-6384
Project: Apache Ozone
Issue Type: Sub-task
Components: SCM
Reporter: Stephen O'Donnell
The current container report handling logic in AbstractContainerReportHandler
does update the container usage and key count correctly in SCM for EC
containers. The current logic says:
{code:java}
for (ContainerReplica r : otherReplicas) {
// Open containers are generally growing in key count and size, the
// overall size should be the min of all reported replicas.
if (containerInfo.getState().equals(HddsProtos.LifeCycleState.OPEN)) {
usedBytes = Math.min(usedBytes, r.getBytesUsed());
keyCount = Math.min(keyCount, r.getKeyCount());
} else {
// Containers which are not open can only shrink in size, so use the
// largest values reported.
usedBytes = Math.max(usedBytes, r.getBytesUsed());
keyCount = Math.max(keyCount, r.getKeyCount());
}
} {code}
For EC Containers, the bytes used will be different across all the replicas due
to partial stripes. The first data replica and the parity replicas should have
the max size, but other data replicas are likely to be smaller. Therefore using
the min as above will not work.
We should either use the max value for open containers, or the min of the first
data and parity group, and ignore the usage in the other replicas.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
