umamaheswararao commented on a change in pull request #3147:
URL: https://github.com/apache/ozone/pull/3147#discussion_r817962237
##########
File path:
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestContainerReportHandler.java
##########
@@ -631,6 +635,158 @@ public void
notOpenContainerKeyAndBytesUsedUpdatedToMaximumOfAllReplicas()
.getNumberOfKeys());
}
+ @Test
+ public void openECContainerKeyAndBytesUsedUpdatedToMinimumOfAllReplicas()
+ throws IOException {
+ final ECReplicationConfig repConfig = new ECReplicationConfig(3, 2);
+ final ContainerReportHandler reportHandler = new ContainerReportHandler(
+ nodeManager, containerManager);
+
+ Pipeline pipeline = pipelineManager.createPipeline(repConfig);
+ Map<Integer, DatanodeDetails> dns = new HashMap<>();
+ final Iterator<DatanodeDetails> nodeIterator = nodeManager.getNodes(
+ NodeStatus.inServiceHealthy()).iterator();
+ for (int i = 1; i <= 5; i++) {
Review comment:
Nit: Probably you may want to use repConfig.getRequiredNodes instead of
hardcoded 5?
##########
File path:
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/AbstractContainerReportHandler.java
##########
@@ -121,30 +122,77 @@ private void updateContainerStats(final DatanodeDetails
datanodeDetails,
containerInfo.updateSequenceId(
replicaProto.getBlockCommitSequenceId());
}
+ if (containerInfo.getReplicationConfig().getReplicationType()
+ == HddsProtos.ReplicationType.EC) {
+ updateECContainerStats(containerInfo, replicaProto, datanodeDetails);
+ } else {
+ updateRatisContainerStats(containerInfo, replicaProto,
datanodeDetails);
+ }
+ }
+ }
+
+ private void updateRatisContainerStats(ContainerInfo containerInfo,
+ ContainerReplicaProto newReplica, DatanodeDetails newSource)
+ throws ContainerNotFoundException {
+ List<ContainerReplica> otherReplicas =
+ getOtherReplicas(containerInfo.containerID(), newSource);
+ long usedBytes = newReplica.getUsed();
+ long keyCount = newReplica.getKeyCount();
+ for (ContainerReplica r : otherReplicas) {
+ usedBytes = calculateUsage(containerInfo, usedBytes, r.getBytesUsed());
+ keyCount = calculateUsage(containerInfo, keyCount, r.getKeyCount());
+ }
+ updateContainerUsedAndKeys(containerInfo, usedBytes, keyCount);
+ }
+
+ private void updateECContainerStats(ContainerInfo containerInfo,
+ ContainerReplicaProto newReplica, DatanodeDetails newSource)
+ throws ContainerNotFoundException {
+ int dataNum =
+ ((ECReplicationConfig)containerInfo.getReplicationConfig()).getData();
+ // The first EC index and the parity indexes must all be the same size
+ // while the other data indexes may be smaller due to partial stripes.
+ // When calculating the stats, we only use the first data and parity and
+ // ignore the others. We only need to run the check if we are processing
Review comment:
I am trying to understand. This usedBytes mainly used for checking
whether we can use this container for new blocks or not right?
So, we are trying to keep the Max size bytes used in EC block group blocks.
Is this understanding correct?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
