devmadhuu commented on code in PR #4509:
URL: https://github.com/apache/ozone/pull/4509#discussion_r1185943517
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/api/ContainerEndpoint.java:
##########
@@ -481,4 +489,83 @@ private List<ContainerBlockMetadata> getBlocks(
return blockIds;
}
+ @GET
+ @Path("/mismatch")
+ public Response getContainerMisMatchInsights() {
+ List<ContainerDiscrepancyInfo> containerDiscrepancyInfoList =
+ new ArrayList<>();
+ try {
+ Map<Long, ContainerMetadata> omContainers =
+ reconContainerMetadataManager.getContainers(-1, -1);
+ List<Long> scmAllContainers = containerManager.getContainers().stream()
+ .map(containerInfo -> containerInfo.getContainerID()).collect(
+ Collectors.toList());
+ List<Long> scmNonDeletedContainers =
+ containerManager.getContainers().stream()
+ .filter(containerInfo -> !(containerInfo.getState() ==
+ HddsProtos.LifeCycleState.DELETED))
+ .map(containerInfo -> containerInfo.getContainerID()).collect(
+ Collectors.toList());
+
+ // Filter list of container Ids which are present in OM but not in SCM.
+ List<Map.Entry<Long, ContainerMetadata>> notSCMContainers =
+ omContainers.entrySet().stream().filter(containerMetadataEntry ->
+
!(scmAllContainers.contains(containerMetadataEntry.getKey())))
Review Comment:
> This still looks incorrect. If a container has got Deleted in SCM and
still present in OM, it will never get flagged as scmAllContainers has Deleted
SCM containers as well. so no discrepancy will ever get reported. you need to
have scmNonDeletedContainers here as well instead of scmAllContainers. What we
are trying to figure out is container is still referred in OM but in SCM that
container has got deleted. @devmadhuu @sumitagrawl
IMO, we want to identify mismatch of container referred in OM but present in
SCM, now why the container is in deleted state in SCM, that is different issue.
But any container which cannot be queried for metadata by OM to SCM is a data
loss situation. @sumitagrawl - I think your 1st point is what we discussed. Let
us know what discussion happened with @errose28
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
