errose28 commented on code in PR #7976:
URL: https://github.com/apache/ozone/pull/7976#discussion_r1974488184
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/AbstractContainerReportHandler.java:
##########
@@ -352,26 +352,32 @@ private boolean updateContainerState(final
DatanodeDetails datanode,
*/
break;
case DELETING:
+ case DELETED:
/*
- HDDS-11136: If a DELETING container has a non-empty CLOSED replica, the
container should be moved back to CLOSED
- state.
+ * HDDS-11136: If a DELETING container has a non-empty CLOSED replica,
the container should be moved back to
+ * CLOSED state.
+ *
+ * HDDS-12421: If a DELETED container has a non-empty CLOSED replica,
the container should also be moved back to
+ * CLOSED state.
*/
+
+ boolean replicaIsEmpty = replica.hasIsEmpty() && replica.getIsEmpty();
+ // If container is in DELETED state and the reported replica is empty,
delete the empty replica.
+ if (container.getState() == HddsProtos.LifeCycleState.DELETED &&
replicaIsEmpty) {
+ deleteReplica(containerId, datanode, publisher, "DELETED");
+ ignored = true;
+ break;
+ }
+
boolean replicaStateAllowed = replica.getState() == State.CLOSED;
Review Comment:
We can remove this. We should make this transition for any non-empty
replicas.
##########
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/TestContainerReportHandling.java:
##########
@@ -64,13 +65,18 @@ public class TestContainerReportHandling {
private static final String KEY = "key1";
/**
- * Tests that a DELETING container moves to the CLOSED state if a non-empty
CLOSED replica is reported. To do this,
- * the test first creates a key and closes its corresponding container. Then
it moves that container to the
- * DELETING state using ContainerManager. Then it restarts a Datanode
hosting that container, making it send a full
- * container report. Then the test waits for the container to move from
DELETING to CLOSED.
+ * Tests that a DELETING (or DELETED) container moves to the CLOSED state if
a non-empty CLOSED replica is reported.
Review Comment:
We will do this for any non-empty replica.
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/AbstractContainerReportHandler.java:
##########
@@ -352,26 +352,32 @@ private boolean updateContainerState(final
DatanodeDetails datanode,
*/
break;
case DELETING:
+ case DELETED:
/*
- HDDS-11136: If a DELETING container has a non-empty CLOSED replica, the
container should be moved back to CLOSED
- state.
+ * HDDS-11136: If a DELETING container has a non-empty CLOSED replica,
the container should be moved back to
+ * CLOSED state.
+ *
+ * HDDS-12421: If a DELETED container has a non-empty CLOSED replica,
the container should also be moved back to
+ * CLOSED state.
*/
+
+ boolean replicaIsEmpty = replica.hasIsEmpty() && replica.getIsEmpty();
+ // If container is in DELETED state and the reported replica is empty,
delete the empty replica.
Review Comment:
Let's add a comment that for DELETING containers this is handled by the
replication manager in `DeletingContainerHandler`. We may want to clean up the
way these two components handle the tasks in the future but for the initial
patch I think this is fine.
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/health/DeletingContainerHandler.java:
##########
@@ -88,6 +88,7 @@ public boolean handle(ContainerCheckRequest request) {
//resend deleteCommand if needed
request.getContainerReplicas().stream()
.filter(r -> !pendingDelete.contains(r.getDatanodeDetails()))
+ .filter(r -> !r.isEmpty())
Review Comment:
```suggestion
.filter(r -> r.isEmpty())
```
From the javadoc:
> Returns a stream consisting of the elements of this stream that match the
given predicate
We only want to delete the known empty replicas with a non-force delete. If
this bug did not trigger a test failure then a new test needs to be added.
##########
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestContainerStateManager.java:
##########
@@ -181,7 +186,7 @@ public void
testTransitionDeletingToClosedStateAllowsOnlyDeletingContainer() thr
HddsProtos.ContainerID cid =
HddsProtos.ContainerID.newBuilder().setId(container.getContainerID()).build();
containerStateManager.addContainer(container);
try {
- containerStateManager.transitionDeletingToClosedState(cid);
+ containerStateManager.transitionDeletingOrDeletedToClosedState(cid);
Review Comment:
We should update the title of this test too, and maybe parameterize it with
the other container states instead of just quasi-closed.
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/AbstractContainerReportHandler.java:
##########
@@ -352,26 +352,32 @@ private boolean updateContainerState(final
DatanodeDetails datanode,
*/
break;
case DELETING:
+ case DELETED:
/*
- HDDS-11136: If a DELETING container has a non-empty CLOSED replica, the
container should be moved back to CLOSED
- state.
+ * HDDS-11136: If a DELETING container has a non-empty CLOSED replica,
the container should be moved back to
+ * CLOSED state.
+ *
+ * HDDS-12421: If a DELETED container has a non-empty CLOSED replica,
the container should also be moved back to
+ * CLOSED state.
Review Comment:
We are doing this for non-empty replicas in general, not just CLOSED ones.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
