errose28 commented on code in PR #7976:
URL: https://github.com/apache/ozone/pull/7976#discussion_r1975843193
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/AbstractContainerReportHandler.java:
##########
@@ -352,26 +352,32 @@ private boolean updateContainerState(final
DatanodeDetails datanode,
*/
break;
case DELETING:
+ case DELETED:
/*
- HDDS-11136: If a DELETING container has a non-empty CLOSED replica, the
container should be moved back to CLOSED
- state.
+ * HDDS-11136: If a DELETING container has a non-empty CLOSED replica,
the container should be moved back to
+ * CLOSED state.
+ *
+ * HDDS-12421: If a DELETING or DELETED container has a non-empty
replica, the container should also be moved
+ * back to CLOSED state.
*/
- boolean replicaStateAllowed = replica.getState() == State.CLOSED;
- boolean replicaNotEmpty = replica.hasIsEmpty() && !replica.getIsEmpty();
- if (replicaStateAllowed && replicaNotEmpty) {
- logger.info("Moving DELETING container {} to CLOSED state, datanode {}
reported replica with state={}, " +
- "isEmpty={} and keyCount={}.", containerId,
datanode.getHostName(), replica.getState(), false,
- replica.getKeyCount());
- containerManager.transitionDeletingToClosedState(containerId);
+ boolean replicaIsEmpty = replica.hasIsEmpty() && replica.getIsEmpty();
+ // If container is in DELETED state and the reported replica is empty,
delete the empty replica.
+ // We should also do this for DELETING containers and currently
DeletingContainerHandler does that
+ if (container.getState() == HddsProtos.LifeCycleState.DELETED &&
replicaIsEmpty) {
+ // should we send a non-force delete instead?
Review Comment:
Yes we should use non-force delete in all cases except over-replication.
Good catch I missed that this method was using force internally. We can add
`force` as a parameter to `deleteReplica`, it only has two callers.
##########
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestContainerReportHandler.java:
##########
@@ -446,64 +484,70 @@ public void testClosingToClosedForECContainer()
}
/**
- * Tests that a DELETING RATIS container transitions to CLOSED if a
non-empty CLOSED replica is reported. It does not
- * transition if a non-empty CLOSING replica is reported.
+ * Tests that a DELETING or DELETED RATIS container transitions to CLOSED if
a non-empty replica in OPEN, CLOSING,
+ * CLOSED, QUASI_CLOSED or UNHEALTHY state is reported. It should not
transition if the replica is in INVALID or
+ * DELETED states.
*/
- @Test
- public void
ratisContainerShouldTransitionFromDeletingToClosedWhenNonEmptyClosedReplica()
throws IOException {
- ContainerInfo container = getContainer(LifeCycleState.DELETING);
+ @ParameterizedTest
+ @MethodSource("containerAndReplicaStates")
+ public void
ratisContainerShouldTransitionFromDeletingOrDeletedToClosedWhenNonEmptyReplica(
Review Comment:
There is also
`ecContainerShouldTransitionFromDeletingToClosedWhenNonEmptyClosedReplica`
later in this class. We can probably just make EC/Ratis a parameter in this
test as well. We would just have to skip quasi-closed replica state for the EC
container when generating the combinations in `containerAndReplicaStates`.
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/AbstractContainerReportHandler.java:
##########
@@ -352,26 +352,32 @@ private boolean updateContainerState(final
DatanodeDetails datanode,
*/
break;
case DELETING:
+ case DELETED:
/*
- HDDS-11136: If a DELETING container has a non-empty CLOSED replica, the
container should be moved back to CLOSED
- state.
+ * HDDS-11136: If a DELETING container has a non-empty CLOSED replica,
the container should be moved back to
+ * CLOSED state.
+ *
+ * HDDS-12421: If a DELETED container has a non-empty CLOSED replica,
the container should also be moved back to
+ * CLOSED state.
*/
+
+ boolean replicaIsEmpty = replica.hasIsEmpty() && replica.getIsEmpty();
+ // If container is in DELETED state and the reported replica is empty,
delete the empty replica.
+ if (container.getState() == HddsProtos.LifeCycleState.DELETED &&
replicaIsEmpty) {
+ deleteReplica(containerId, datanode, publisher, "DELETED");
Review Comment:
Yes as I [commented
earlier](https://github.com/apache/ozone/pull/7976#issuecomment-2689055000)
let's only use force delete for over-replicated cases.
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/AbstractContainerReportHandler.java:
##########
@@ -352,26 +352,32 @@ private boolean updateContainerState(final
DatanodeDetails datanode,
*/
break;
case DELETING:
+ case DELETED:
/*
- HDDS-11136: If a DELETING container has a non-empty CLOSED replica, the
container should be moved back to CLOSED
- state.
+ * HDDS-11136: If a DELETING container has a non-empty CLOSED replica,
the container should be moved back to
+ * CLOSED state.
+ *
+ * HDDS-12421: If a DELETED container has a non-empty CLOSED replica,
the container should also be moved back to
+ * CLOSED state.
*/
+
+ boolean replicaIsEmpty = replica.hasIsEmpty() && replica.getIsEmpty();
+ // If container is in DELETED state and the reported replica is empty,
delete the empty replica.
+ if (container.getState() == HddsProtos.LifeCycleState.DELETED &&
replicaIsEmpty) {
+ deleteReplica(containerId, datanode, publisher, "DELETED");
+ ignored = true;
+ break;
+ }
+
boolean replicaStateAllowed = replica.getState() == State.CLOSED;
boolean replicaNotEmpty = replica.hasIsEmpty() && !replica.getIsEmpty();
if (replicaStateAllowed && replicaNotEmpty) {
- logger.info("Moving DELETING container {} to CLOSED state, datanode {}
reported replica with state={}, " +
- "isEmpty={} and keyCount={}.", containerId,
datanode.getHostName(), replica.getState(), false,
- replica.getKeyCount());
- containerManager.transitionDeletingToClosedState(containerId);
+ logger.info("Moving container {} from {} to CLOSED state, datanode {}
reported replica with state={}, " +
Review Comment:
I don't think containers in these states would currently be reported to SCM
anyways, but I think `isEmpty` is more robust than any state check in this
context so that is all we should use.
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/AbstractContainerReportHandler.java:
##########
@@ -352,26 +352,32 @@ private boolean updateContainerState(final
DatanodeDetails datanode,
*/
break;
case DELETING:
+ case DELETED:
/*
- HDDS-11136: If a DELETING container has a non-empty CLOSED replica, the
container should be moved back to CLOSED
- state.
+ * HDDS-11136: If a DELETING container has a non-empty CLOSED replica,
the container should be moved back to
+ * CLOSED state.
+ *
+ * HDDS-12421: If a DELETING or DELETED container has a non-empty
replica, the container should also be moved
+ * back to CLOSED state.
*/
- boolean replicaStateAllowed = replica.getState() == State.CLOSED;
- boolean replicaNotEmpty = replica.hasIsEmpty() && !replica.getIsEmpty();
- if (replicaStateAllowed && replicaNotEmpty) {
- logger.info("Moving DELETING container {} to CLOSED state, datanode {}
reported replica with state={}, " +
- "isEmpty={} and keyCount={}.", containerId,
datanode.getHostName(), replica.getState(), false,
- replica.getKeyCount());
- containerManager.transitionDeletingToClosedState(containerId);
+ boolean replicaIsEmpty = replica.hasIsEmpty() && replica.getIsEmpty();
+ // If container is in DELETED state and the reported replica is empty,
delete the empty replica.
+ // We should also do this for DELETING containers and currently
DeletingContainerHandler does that
+ if (container.getState() == HddsProtos.LifeCycleState.DELETED &&
replicaIsEmpty) {
+ // should we send a non-force delete instead?
Review Comment:
In a future change we could consider a refactoring where force is never
given as a parameter. All methods would be of the form `sendDeleteCommand` or
`sendForceDeleteCommand`. `DeleteContainerCommand` could have two static
factories to build force and non-force variants with private constructors. This
would make it clearer when reading the code which paths are doing the
"dangerous" force delete operation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
