dongjoon-hyun commented on code in PR #869:
URL:
https://github.com/apache/spark-kubernetes-operator/pull/869#discussion_r4080830431
##########
spark-operator/src/main/java/org/apache/spark/k8s/operator/reconciler/reconcilesteps/AppCleanUpStep.java:
##########
@@ -195,10 +195,13 @@ public ReconcileProgress reconcile(
context, statusRecorder, state,
Duration.ofMillis(requeueAfterMillis));
} else {
+ // The resources have been released above. An application retaining them
has returned
+ // already, so it cannot terminate as
`TerminatedWithoutReleaseResources` here even if the
+ // retain policy applies and no more attempt is made.
updatedStatus =
currentStatus.terminateOrRestart(
tolerations.getRestartConfig(),
- tolerations.getResourceRetainPolicy(),
+ ResourceRetainPolicy.Never,
Review Comment:
Thank you. Updated `docs/spark_custom_resources.md` (the prose and state
`12` of the example) and the mermaid edge label in `docs/architecture.md` to
`Retain Policy, No Restart` in aacb729.
##########
spark-operator/src/main/java/org/apache/spark/k8s/operator/reconciler/reconcilesteps/AppCleanUpStep.java:
##########
@@ -195,10 +195,13 @@ public ReconcileProgress reconcile(
context, statusRecorder, state,
Duration.ofMillis(requeueAfterMillis));
} else {
+ // The resources have been released above. An application retaining them
has returned
+ // already, so it cannot terminate as
`TerminatedWithoutReleaseResources` here even if the
+ // retain policy applies and no more attempt is made.
Review Comment:
Thank you. I agree that it is dead code now. Since `ApplicationStatus` is a
public class of `spark-operator-api`, let me handle it in a follow-up JIRA
issue to keep this PR focused.
##########
spark-operator/src/test/java/org/apache/spark/k8s/operator/reconciler/reconcilesteps/AppCleanUpStepTest.java:
##########
@@ -352,6 +354,61 @@ void
cleanupWithRetainPolicyWithoutQueueNameDoesNotFinishKueueWorkload() {
}
}
+ @Test
+ void cleanupWithRetainPolicyAndRestartConfigTerminatesAsResourceReleased() {
+ // The resources of an application configured to restart are force
released, so it terminates
+ // as `ResourceReleased` even when the retain policy applies and no more
attempt is made.
+ Map<RestartConfig, ApplicationStateSummary> cases =
+ Map.of(
+ // The restart policy does not restart a succeeded application
+
RestartConfig.builder().restartPolicy(RestartPolicy.OnFailure).build(),
+ ApplicationStateSummary.Succeeded,
+ // The restart limit is exceeded
+ RestartConfig.builder()
+ .restartPolicy(RestartPolicy.Always)
+ .maxRestartAttempts(0L)
+ .build(),
+ ApplicationStateSummary.Failed);
+ for (Map.Entry<RestartConfig, ApplicationStateSummary> entry :
cases.entrySet()) {
+ SparkAppStatusRecorder mockRecorder = mock(SparkAppStatusRecorder.class);
+ AppCleanUpStep routineCheck = new AppCleanUpStep();
+ SparkApplication app = new SparkApplication();
+ app.setMetadata(
+ new ObjectMetaBuilder()
+ .withName("app1")
+ .withNamespace("default")
+ .withLabels(Map.of(Constants.LABEL_QUEUE_NAME, "test-queue"))
+ .build());
+ app.setSpec(
+ ApplicationSpec.builder()
+ .applicationTolerations(
+ ApplicationTolerations.builder()
+ .resourceRetainPolicy(ResourceRetainPolicy.Always)
Review Comment:
Thank you. Refactored the test with the `assertTerminatesAsResourceReleased`
helper and added the `OnFailure` case in aacb729.
##########
spark-operator/src/main/java/org/apache/spark/k8s/operator/reconciler/reconcilesteps/AppCleanUpStep.java:
##########
@@ -195,10 +195,13 @@ public ReconcileProgress reconcile(
context, statusRecorder, state,
Duration.ofMillis(requeueAfterMillis));
} else {
+ // The resources have been released above. An application retaining them
has returned
+ // already, so it cannot terminate as
`TerminatedWithoutReleaseResources` here even if the
+ // retain policy applies and no more attempt is made.
updatedStatus =
currentStatus.terminateOrRestart(
tolerations.getRestartConfig(),
- tolerations.getResourceRetainPolicy(),
+ ResourceRetainPolicy.Never,
stateUpdateMessage,
Review Comment:
Thank you. Added the space and asserted the message in the new test in
aacb729.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
