[
https://issues.apache.org/jira/browse/YUNIKORN-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17710975#comment-17710975
]
Craig Condit commented on YUNIKORN-1555:
----------------------------------------
Just saw this one. While I agree we can ignore Succeeded or Failed pods, we do
still need to track Terminating pods during recovery as these represent
resources that are still in-use and need to be accounted for. Once the pod
transitions into a terminal state (after Terminating), only then can we release
those resources.
> Completed Spark applications are recovered/remain as New
> --------------------------------------------------------
>
> Key: YUNIKORN-1555
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1555
> Project: Apache YuniKorn
> Issue Type: Bug
> Components: shim - kubernetes
> Affects Versions: 1.1.0, 1.2.0
> Reporter: Brandon Grams
> Assignee: Brandon Grams
> Priority: Major
> Labels: pull-request-available
> Attachments: CEEB85D7-D0D5-4D3D-8C0A-730CBA48B150.jpeg
>
>
> The k8s Spark plugin is
> [documented|https://github.com/apache/yunikorn-k8shim/blob/master/pkg/appmgmt/sparkoperator/spark.go#L33]
> as implementing the Recoverable interface, however it does not. This leads
> to the following recovery outcome when driver pods in a terminal state remain
> in the cluster pending garbage collection:
> !CEEB85D7-D0D5-4D3D-8C0A-730CBA48B150.jpeg|width=1063,height=157!
> Instead, we should implement the Recoverable interface so that completed
> applications have their proper state propagated in such a scenario.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
