[
https://issues.apache.org/jira/browse/YUNIKORN-88?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17080657#comment-17080657
]
Weiwei Yang commented on YUNIKORN-88:
-------------------------------------
Hi [~ayubpathan]
Thank you. This is a known issue.
Well, the reason is on K8s, it is "difficult" to tell if an app is completed.
When we run workloads on K8s, yunikorn groups a bunch of pods and claim them
belong to one application, usually leverage some meta info in the pods' spec,
such as label "applicationID". YuniKorn only sees pods with app IDs, we can
tell if a pod is completed, but not the app. How can we know if there will be
new pods added for this app?
This is something we need to improve for certain cases:
# The framework already supports K8s operators, e.g spark-k8s-operator. The
operator pattern manages the job's lifecycle in a CRD, so we can easily
retrieve the status of a job from the CRD.
# For the native Spark on k8s, with spark-submit. We can check the status of
the spark driver pod, when the driver pod is finished, we can say the job is
finished.
Unfortunately, we have to improve this case by case. There is no common fix for
all sorts of applications so far I can see.
Please let me know your thoughts, thanks!
> Yunikorn webapp shows the apps as running even though they are completed and
> terminated
> ---------------------------------------------------------------------------------------
>
> Key: YUNIKORN-88
> URL: https://issues.apache.org/jira/browse/YUNIKORN-88
> Project: Apache YuniKorn
> Issue Type: Bug
> Components: webapp
> Reporter: Ayub Pathan
> Assignee: Weiwei Yang
> Priority: Major
> Attachments: image-2020-04-09-19-18-03-414.png,
> image-2020-04-09-19-18-37-721.png
>
>
> Attaching the snapshots
> !image-2020-04-09-19-18-03-414.png|thumbnail!
> !image-2020-04-09-19-18-37-721.png|thumbnail!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
