[
https://issues.apache.org/jira/browse/YUNIKORN-2092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17782042#comment-17782042
]
Wilfred Spiegelenburg commented on YUNIKORN-2092:
-------------------------------------------------
This will need a follow up to clean up more code as a number of functions, and
even application states and event, that are no longer called or used by
anything but the test code.
This should also not lead to any changes in behaviour on the YuniKorn side even
if the SparkOperator is used. The only two additions the operator has inside
YuniKorn is the application state changes. The operator bases its state changes
on the pods that are updated. That is the same as YuniKorn which means that
with or without the operator we come to the same conclusion for the application
state.
> Remove Spark Operator AppManager
> --------------------------------
>
> Key: YUNIKORN-2092
> URL: https://issues.apache.org/jira/browse/YUNIKORN-2092
> Project: Apache YuniKorn
> Issue Type: Improvement
> Components: shim - kubernetes
> Reporter: Craig Condit
> Assignee: Craig Condit
> Priority: Major
> Labels: pull-request-available, release-notes
>
> For some time, YuniKorn has supported a configurable AppManager interface.
> However, there are only two implementations:
> * general: used for general application management (maps via applicationID)
> * spark-k8s-operator: mapping via spark operator
> This interface has grown in complexity over time, but is not used
> consistently within the code base. Disabling the general AppManager results
> in a broken scheduler, as pods do not map correctly. Additionally, the
> spark-k8s-operator AppManager is only used in non-default configurations and
> only maps Spark Application completion state onto YuniKorn application
> completion state via calling NotifyApplicationFail() or
> NotifyApplicationComplete(). This can conflict with the normal processing of
> these events and lead to failures.
> To simplify and eventually eliminate the AppManager abstraction, we should
> remove the spark-k8s-operator AppManager entirely. No functionality will be
> lost (except for a questionable mapping between spark app success/fail and
> yunikorn app success/fail). This mapping is questionable anyway as there is
> no real concept of Application success in Kubernetes, so it is not used in
> the vast majority of cases.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
