We added the YuniKorn application CRD via YUNIKORN-170 [1]. The idea was back then that we used the CRD to implement gang scheduling. During the later design of gang scheduling we completely stepped away from using the CRD as the basis for gang scheduling. Some of the other advantages that we were expecting from the CRD were never observed (finished state for an application, one point to manage pods). The Spark CRD integration was mostly reversed [2] in favour of normal pod handling due to issues. The second phase for the application CRD YUNIKORN-599 [3] was never started due to the limited advantages we expected to get.
There have been no changes in the code or jiras logged against the CRD for two years besides making the build work on later K8s versions [4] The current usage of the application CRD is limited to the TaskGrooupDefinition being used for gang scheduling. Based on all this I would like to start the discussion on removing the application CRD from YuniKorn. Frank Yang has looked at the changes needed to remove the CRD and the impact on the code for the K8shim. A commit with all the changes can be seen in his repo [5] to reference. The change will remove almost 3,000 lines of code just from the K8shim repository. There will be some further changes needed to clean up the build (K8shim) and helmchart (release). Those changes will be removal of scripts and code only. Before we progress with this further we would like to know: * If the application CRD is used by anyone. * If it is used, what part(s) of the CRD are used and what is it used for? * Is removing the CRD in YuniKorn 1.4.0 OK or do we need to push to a later release. Objections, comments please let us know. Thank you, Wilfred [1] https://issues.apache.org/jira/browse/YUNIKORN-170 [2] https://issues.apache.org/jira/browse/YUNIKORN-643 [3] https://issues.apache.org/jira/browse/YUNIKORN-599 [4] https://github.com/apache/yunikorn-k8shim/commits/master/pkg/apis/yunikorn.apache.org [5] https://github.com/FrankYang0529/yunikorn-k8shim/commit/354248b3c24accd679ff2d5a557c599123a58408 --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
