I’m definitely in favor of removing the CRD, and sooner rather than later. It negatively impacts some of the ongoing refactoring tasks as it influences the recovery pipeline. I think given the alpha state of the CRD, if there are no objections we can remove this in 1.4.0.
Craig > On Jul 27, 2023, at 9:41 PM, Wilfred Spiegelenburg <[email protected]> > wrote: > > We added the YuniKorn application CRD via YUNIKORN-170 [1]. The idea > was back then that we used the CRD to implement gang scheduling. > During the later design of gang scheduling we completely stepped away > from using the CRD as the basis for gang scheduling. Some of the other > advantages that we were expecting from the CRD were never observed > (finished state for an application, one point to manage pods). The > Spark CRD integration was mostly reversed [2] in favour of normal pod > handling due to issues. > The second phase for the application CRD YUNIKORN-599 [3] was never > started due to the limited advantages we expected to get. > > There have been no changes in the code or jiras logged against the CRD > for two years besides making the build work on later K8s versions [4] > The current usage of the application CRD is limited to the > TaskGrooupDefinition being used for gang scheduling. > > Based on all this I would like to start the discussion on removing the > application CRD from YuniKorn. Frank Yang has looked at the changes > needed to remove the CRD and the impact on the code for the K8shim. A > commit with all the changes can be seen in his repo [5] to reference. > The change will remove almost 3,000 lines of code just from the K8shim > repository. There will be some further changes needed to clean up the > build (K8shim) and helmchart (release). Those changes will be removal > of scripts and code only. > > Before we progress with this further we would like to know: > * If the application CRD is used by anyone. > * If it is used, what part(s) of the CRD are used and what is it used for? > * Is removing the CRD in YuniKorn 1.4.0 OK or do we need to push to a > later release. > > Objections, comments please let us know. > > Thank you, > Wilfred > > [1] https://issues.apache.org/jira/browse/YUNIKORN-170 > [2] https://issues.apache.org/jira/browse/YUNIKORN-643 > [3] https://issues.apache.org/jira/browse/YUNIKORN-599 > [4] > https://github.com/apache/yunikorn-k8shim/commits/master/pkg/apis/yunikorn.apache.org > [5] > https://github.com/FrankYang0529/yunikorn-k8shim/commit/354248b3c24accd679ff2d5a557c599123a58408 > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
