Hi Chenya The use case is similar, YK will play a big role there. Lots of features are relevant, such as queues, job ordering, user/group ACLs, preemption, over-subscription, and performance etc. Some of the basic functionalities are available in YK, some more needs to be built. Please take a look at the slides from the Alibaba Flink team, they have shared how they use YK to address their use cases. This was presented in ApacheConf: https://www.youtube.com/watch?v=4hghJCuZk5M
On Tue, Jan 4, 2022 at 6:35 PM Chenya Zhang <[email protected]> wrote: > Hey folks, > > We have some new streaming use cases with Apache Flink that could > potentially leverage YuniKorn for resource scheduling. > > The initial implementation is to use K8s namespace for resource quota > management. We are investigating what could be some strong benefits > switching to YuniKorn in streaming cases for long-running services. For > example: Job queueing, job ordering, resource reservation, user groups etc > all seem to be more desirable for batch use cases. > > Any thoughts or suggestions? > > Thanks, > Chenya >
