Hi Gyula/Team,
Thanks for your response. I see that the above one works on Kubernetes, but all
our applications are currently running on EMR Yarn and looking to integrate
with EMR-managed scaling with adaptive scheduler.
Do we have any features or operators which we can use to run on Yarn similar to
Flink Kubernetes Operator which can perform auto-scaling.
Regards,Madan
On Friday, 13 January 2023 at 09:37:11 pm GMT-8, Gyula Fóra
<gyula.f...@gmail.com> wrote:
Hi Madan,
With reactive mode you need to build a completely custom auto scaling logic, it
can work but it takes considerable effort.
Instead I recommend using the Flink Kubernetes Operator which now contains the
first version of an actual autoscaler module that collects metrics and scales
Flink jobs .
https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/custom-resource/autoscaler/
The operator autoscaler uses Kubernetes native integration and works more or
less out of the box.
Cheers Gyula
On Sat, 14 Jan 2023 at 01:16, Madan D via user <user@flink.apache.org> wrote:
Hello Team,I would like to understand auto scaling on EMR using either reactive
mode or adaptive scheduler with custom or managed scaling.Can some one help me
on this.
Regards,Madan
