AdheipSingh commented on issue #8801:
URL: https://github.com/apache/druid/issues/8801#issuecomment-723437981
@AWaterColorPen
> When statefulset controller scales down a MM we need to make sure we
don't scale down a MM pod which is running a task. How can we protect a MM
during scale down ?
> Here` to use Statefulsets for MM is another blocker. Lets say i have two
MM running with statefulsets MM-0 and MM-1. I have a scenario where MM-0 is not
running any task where as MM-1 is running a task, in case i scale down
kubernetes will delete MM-1, since StatefulSet controllers always removes the
most recently created MM.
^^ this is a blocker regardless of you using HPA's whatever feature, with
custom logic
The only righteous way as @himanshug has mentioned is to use cloneset ,
which works fine since it does not use deployment or statefullset controller to
run pods, each MM pod is treated as unique.
> Another blocker here is we cannot have a single service encapsulating MM
pods. We need to have separate service for each MM. In that way i can have a
total distinction of MM pod. So that if i hit this endpoint
/druid/worker/v1/tasks for MM-0 and MM-1 i get exact counts for each, with a
single service encapsulating it, it will LB between the two pods and give a
count for only one of them. So the operator for MM should not have replicas but
a count, so in case i specify 2 counts of MM the operator shall deploy 2
deployments of MM with 2 separate services.
Scale up is fine with HPA on pending Tasks count but scale down can only and
only be achieved if you treat each MM as unique which means you will not wrap a
MM under a single service, but for each MM-0, MM-1 you will have a seperate
service for each.
Plus supporting Mem/CPU for MM does not make any sense to me, Mem/CPU
should be only used with brokers nodetype in my knowledge
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
