Jobs/tasks wait in framework schedulers, not mesos master. Autoscaling triggers must come from schedulers, not only because that's who knows the pending task set size, but, also because it knows how many of them need to be launched right away, on what kind of machines.
We built such an autoscaling capability in our framework schedulers. The autoscaling is achieved by our library Fenzo <https://github.com/Netflix/Fenzo> which we open sourced recently. Also read about Fenzo autoscaling here <https://github.com/Netflix/Fenzo/wiki/Autoscaling>. You should look into using that if you are developing your own scheduler. Or, have your scheduler team pick up Fenzo for autoscaling. Also, note that scaling up is temptingly easy by watching the pending task queue. But, scaling down requires bin packing, etc. Other issues pop up as well, for example: - what if a user submits tasks that cannot be satisfied? Will autoscale keep increasing the cluster size unbounded? - what if you would like to have a heterogeneous mix of hosts and tasks? which kind of hosts do you need to autoscale based on which tasks are pending? These are automatically addressed in Fenzo. Sharma On Wed, Sep 23, 2015 at 4:56 AM, Aaron Carey <[email protected]> wrote: > No, I basically had the same question as Jim (but maybe didn't word it so > well ;)) > > I'll have a look at your response there :) > > ------------------------------ > *From:* haosdent [[email protected]] > *Sent:* 23 September 2015 10:12 > *To:* [email protected] > *Subject:* Re: Metric for tasks queued/waiting? > > Does /metrics/snapshot not satisfy your requirement? > > On Wed, Sep 23, 2015 at 4:50 PM, Aaron Carey <[email protected]> wrote: > >> Hi all, >> >> Is there any way to get a metric of all tasks currently waiting/queued in >> Mesos (across all schedulers)? The snapshot metrics seem to cover ever >> other kind of task state? This would be quite useful for auto-scaling >> purposes.. >> >> Thanks, >> Aaron >> > > > > -- > Best Regards, > Haosdent Huang >
