Feel free to open a story in jira if you think you ideas are awesome. :-) On Sep 23, 2015 10:54 PM, "Sharma Podila" <[email protected]> wrote:
> Ah, OK, thanks. Yes, Fenzo is a Java library. > > It might be a nice addition to Mesos master to get a global view of > contention for resources. In addition to autoscaling, it would be useful in > the allocator. > > > > On Wed, Sep 23, 2015 at 7:29 AM, Aaron Carey <[email protected]> wrote: > >> Thanks Sharma, >> >> I was in the audience for a talk you did about Fenzo at MesosCon :) It >> looked great but we're a python shop primarily so the Java requirement >> would be a problem for us. >> >> The scaling in the scheduler makes total sense, (obvious when you think >> about it!), I was naively hoping for some sort of knowledge of that back in >> the Mesos master as we were hoping to have scaling be independent of >> schedulers. I think this'll need a re-think! >> >> Thanks for your help! >> >> Aaron >> >> ------------------------------ >> *From:* Sharma Podila [[email protected]] >> *Sent:* 23 September 2015 15:22 >> >> *To:* [email protected] >> *Subject:* Re: Metric for tasks queued/waiting? >> >> Jobs/tasks wait in framework schedulers, not mesos master. Autoscaling >> triggers must come from schedulers, not only because that's who knows the >> pending task set size, but, also because it knows how many of them need to >> be launched right away, on what kind of machines. >> >> We built such an autoscaling capability in our framework schedulers. The >> autoscaling is achieved by our library Fenzo >> <https://github.com/Netflix/Fenzo> which we open sourced recently. Also >> read about Fenzo autoscaling here >> <https://github.com/Netflix/Fenzo/wiki/Autoscaling>. You should look >> into using that if you are developing your own scheduler. Or, have your >> scheduler team pick up Fenzo for autoscaling. >> >> Also, note that scaling up is temptingly easy by watching the pending >> task queue. But, scaling down requires bin packing, etc. Other issues pop >> up as well, for example: >> >> - what if a user submits tasks that cannot be satisfied? Will autoscale >> keep increasing the cluster size unbounded? >> - what if you would like to have a heterogeneous mix of hosts and tasks? >> which kind of hosts do you need to autoscale based on which tasks are >> pending? >> >> These are automatically addressed in Fenzo. >> >> Sharma >> >> >> On Wed, Sep 23, 2015 at 4:56 AM, Aaron Carey <[email protected]> wrote: >> >>> No, I basically had the same question as Jim (but maybe didn't word it >>> so well ;)) >>> >>> I'll have a look at your response there :) >>> >>> ------------------------------ >>> *From:* haosdent [[email protected]] >>> *Sent:* 23 September 2015 10:12 >>> *To:* [email protected] >>> *Subject:* Re: Metric for tasks queued/waiting? >>> >>> Does /metrics/snapshot not satisfy your requirement? >>> >>> On Wed, Sep 23, 2015 at 4:50 PM, Aaron Carey <[email protected]> wrote: >>> >>>> Hi all, >>>> >>>> Is there any way to get a metric of all tasks currently waiting/queued >>>> in Mesos (across all schedulers)? The snapshot metrics seem to cover ever >>>> other kind of task state? This would be quite useful for auto-scaling >>>> purposes.. >>>> >>>> Thanks, >>>> Aaron >>>> >>> >>> >>> >>> -- >>> Best Regards, >>> Haosdent Huang >>> >> >> >
