Re: Scheduling tasks based on dependancy

Hi Sharma, Thanks for the clarification. Now I got it! Regards, Pradeep On 6 October 2015 at 19:35, Sharma Podila wrote: > Pradeep, attributes show up as name value pairs in the offers. Custom > attributes can also be used in Fenzo for assignment optimizations. For >

Re: Scheduling tasks based on dependancy

Hi Sharma, Awesome!. This is what I was looking for. Thanks for the reply. I will have a look for more info in wiki. Regards, Pradeep On 5 October 2015 at 18:28, Sharma Podila wrote: > Pradeep, > > We recently open sourced Fenzo (wiki >

Re: Scheduling tasks based on dependancy

Hi Sharma, Is this how you collect the network info from the VMs? First you get the resource offers from the Mesos and then you collect the network bandwidth info and then you use that for assigning for your tasks? Or The mesos-slave collects the resource information? But I don't see any code to

Re: Scheduling tasks based on dependancy

Hi Guangya, One doubt about the --attributes=rackid:r1;groupid:g1 option. How does the master provisions the resources? How will be the resource offer? Is it like (Rack 1 , G1, System)? how does this way of doing resource offer will help? Can you please give me more information? -Pradeep

Re: Scheduling tasks based on dependancy

Hi Pradeep, >From here https://github.com/apache/mesos/blob/master/src/master/master.cpp#L4913 you can see that the offer does include resource attributes, so you can get the filter the offers based on those attribute. Thanks, Guangya On Tue, Oct 6, 2015 at 7:03 PM, Pradeep Kiruvale

Re: Scheduling tasks based on dependancy

Hi Pradeep, We augment the mesos-slave command line on each agent to report the available network bandwidth in Mbps. For example, the agents that have 1Gbps bandwidth have this additional custom resources (--resources command line option) set: "network:1024". This shows up in the offers as a

Re: Scheduling tasks based on dependancy

Pradeep, attributes show up as name value pairs in the offers. Custom attributes can also be used in Fenzo for assignment optimizations. For example, we set custom attributes for AWS EC2 ZONE names and ASG names. We use the ZONE name custom attribute to balance tasks of a job across zones via the

Re: Scheduling tasks based on dependancy

Pradeep, There is a JIRA ticket doing your requirement but not finished yet, please refer to https://issues.apache.org/jira/browse/MESOS-3366 for detail. The basic idea is using customized hook modules to collect customized metrics. For now, you have to set metrics/resource manually for each

Re: Scheduling tasks based on dependancy

Hi Guangya, Thanks for information. Regards, Pradeep On 5 October 2015 at 17:57, Guangya Liu wrote: > Pradeep, > > There is a JIRA ticket doing your requirement but not finished yet, please > refer to https://issues.apache.org/jira/browse/MESOS-3366 for detail. The >

Re: Scheduling tasks based on dependancy

Hi Pradeep, Our opensource framework VAMP (www.vamp.io) might help you. It's possible to define entire topologies of services, and has drivers to run on mesos/marathon. Network bandwidth detection is not a current feature, but as our system is modular and open all kinds of metrics and

Re: Scheduling tasks based on dependancy

Pradeep, We recently open sourced Fenzo (wiki ) to handle these scenarios. We add a custom attribute for network bandwidth for each agent's "mesos-slave" command line. And we have Fenzo assign resources to tasks based on

RE: Scheduling tasks based on dependancy

Take a look at kubernetes. https://github.com/mesosphere/kubernetes-mesos From: Pradeep Kiruvale [pradeepkiruv...@gmail.com] Sent: Monday, October 05, 2015 11:21 AM To: user; d...@mesos.apache.org Subject: Scheduling tasks based on dependancy Hi All, Are

Re: Scheduling tasks based on dependancy

Hi Guangya, Thanks for your reply. That is nice feature to group the slaves into different racks and etc...But is there any way I can get metric of other hardware features other than CPU,MEM, DISK like IO, PCI devices that exists with the node etc? Thanks, Pradeep On 5 October 2015 at 17:45,