Re: Specifying a preferred host with a Resource Request

2016-02-08 Thread Klaus Ma
Hi Jagadish, Yes, the resources will offer to the framework if it's reserved and only one framework in the role. Reservation is role based; if there're two frameworks in the same role, both frameworks has the chance to get the reserved offer. For your case, if you want to get the reserved

Re: Specifying a preferred host with a Resource Request

2016-02-08 Thread Jagadish Venkatraman
Thank you Klaus! Out of curiosity, does Mesos persist these reservations for principals durably? Do dynamic resource reservations persist restart of the Mesos master? On Mon, Feb 8, 2016 at 12:15 AM, Klaus Ma wrote: > Hi Jagadish, > > Yes, the resources will offer to

Re: Specifying a preferred host with a Resource Request

2016-02-08 Thread Guangya Liu
Yes, those infos will be persisted by Mesos and will not be lost even after mesos restart. Thanks, Guangya On Tue, Feb 9, 2016 at 3:23 AM, Jagadish Venkatraman wrote: > Thank you Klaus! Out of curiosity, does Mesos persist these reservations > for principals durably?

Re: Specifying a preferred host with a Resource Request

2016-02-07 Thread Jagadish Venkatraman
Hi Guangya and Klaus, Thanks for the helpful pointers to the docs. I really appreciate it! Consider this scenario: - A framework F with role 'test'. F reserves resources and creates 3 persistent volumes on 3 separate hosts. (call them h1, h2, h3) and launches tasks. Everything runs

Re: Specifying a preferred host with a Resource Request

2016-02-06 Thread Vinod Kone
What you want here are dynamic reservations and persistent volumes. Take a look at our docs for these features. @vinodkone > On Feb 6, 2016, at 10:31 AM, Jagadish Venkatraman > wrote: > > Hi Guangya, > > Thanks for the response! Let me provide more background to

Re: Specifying a preferred host with a Resource Request

2016-02-06 Thread Jagadish Venkatraman
Hi Guangya, Thanks for the response! Let me provide more background to this request. *Background:* I work on Apache Samza , a distributed stream processing framework. Currently Samza supports only Yarn as a resource manager. (there have been requests to run Samza with

Re: Specifying a preferred host with a Resource Request

2016-02-06 Thread Klaus Ma
Hi Jagadish, This the requirement for dynamic reservation and persistent volumes :). Here's the related document: Reservation: http://mesos.apache.org/documentation/latest/reservation/ Persistent Volume: http://mesos.apache.org/documentation/latest/persistent-volume/ Thanks Klaus On Sun, Feb

Specifying a preferred host with a Resource Request

2016-02-05 Thread Jagadish Venkatraman
I have fair experience in writing frameworks on Yarn. In the Yarn world, the amClient supports a method where I can specify the preferredHost with the resource request. Is there a way to specify a preferred host with the resource request in Mesos? I currently do: driver.requestResources

Re: Specifying a preferred host with a Resource Request

2016-02-05 Thread Guangya Liu
Hi Jagadish, Even though Mesos have the interface of "requestResources", it was not implemented in the built-in allocator at the moment, so the call of "driver.requestResources (resources);" will not work. Is it possible that you update your framework logic as this: 1) framework get resoruce