Hi Sharma, thank you for your feedback. I had also considered caching "just a couple of offers" while declining the rest, so I guess that's one possible approach. Also thanks for pointing me to Fenzo, I will give it a good look and consider using it in my scheduler.
Thanks, Tobias On Wed, Aug 31, 2016 at 10:48 PM, Sharma Podila <[email protected]> wrote: > Tobias, we have taken the approach of holding on to "some" offers to > support fast launch of tasks while being a "good citizen" of a cluster by > not holding on to all unused offers for too long. We routinely have tasks > launched within a low number of milliSeconds after submission. This is > supported in our open source Fenzo scheduling library. While I don't want > to assume you can use Fenzo for your scheduler (although, you should be > able to if your framework runs on the JVM), you may look at Fenzo > documentation to understand the approach we have taken. Specifically, look > at the JavaDocs for OfferExpirySecs > <http://netflix.github.io/Fenzo/fenzo-core/com/netflix/fenzo/TaskScheduler.Builder.html#withLeaseOfferExpirySecs-long->, > maxOffersToReject > <http://netflix.github.io/Fenzo/fenzo-core/com/netflix/fenzo/TaskScheduler.Builder.html#withMaxOffersToReject-int->, > and rejectAllExpiredOffers > <http://netflix.github.io/Fenzo/fenzo-core/com/netflix/fenzo/TaskScheduler.Builder.html#withRejectAllExpiredOffers-->. > Fenzo is being used in multi-framework clusters using these controls. > > On Wed, Aug 31, 2016 at 2:25 PM, Tobias Pfeiffer <[email protected]> wrote: > >> Hi Alexander, >> >> thanks for your suggestion. >> >> On Wed, Aug 31, 2016 at 9:11 PM, Alexander Rojas <[email protected] >> > wrote: >>> >>> There are some things you can try, one would be to change the default >>> value of the master’s `allocation_interval` flag, which is 1. This will >>> increase the rate at which offers are being generated. >>> >> >> I see. Do you think that in general this is the right approach, >> processing all tasks that have piled up in the last second within the >> `resourceOffers()` method? >> Also, I read somewhere that on accepting an offer, all other offers will >> be declined; but is this only true for offers from the same slave? >> >> Thanks, >> Tobias >> >> >> >
