Tobias, we have taken the approach of holding on to "some" offers to support fast launch of tasks while being a "good citizen" of a cluster by not holding on to all unused offers for too long. We routinely have tasks launched within a low number of milliSeconds after submission. This is supported in our open source Fenzo scheduling library. While I don't want to assume you can use Fenzo for your scheduler (although, you should be able to if your framework runs on the JVM), you may look at Fenzo documentation to understand the approach we have taken. Specifically, look at the JavaDocs for OfferExpirySecs <http://netflix.github.io/Fenzo/fenzo-core/com/netflix/fenzo/TaskScheduler.Builder.html#withLeaseOfferExpirySecs-long->, maxOffersToReject <http://netflix.github.io/Fenzo/fenzo-core/com/netflix/fenzo/TaskScheduler.Builder.html#withMaxOffersToReject-int->, and rejectAllExpiredOffers <http://netflix.github.io/Fenzo/fenzo-core/com/netflix/fenzo/TaskScheduler.Builder.html#withRejectAllExpiredOffers-->. Fenzo is being used in multi-framework clusters using these controls.
On Wed, Aug 31, 2016 at 2:25 PM, Tobias Pfeiffer <[email protected]> wrote: > Hi Alexander, > > thanks for your suggestion. > > On Wed, Aug 31, 2016 at 9:11 PM, Alexander Rojas <[email protected]> > wrote: >> >> There are some things you can try, one would be to change the default >> value of the master’s `allocation_interval` flag, which is 1. This will >> increase the rate at which offers are being generated. >> > > I see. Do you think that in general this is the right approach, processing > all tasks that have piled up in the last second within the > `resourceOffers()` method? > Also, I read somewhere that on accepting an offer, all other offers will > be declined; but is this only true for offers from the same slave? > > Thanks, > Tobias > > >
