True. I also see that the pool of threads for processing offers has a size of 1. Am I reading that wrong? Will a larger pool increase performance? Or is there a lock somewhere (like DB updates) that makes that useless?
On Mon, Oct 2, 2017 at 5:07 PM, Bill Farner <[email protected]> wrote: > Is there a place where we store "used" offers > > > Once the scheduler has decided to use an offer, the offer is removed > <https://github.com/apache/aurora/blob/6fd6d50288d6155fda212cecab539c5fb669a9aa/src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java#L418> > from OfferManager. The situation you describe is indeed possible. > However, i don't suspect that there's much to gain from trying to mitigate > an accept/rescind race. That type of race also wouldn't cause any state > integrity issues in the scheduler, which is what this 'global ban' routine > was intending ti plug. > > On Mon, Oct 2, 2017 at 4:49 PM, Mohit Jaggi <[email protected]> wrote: > >> I was wondering if that case can be checked and the banning skipped. Is >> there a place where we store "used" offers? At first glance it looks like >> there isn't...perhaps deep down in job/task state but that will be too >> expensive to check. >> >> On Mon, Oct 2, 2017 at 4:44 PM, Bill Farner <[email protected]> wrote: >> >>> That's true, but it doesn't appear the comment is trying to lay out all >>> possible scenarios. Instead, it is attempting to explain the rationale for >>> offerManager.banOffer(offerId) a few lines later. >>> >>> On Mon, Oct 2, 2017 at 4:30 PM, Mohit Jaggi <[email protected]> >>> wrote: >>> >>>> Folks, >>>> In the code below, isn't there a 3rd case where the offer was "used" >>>> and hence not found for canceling? >>>> >>>> Mohit. >>>> >>>> public void handleRescind(OfferID offerId) { >>>> log.info("Offer rescinded: {}", offerId.getValue()); >>>> >>>> // For rescinds, we want to ensure they are processed quickly before we >>>> attempt to use an >>>> // invalid offer. There are a few scenarios we want to be aware of: >>>> // 1. We receive an offer, add it to OfferManager, and then get a >>>> rescind. In this scenario, >>>> // we can just remove the offer from the offers list. >>>> // 2. We receive an offer, but before we add it to the OfferManager >>>> list we get a rescind. >>>> // In this scenario, we want to ensure that we do not use it/accept >>>> it when the executor >>>> // finally processes the offer. We will temporarily ban it and add >>>> a command for the >>>> // executor to unban it so future offers can be processed normally. >>>> boolean offerCancelled = offerManager.cancelOffer(offerId); >>>> >>>> >>> >> >
