So "dispatches immediately to an invoker" is currently implemented as a Kafka topic-per-invoker. If we don't change that Kafka transport to something else, I'm not sure what you have in mind to get down to "only 2 queues". In other words the "ready queue" seems problematic to dispatch via a single queue to multiple invokers if based on the current Kafka impl.
> On Oct 4, 2017, at 6:29 PM, Rodric Rabbah <[email protected]> wrote: > > What you’re describing is generalized to essentially two queues. The ready > queue which dispatches immediately to an invoker with capacity and an > overflow queue. Whether the latter is one per invoker or global may not > matter much as long as it’s drained by the load balancer and not committed > otherwise to the invoker ready queue where it is difficult to reassign to an > available invoker later (ala work stealing). There are pathologies one has to > guard against and we should consider fairness policies as well. There’s good > theory in this space that I can imagine using to model the heuristics. > > -r > >> On Oct 4, 2017, at 8:39 PM, Tyson Norris <[email protected]> wrote: >> >> Not sure what you mean by two queues - you mean two queues per invoker? Or >> total? >> >>> On Oct 4, 2017, at 5:14 PM, Rodric Rabbah <[email protected]> wrote: >>> >>> A two queue (topic) approach can mitigate the lack of random access from a >>> Kafka topic/queue once a request is committed (in today’s architecture). >>> This could enable work stealing in particular since the (overflow) queue >>> can be drained upstream (ie load balancer) and reassigned to free invokers. >>> Balancing cold start that container locality would them be a heuristic we >>> can apply more judiciously as capacity is available (versus the current >>> approach which binds too early and prevents rebalancing). >>> >>> -r >>> >>>> On Oct 4, 2017, at 7:45 PM, Tyson Norris <[email protected]> wrote: >>>> >>>> Hi - >>>> I’ve been discussing a bit with a few about optimizing the queueing that >>>> goes on ahead of invokers so that things behave more simply and >>>> predictable. >>>> >>>> In short: Instead of scheduling activations to an invoker on receipt, do >>>> the following: >>>> - execute the activation "immediately" if capacity is available >>>> - provide a single overflow topic for activations that cannot execute >>>> “immediately" >>>> - schedule from the overflow topic when capacity is available >>>> >>>> (BTW “Immediately” means: still queued via existing invoker topics, but >>>> ONLY gets queued there in the case that the invoker is not fully loaded, >>>> and therefore should execute it “very soon") >>>> >>>> Later: it would also be good to provide more container state data from >>>> invoker to controller, to get better scheduling options - e.g. if some >>>> invokers can handle running more containers than other invokers, that info >>>> can be used to avoid over/under-loading the invokers (currently we assume >>>> each invoker can handle 16 activations, I think) >>>> >>>> I put a wiki page proposal here: >>>> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FOPENWHISK%2FInvoker%2BActivation%2BQueueing%2BChange&data=02%7C01%7C%7C3ff41b0527be4e13398c08d50b8614ee%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636427592860490625&sdata=afOAkeXRW6iArXa72IGd6gnhVyrOwXsDxnjfSAQqL18%3D&reserved=0 >>>> >>>> WDYT? >>>> >>>> Thanks >>>> Tyson
