----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35702/ -----------------------------------------------------------
(Updated Sept. 4, 2015, 11:19 p.m.) Review request for mesos, Adam B, Benjamin Hindman, Ben Mahler, Jie Yu, Joris Van Remoortere, and Vinod Kone. Changes ------- Rebased. NNFR. Bugs: MESOS-2600 https://issues.apache.org/jira/browse/MESOS-2600 Repository: mesos Description ------- This involved a lot more challenges than I anticipated, I've captured the various approaches and limitations and deal-breakers of those approaches here: [Master Endpoint Implementation Challenges](https://docs.google.com/document/d/1cwVz4aKiCYP9Y4MOwHYZkyaiuEv7fArCye-vPvB2lAI/edit#) Key points: * This is a stop-gap solution until we shift the offer creation/management logic from the master to the allocator. * `updateAvailable` and `updateSlave` are kept separate because (1) `updateAvailable` is allowed to fail whereas `updateSlave` must not. (2) `updateAvailable` returns a `Future` whereas `updateSlave` does not. (3) `updateAvailable` never leaves the allocator in an over-allocated state and must not, whereas `updateSlave` does, and can. * The algorithm: * Initially, the master pessimistically assume that what seems like "available" resources will be gone. This is due to the race between the allocator scheduling an `allocate` call to itself vs master's `allocator->updateAvailable` invocation. As such, we first try to satisfy the request only with the offered resources. * We greedily rescind one offer at a time until we've rescinded sufficiently many offers. IMPORTANT: We perform `recoverResources(..., Filters())` which has a default `refuse_sec` of 5 seconds, rather than `recoverResources(..., None())` so that we can virtually always win the race against `allocate`. In the rare case that we do lose, no disaster occurs. We simply fail to satisfy the request. * If we still don't have enough resources after resciding all offers, be semi-optimistic and forward the request to the allocator since there may be available resources to satisfy the request. * If the allocator returns a failure, report the error to the user with `Conflict`. This approach is clearly not ideal, since we would prefer to rescind as little offers as possible. Diffs (updated) ----- src/master/http.cpp 94e97a2898106579434e8cdec04b7b0e130a810e src/master/master.hpp e1331851c19e3372a4a525dcfd7ba2a01c3e97a6 src/master/master.cpp 5589eca4317b597de509f3387cfc349083b361ac src/master/validation.hpp 43b8d84556e7f0a891dddf6185bbce7ca50b360a src/master/validation.cpp ffb7bf07b8a40d6e14f922eabcf46045462498b5 Diff: https://reviews.apache.org/r/35702/diff/ Testing ------- `make check` Thanks, Michael Park