> On 九月 12, 2016, 8:46 p.m., Benjamin Mahler wrote: > > src/master/allocator/mesos/hierarchical.cpp, lines 273-274 > > <https://reviews.apache.org/r/51027/diff/4/?file=1490734#file1490734line273> > > > > It seems a bit odd that the caller has to both touch allocation > > candidates and then call ensureAllocation. > > > > A simpler way to think about this may be that `allocate` has changed > > from a synchronous function to an asynchronous one. I.e. the call-site here > > would be: > > > > ``` > > void allocate(...); // becomes: > > Future<Nothing> allocate(...); > > > > // Call-site: > > Future<Nothing> allocated = allocate(slave.keys()); > > > > // Actual allocation logic is a continuation now: > > Nothing _allocate(...); > > ``` > > > > (We probably do not need the Future just yet since callers don't need > > to set up continuations for now, however we might as well add it to express > > the asynchronous nature of the function.) > > > > This also avoids the need for the callers to touch the data structure > > correctly, which seems error-prone. They just call `allocate` and it deals > > with adding the SlaveIDs that need allocations performed.
The reason that `the caller touch both allocation candidates and then call ensureAllocation` is because the caller need to handle different allocation event. 1) For some allocation event, we need to use `allocationCandidates = slaves.keys();` such as `addFramework` etc, as we want to get all free resources from the resource pool and allocate to the new framework. 2) For some allocation event, such as `addSlave`, there is no need to use `slaves.keys()` but only adding the new added agent to the `allocationCandidates` is good enough, this can reduce the time of the allocation cycle: Allocating only one agent is fast than allocating all agent in an allocation cycle if there are bunch of agents. 3) The `allocationCandidates` needs to be cleaned after each allocation cycle. - Guangya ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51027/#review148579 ----------------------------------------------------------- On 九月 3, 2016, 6:05 a.m., Jacob Janco wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51027/ > ----------------------------------------------------------- > > (Updated 九月 3, 2016, 6:05 a.m.) > > > Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus > Ma, and Jiang Yan Xu. > > > Bugs: MESOS-3157 > https://issues.apache.org/jira/browse/MESOS-3157 > > > Repository: mesos > > > Description > ------- > > - Triggered allocations dispatch allocate() only > if there is no pending allocation in the queue. > - Allocation candidates are accumulated and only > cleared when enqueued allocations are processed. > - Batched allocations are handled synchronously. > > > Diffs > ----- > > src/master/allocator/mesos/hierarchical.hpp > dd07ed221d2c1755d2478369641ffdc46ecc4471 > src/master/allocator/mesos/hierarchical.cpp > 9e5db2196c6a541dc1208ba8b9f13ef9a518bcc4 > > Diff: https://reviews.apache.org/r/51027/diff/ > > > Testing > ------- > > make check > > note: check without filters depends on https://reviews.apache.org/r/51028 > > With new benchmark https://reviews.apache.org/r/49617: > Sample output without 51027: > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 > Using 10000 agents and 3000 frameworks > Added 3000 frameworks in 57251us > Added 10000 agents in 3.21345353333333mins > allocator settled after 1.61236038333333mins > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 > (290578 ms) > > Sample output with 51027: > [ RUN ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 > Using 10000 agents and 3000 frameworks > Added 3000 frameworks in 39817us > Added 10000 agents in 3.22860541666667mins > allocator settled after 25.525654secs > [ OK ] > SlaveAndFrameworkCount/HierarchicalAllocator_BENCHMARK_Test.FrameworkFailover/22 > (220137 ms) > > > Thanks, > > Jacob Janco > >
