-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5599/
-----------------------------------------------------------
(Updated July 11, 2012, 9:47 p.m.)
Review request for mesos and Benjamin Hindman.
Changes
-------
Bug fix.
Description (updated)
-------
Rewrote the DRF algorithm in DominantShareAllocator to calculate shares on a
per user, rather than per framework, basis and to store those shares in sorted
order so that allocations have log(n) time complexity instead of n^2.
Significant restructuring of the allocator code to make writing new allocators
easier:
- The Allocator class is renamed AllocatorProcess.
- A new Allocator class is introduced which knows only about "elements" (either
user names or frameworkIds) and how many resources they have been allocated,
and then has an interface for retrieving the elements in the order they should
be allocated to.
- SimpleAllocatorProcess which can be used with different Allocators to carry
out different allocation policies, including the possibility of having
different policies for the per-user and per-framework allocations.
Why is this restructuring a good thing?
Now that we're doing allocations based on users, rather than frameworks, we
have to do two levels of allocation - determine which user to give resources
to, then determine which of their frameworks to actually make the offer to.
Rather than hard-coding the allocation policies in, the new Allocator class
allows us to reuse the same code to do both levels of allocation, and it allows
us to easily swap different policies in or mix and match. You can even imagine
some day having an allocator that allows each user to pick the per-framework
allocation policy they prefer.
Merely giving SimpleAllocatorProcess different Allocators is not sufficient to
implement all allocation policies - only those where all resources are
considered equal and there is no revocation. But, you can always create new
AllocatorProcess subclasses (there will soon be a code review where I implement
a static allocator this way) and these new AllocatorProcess classes can still
reuse Allocators as appropriate.
It also makes testing the allocator logic a lot easier since we can instantiate
Allocators, give them arbitrary elements, and check that they return the right
thing, instead of worrying about instantiating slaves/frameworks and using mock
objects.
This addresses bugs MESOS-225 and MESOS-226.
https://issues.apache.org/jira/browse/MESOS-225
https://issues.apache.org/jira/browse/MESOS-226
Diffs (updated)
-----
src/Makefile.am eb1944f
src/local/local.hpp 55f9eaf
src/local/local.cpp 8186595
src/master/allocator.hpp 12f31db
src/master/allocator.cpp PRE-CREATION
src/master/allocator_process.hpp PRE-CREATION
src/master/dominant_share_allocator.hpp 650e441
src/master/dominant_share_allocator.cpp a1f2fdc
src/master/main.cpp c321618
src/master/master.hpp e523aa1
src/master/master.cpp f6e5459
src/master/simple_allocator_process.hpp PRE-CREATION
src/master/simple_allocator_process.cpp PRE-CREATION
src/tests/allocator_tests.cpp 610826b
src/tests/fault_tolerance_tests.cpp dd578aa
src/tests/master_detector_tests.cpp afd8a74
src/tests/master_tests.cpp 69a07e9
src/tests/resource_offers_tests.cpp d06cae2
src/tests/slave_tests.cpp a8d6b74
src/tests/utils.hpp ca51af5
Diff: https://reviews.apache.org/r/5599/diff/
Testing
-------
make check on Lion
Thanks,
Thomas Marshall
