> On March 17, 2017, 1:33 a.m., Benjamin Mahler wrote: > > Hm.. this seems to introduce a performance regression in the case where > > there are many inactive roles in the system? I assume it's difficult to > > avoid in the hierarchical case? Otherwise, this looks good. > > Neil Conway wrote: > True -- if there are many inactive clients, we'll now calculate their > share and include them in the `std::set`, whereas we wouldn't do so before > (conversely, activating and deactivating clients is a lot faster than it used > to be). My guess is that the bottlenecks in sorter performance likely lie > elsewhere (e.g., updating resources, generating the entire `vector` in > `sort()` when the allocator might only need the first few entries). > > We could avoid this overhead by skipping `calculateShare` for inactive > clients, and then dirtying the whole sorter when a client is activated. My > guess is that this isn't a net win (if you have a lot of inactive clients, it > seems unfortunate to dirty the entire sorter whenever a client becomes > active), but I can do some benchmarks if you think this case is important.
Another alternative to avoid dirtying the whole sorter when a client is activated is to a per-client dirty bit, or `Option<double> share` where if set, it is accurate (and we can unset to induce the lazy calculation). `dirty = true` would be equivalent to clearing all of the clients' shares. Thinking about it, the `Option<double>` seems simpler to understand as well (no way for a stale value to be there). - Benjamin ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/57564/#review169245 ----------------------------------------------------------- On March 13, 2017, 6:04 p.m., Neil Conway wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/57564/ > ----------------------------------------------------------- > > (Updated March 13, 2017, 6:04 p.m.) > > > Review request for mesos, Benjamin Bannier, Benjamin Mahler, and Michael Park. > > > Repository: mesos > > > Description > ------- > > DRFSorter previously removed inactive clients from the `clients` > collection, and then re-added clients when they were reactivated. This > resulted in resetting the allocation count for the client, which is > unfortunate. This scheme would also be more difficult to adapt to > hierarchical sorting. > > This commit changes DRFSorter to continue to store inactive clients in > the `clients`; inactive clients are indicated by a new field in the > `Client` struct, and are omitted from the return value of > `DRFSorter::sort`. > > > Diffs > ----- > > src/master/allocator/sorter/drf/sorter.hpp > 76329220e1115c1de7810fb69b943c78c078be59 > src/master/allocator/sorter/drf/sorter.cpp > ed54680cecb637931fc344fbcf8fd3b14cc24295 > src/tests/sorter_tests.cpp ec0636beb936d46a253d19322f2157abe95156b6 > > > Diff: https://reviews.apache.org/r/57564/diff/1/ > > > Testing > ------- > > `make check` > > > Thanks, > > Neil Conway > >
