+ben and vinod for more context This review fixes the issue related to aggregating 'Resources' across slaves: https://issues.apache.org/jira/browse/MESOS-2373
Today, this only works for scalars inside 'Resources' objects. For sets (and ranges, which are sets), 'Resources' will merge these as it doesn't have an understanding of the namespacing of resources. For example ([31000-32000]) + ([31000-32000]) = ([31000-32000]), not ([31000-32000], [31000-32000]). There's kind of two approaches here: (1) Accept that all 'Resources' objects are implicitly namespaced (e.g. by SlaveID), and they cannot be merged across namespaces. For example, you can't sum the ports across slaves because it will try to merge them together. e.g. [31000-32000] + [31000-32000] = [31000-32000], which leads to information loss. This means we have to update all 'Resources' aggregation to deal with the namespacing. Often this means 'Resources' becomes 'hashmap<SlaveID, Resources>'. (2) But (1) seems to go against the original intent of the code, where there seems to be an assumption that aggregating 'Resources' across slaves is safe. I was leaning towards (1) where we add explicit namespacing to our aggregated Resources. But that seems a bit clunky. Ben and Vinod, do you guys have more context on this? Was the intent to make 'Resources' correctly handle aggregation across slaves? Or was it an oversight that 'Resources' cannot be aggregated across slaves? ---------- Forwarded message ---------- From: Michael Park <mcyp...@gmail.com> Date: Mon, Feb 23, 2015 at 10:14 AM Subject: Re: Review Request 31183: Distinguished resources from different slaves in DRFSorter. To: Alexander Rukletsov <a...@mesosphere.io>, Benjamin Hindman < b...@berkeley.edu>, Ben Mahler <benjamin.mah...@gmail.com>, Jie Yu < yujie....@gmail.com> Cc: Mesos ReviewBot <dev@mesos.apache.org>, Till Toenshoff <toensh...@me.com>, Michael Park <mcyp...@gmail.com>, Alexander Rojas <alexan...@mesosphere.io> ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31183/ ----------------------------------------------------------- (Updated Feb. 23, 2015, 6:14 p.m.) Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu. Changes ------- s/Distinguish/Distinguished Summary (updated) ----------------- Distinguished resources from different slaves in DRFSorter. Bugs: MESOS-2373 https://issues.apache.org/jira/browse/MESOS-2373 Repository: mesos Description ------- Distinguish resources from different slaves in DRFSorter. Diffs ----- src/master/allocator/allocator.hpp c2461a32e18dbafe637f37180f86bfa2e1d78735 src/master/allocator/mesos/allocator.hpp 99b40a01749b77fc7c1fa025a7ac81d5cca6272f src/master/allocator/mesos/hierarchical.hpp 2680d6231927867d5a8d75cbc42b81d6c75fc7f2 src/master/allocator/sorter/drf/sorter.hpp 966fe03de389246dc71e933f9a51ad5b55f2834d src/master/allocator/sorter/drf/sorter.cpp 7baee453848c7587472b2e9358c9e4ee6df6e860 src/master/allocator/sorter/sorter.hpp 8cd70dd91f72109222d8a0cd67cbf66ee4188ab5 src/master/http.cpp 5e3afdf8f32a7bf146eb39e53d1543c92e6180ba src/master/master.hpp 6a39df04514c756415354fae66c5835ada191c52 src/master/master.cpp f10a3cffdb0b9831b2b8af0dfd30f73d1d8ed937 src/tests/hierarchical_allocator_tests.cpp eeecfb64540b16666915074aaffaa5d506b203bc src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124 src/tests/sorter_tests.cpp 7ba32b78bdedf754f76f0695dfd1d86aa1f1b5bf Diff: https://reviews.apache.org/r/31183/diff/ Testing ------- make check Thanks, Michael Park
