Review Request 34306: Added 'revocable_offers' field to FrameworkInfo.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34306/ --- Review request for mesos, Jie Yu and Niklas Nielsen. Bugs: MESOS-2654 https://issues.apache.org/jira/browse/MESOS-2654 Repository: mesos Description --- Currently setting this field is a no-op. Just wanted to send it out for feedback, esp whether a 'bool' type is enough. Diffs - include/mesos/mesos.proto 9cc5782256156ed59fd4640091413b76480d939f Diff: https://reviews.apache.org/r/34306/diff/ Testing --- Thanks, Vinod Kone
Review Request 34304: Updated documentation of FrameworkInfo.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34304/ --- Review request for mesos, Ben Mahler and Jie Yu. Repository: mesos Description --- Moved the documenation close to the fields for clarity. No functional changes. Diffs - include/mesos/mesos.proto 9cc5782256156ed59fd4640091413b76480d939f Diff: https://reviews.apache.org/r/34304/diff/ Testing --- Thanks, Vinod Kone
Review Request 34309: Support manipulating scheduler policy on Linux.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34309/ --- Review request for mesos, Joris Van Remoortere, Niklas Nielsen, and Vinod Kone. Bugs: MESOS-2652 https://issues.apache.org/jira/browse/MESOS-2652 Repository: mesos Description --- Support manipulating scheduler policy on Linux. Diffs - src/Makefile.am 34755cf795391c9b8051a5e4acc6caf844984496 src/linux/sched.hpp PRE-CREATION src/tests/sched_tests.cpp PRE-CREATION Diff: https://reviews.apache.org/r/34309/diff/ Testing --- Added test. Thanks, Ian Downes
Review Request 34310: Use IDLE scheduling for revocable CPU in cgroups isolator.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34310/ --- Review request for mesos, Joris Van Remoortere, Niklas Nielsen, and Vinod Kone. Bugs: MESOS-2652 https://issues.apache.org/jira/browse/MESOS-2652 Repository: mesos Description --- Use IDLE scheduling for revocable CPU in cgroups isolator. Diffs - src/slave/containerizer/isolators/cgroups/cpushare.hpp ff4a9dbdb1b655e71bf87dcee8fe62433d396f52 src/slave/containerizer/isolators/cgroups/cpushare.cpp 6a5b2b5c6e2844fe1a10815956569194b6f56681 Diff: https://reviews.apache.org/r/34310/diff/ Testing --- Thanks, Ian Downes
Re: Review Request 34299: Changed to use a push model for resource estimator.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34299/#review84003 --- Looks good. My main comments are around naming. include/mesos/slave/resource_estimator.hpp https://reviews.apache.org/r/34299/#comment135109 s/estimation about/estimation of/ include/mesos/slave/resource_estimator.hpp https://reviews.apache.org/r/34299/#comment135110 s/oversubscribed/oversubscribable/ ? or s/oversubscribed/oversubscribe/ because these are not resources that are already oversubscribed? src/slave/slave.hpp https://reviews.apache.org/r/34299/#comment135118 I would actually s/receive/update/ s/update/send/ src/slave/slave.hpp https://reviews.apache.org/r/34299/#comment135120 Make this optional, so that slave will only send an update after the first estimate is received? src/slave/slave.cpp https://reviews.apache.org/r/34299/#comment135124 s/update/send/ src/slave/slave.cpp https://reviews.apache.org/r/34299/#comment135123 LOG(INFO) Sending oversubscribed resources oversubscribedResources; src/slave/slave.cpp https://reviews.apache.org/r/34299/#comment135127 s/Update// src/slave/slave.cpp https://reviews.apache.org/r/34299/#comment135128 s/update_// src/tests/oversubscription_tests.cpp https://reviews.apache.org/r/34299/#comment135130 s/UpdateOversubcribedResourcesMessage/SendOversubscribedResources/ src/tests/oversubscription_tests.cpp https://reviews.apache.org/r/34299/#comment135131 s/estimation/estimation of oversubscribed resources/ - Vinod Kone On May 15, 2015, 11:38 p.m., Jie Yu wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34299/ --- (Updated May 15, 2015, 11:38 p.m.) Review request for mesos, Joris Van Remoortere, Niklas Nielsen, and Vinod Kone. Bugs: MESOS-2735 https://issues.apache.org/jira/browse/MESOS-2735 Repository: mesos Description --- Changed to use a push model for resource estimator. See ticket for details. This patch also makes the interval configurable through slave flags. Diffs - include/mesos/slave/resource_estimator.hpp 363961541c9b49763e8cbafe982421d45516db0d src/slave/constants.hpp df02043fcd1b5bffc13aa0683ad9b64372188300 src/slave/constants.cpp 07f699a9e5c1b56d6e0435077823ad0f863ad63c src/slave/flags.hpp ca7cc131f70d58982262fd29986d46877d563035 src/slave/flags.cpp f35c76a342d03700710bb91bf4c523cc99769228 src/slave/resource_estimator.hpp bdf62bab9b632aa16801e47978962a7ab24b53b4 src/slave/resource_estimator.cpp 13d706c30d36c9f304acf288f02d6c9558bac781 src/slave/slave.hpp b62ed7b207fa337027baffa1c75139641a41dad6 src/slave/slave.cpp 132f83e05dd4113c8f6d20a6a64b12fb8f05dd36 src/tests/mesos.hpp df8cd200a6970209588b63a1d58709b1f913a305 src/tests/oversubscription_tests.cpp 64c2ede1f50546f6de1ecb2972036b51fe17e358 Diff: https://reviews.apache.org/r/34299/diff/ Testing --- make check Thanks, Jie Yu
Review Request 34298: Explictly validated that all resources specified in reserve or unreserve operations are dynamically reserved.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34298/ --- Review request for mesos, Jie Yu and switched to 'mcypark'. Repository: mesos Description --- Did this for consistency with validateDiskInfo(). Diffs - src/master/validation.cpp c3e96ae0e684f3f365e9aa365bccc953d32b0452 Diff: https://reviews.apache.org/r/34298/diff/ Testing --- make check Thanks, Vinod Kone
Review Request 34299: Changed to use a push model for resource estimator.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34299/ --- Review request for mesos, Joris Van Remoortere, Niklas Nielsen, and Vinod Kone. Bugs: MESOS-2735 https://issues.apache.org/jira/browse/MESOS-2735 Repository: mesos Description --- Changed to use a push model for resource estimator. See ticket for details. This patch also makes the interval configurable through slave flags. Diffs - include/mesos/slave/resource_estimator.hpp 363961541c9b49763e8cbafe982421d45516db0d src/slave/constants.hpp df02043fcd1b5bffc13aa0683ad9b64372188300 src/slave/constants.cpp 07f699a9e5c1b56d6e0435077823ad0f863ad63c src/slave/flags.hpp ca7cc131f70d58982262fd29986d46877d563035 src/slave/flags.cpp f35c76a342d03700710bb91bf4c523cc99769228 src/slave/resource_estimator.hpp bdf62bab9b632aa16801e47978962a7ab24b53b4 src/slave/resource_estimator.cpp 13d706c30d36c9f304acf288f02d6c9558bac781 src/slave/slave.hpp b62ed7b207fa337027baffa1c75139641a41dad6 src/slave/slave.cpp 132f83e05dd4113c8f6d20a6a64b12fb8f05dd36 src/tests/mesos.hpp df8cd200a6970209588b63a1d58709b1f913a305 src/tests/oversubscription_tests.cpp 64c2ede1f50546f6de1ecb2972036b51fe17e358 Diff: https://reviews.apache.org/r/34299/diff/ Testing --- make check Thanks, Jie Yu
Review Request 34308: Filter revocable resources.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34308/ --- Review request for mesos and Vinod Kone. Repository: mesos Description --- Filter revocable resources. Diffs - include/mesos/resources.hpp 1e98c13fe8075b14454f7899b98006fdaf88f484 src/common/resources.cpp 92b9e7f60323e0f7cf69c42e712468b631f3 Diff: https://reviews.apache.org/r/34308/diff/ Testing --- Thanks, Ian Downes
Re: Review Request 33865: Added RevocableInfo message to Resource protobuf.
On May 15, 2015, 6:09 p.m., Niklas Nielsen wrote: src/master/validation.cpp, line 90 https://reviews.apache.org/r/33865/diff/2/?file=960728#file960728line90 Do you want to stringify the resource for the error message here too? wanted to be consistent with the rest of the error messages here. i think we didn't originally stringify because it is always 'disk' resource. - Vinod --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33865/#review83954 --- On May 14, 2015, 11:57 p.m., Vinod Kone wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33865/ --- (Updated May 14, 2015, 11:57 p.m.) Review request for mesos, Jie Yu, Joris Van Remoortere, and Niklas Nielsen. Bugs: MESOS-2691 https://issues.apache.org/jira/browse/MESOS-2691 Repository: mesos Description --- RevocableInfo currently supports OVERSUBSCRIBED resources. In the future it can be easily extended to use other types of revocable reosurces (e.g., resources allocated to other roles). Disabled the ability to use revocable resources for reservation or persistence because the semantics seem weird. We can enable it in the future if there is a use case for that. Diffs - include/mesos/mesos.proto 15f55a3c54be3475e77d561f106e00ea6e53c2fa src/common/resources.cpp 843a06d6c4d3e9ff0d1665360bae7c57bcfecb83 src/master/validation.cpp c3e96ae0e684f3f365e9aa365bccc953d32b0452 src/tests/resources_tests.cpp a7ec59ea217ad71f7d1e93ca6039d5b2491b3237 Diff: https://reviews.apache.org/r/33865/diff/ Testing --- make check Thanks, Vinod Kone
Re: Review Request 33865: Added RevocableInfo message to Resource protobuf.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33865/ --- (Updated May 15, 2015, 11:28 p.m.) Review request for mesos, Jie Yu, Joris Van Remoortere, and Niklas Nielsen. Changes --- jie's and nik's comments. PTAL. Bugs: MESOS-2691 https://issues.apache.org/jira/browse/MESOS-2691 Repository: mesos Description --- RevocableInfo currently supports OVERSUBSCRIBED resources. In the future it can be easily extended to use other types of revocable reosurces (e.g., resources allocated to other roles). Disabled the ability to use revocable resources for reservation or persistence because the semantics seem weird. We can enable it in the future if there is a use case for that. Diffs (updated) - include/mesos/mesos.proto 15f55a3c54be3475e77d561f106e00ea6e53c2fa src/common/resources.cpp 843a06d6c4d3e9ff0d1665360bae7c57bcfecb83 src/master/validation.cpp c3e96ae0e684f3f365e9aa365bccc953d32b0452 src/tests/master_validation_tests.cpp 9a6f4fa464d6daca088c8634e73e04c5fd380677 src/tests/resources_tests.cpp a7ec59ea217ad71f7d1e93ca6039d5b2491b3237 Diff: https://reviews.apache.org/r/33865/diff/ Testing --- make check Thanks, Vinod Kone
Re: Review Request 34295: Added maintainers documentation.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34295/#review84004 --- Ship it! Thanks Ben! LGTM. - Jie Yu On May 15, 2015, 10:25 p.m., Ben Mahler wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34295/ --- (Updated May 15, 2015, 10:25 p.m.) Review request for mesos, Adam B, Benjamin Hindman, Dave Lester, Ian Downes, Jie Yu, Niklas Nielsen, Till Toenshoff, Timothy Chen, Vinod Kone, and Jiang Yan Xu. Bugs: MESOS-2737 https://issues.apache.org/jira/browse/MESOS-2737 Repository: mesos Description --- Per the proposal thread, this adds documentation for maintainers and an initial set of maintainers. Please share your thoughts! Diffs - docs/committers.md PRE-CREATION docs/home.md 4b8e7f8d2dcd9e26816353c939a2a5c18314ac45 Diff: https://reviews.apache.org/r/34295/diff/ Testing --- Markdown: https://gist.github.com/bmahler/962d1ddc9f14e1d342a6 Thanks, Ben Mahler
Re: Review Request 34299: Changed to use a push model for resource estimator.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34299/#review84011 --- src/slave/slave.cpp https://reviews.apache.org/r/34299/#comment135135 This is being executed in the context of the estimator thread? Is this safe? - Niklas Nielsen On May 15, 2015, 4:38 p.m., Jie Yu wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34299/ --- (Updated May 15, 2015, 4:38 p.m.) Review request for mesos, Joris Van Remoortere, Niklas Nielsen, and Vinod Kone. Bugs: MESOS-2735 https://issues.apache.org/jira/browse/MESOS-2735 Repository: mesos Description --- Changed to use a push model for resource estimator. See ticket for details. This patch also makes the interval configurable through slave flags. Diffs - include/mesos/slave/resource_estimator.hpp 363961541c9b49763e8cbafe982421d45516db0d src/slave/constants.hpp df02043fcd1b5bffc13aa0683ad9b64372188300 src/slave/constants.cpp 07f699a9e5c1b56d6e0435077823ad0f863ad63c src/slave/flags.hpp ca7cc131f70d58982262fd29986d46877d563035 src/slave/flags.cpp f35c76a342d03700710bb91bf4c523cc99769228 src/slave/resource_estimator.hpp bdf62bab9b632aa16801e47978962a7ab24b53b4 src/slave/resource_estimator.cpp 13d706c30d36c9f304acf288f02d6c9558bac781 src/slave/slave.hpp b62ed7b207fa337027baffa1c75139641a41dad6 src/slave/slave.cpp 132f83e05dd4113c8f6d20a6a64b12fb8f05dd36 src/tests/mesos.hpp df8cd200a6970209588b63a1d58709b1f913a305 src/tests/oversubscription_tests.cpp 64c2ede1f50546f6de1ecb2972036b51fe17e358 Diff: https://reviews.apache.org/r/34299/diff/ Testing --- make check Thanks, Jie Yu
Re: Review Request 34306: Added 'revocable_offers' field to FrameworkInfo.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34306/#review84016 --- Patch looks great! Reviews applied: [34304, 34306] All tests passed. - Mesos ReviewBot On May 16, 2015, 12:24 a.m., Vinod Kone wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34306/ --- (Updated May 16, 2015, 12:24 a.m.) Review request for mesos, Jie Yu and Niklas Nielsen. Bugs: MESOS-2654 https://issues.apache.org/jira/browse/MESOS-2654 Repository: mesos Description --- Currently setting this field is a no-op. Just wanted to send it out for feedback, esp whether a 'bool' type is enough. Diffs - include/mesos/mesos.proto 9cc5782256156ed59fd4640091413b76480d939f Diff: https://reviews.apache.org/r/34306/diff/ Testing --- Thanks, Vinod Kone
Re: Review Request 34310: Use IDLE scheduling for revocable CPU in cgroups isolator.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34310/#review84024 --- src/slave/containerizer/isolators/cgroups/cpushare.cpp https://reviews.apache.org/r/34310/#comment135147 What if the same set of resources contains both revocable and non-revocable resources? - Timothy Chen On May 16, 2015, 1:14 a.m., Ian Downes wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34310/ --- (Updated May 16, 2015, 1:14 a.m.) Review request for mesos, Joris Van Remoortere, Niklas Nielsen, and Vinod Kone. Bugs: MESOS-2652 https://issues.apache.org/jira/browse/MESOS-2652 Repository: mesos Description --- Use IDLE scheduling for revocable CPU in cgroups isolator. Diffs - src/slave/containerizer/isolators/cgroups/cpushare.hpp ff4a9dbdb1b655e71bf87dcee8fe62433d396f52 src/slave/containerizer/isolators/cgroups/cpushare.cpp 6a5b2b5c6e2844fe1a10815956569194b6f56681 Diff: https://reviews.apache.org/r/34310/diff/ Testing --- Thanks, Ian Downes
Re: Review Request 30774: Fetcher Cache
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30774/#review83944 --- src/tests/fetcher_cache_tests.cpp https://reviews.apache.org/r/30774/#comment135003 AWAIT_READY, not AWAIT_READY_FOR, coz the default is 15 and that works fine, too. - Bernd Mathiske On May 13, 2015, 3:07 p.m., Bernd Mathiske wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30774/ --- (Updated May 13, 2015, 3:07 p.m.) Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen. Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074 https://issues.apache.org/jira/browse/MESOS-2057 https://issues.apache.org/jira/browse/MESOS-2069 https://issues.apache.org/jira/browse/MESOS-2070 https://issues.apache.org/jira/browse/MESOS-2072 https://issues.apache.org/jira/browse/MESOS-2073 https://issues.apache.org/jira/browse/MESOS-2074 Repository: mesos Description --- Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features. Diffs - docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b docs/fetcher-cache-internals.md PRE-CREATION docs/fetcher.md PRE-CREATION include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc include/mesos/mesos.proto 967b1e3bbfb3f6b71d5a15d02cba7ed5ec21816f include/mesos/type_utils.hpp 044637481e5405d4d6f61653a9f9386edd191deb src/Makefile.am 93c7c8a807a33ab639be6289535bbd32022aa85b src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec src/slave/containerizer/docker.hpp b25ec55bf3cd30d6e8a804d09d90c632a7d12e3f src/slave/containerizer/docker.cpp f9fc89ad7e3c853c3f9f6dcf9aa68e54dc1888c6 src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 src/slave/containerizer/mesos/containerizer.hpp 5e5f13ed8a71ff9510b40b6032d00fd16d312622 src/slave/containerizer/mesos/containerizer.cpp f2587280dc0e1d566d2b856a80358c7b3896c603 src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 src/slave/flags.cpp d0932b04e3825abb6173efe0d1aee199aa356932 src/slave/slave.cpp c78ee3c9e7fc38ad364e83f4abe267e86bfbbc13 src/tests/docker_containerizer_tests.cpp c9d66b3fbc7d081f36c26781573dca50de823c44 src/tests/fetcher_cache_tests.cpp PRE-CREATION src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e src/tests/mesos.hpp 19db71217f0a3f1ab17a6fd4408f8251410d731d src/tests/mesos.cpp bc082e8d91deb2c5dd64bbc3f0a8a50fa7d19264 Diff: https://reviews.apache.org/r/30774/diff/ Testing --- make check --- longer Description: --- -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order: 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway. 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.) 30036: Extends the CommandInfo::URI protobuf with a boolean caching field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where
Re: Review Request 30774: Fetcher Cache
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30774/#review83946 --- - Bernd Mathiske On May 13, 2015, 3:07 p.m., Bernd Mathiske wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30774/ --- (Updated May 13, 2015, 3:07 p.m.) Review request for mesos, Adam B, Benjamin Hindman, Till Toenshoff, and Timothy Chen. Bugs: MESOS-2057, MESOS-2069, MESOS-2070, MESOS-2072, MESOS-2073, and MESOS-2074 https://issues.apache.org/jira/browse/MESOS-2057 https://issues.apache.org/jira/browse/MESOS-2069 https://issues.apache.org/jira/browse/MESOS-2070 https://issues.apache.org/jira/browse/MESOS-2072 https://issues.apache.org/jira/browse/MESOS-2073 https://issues.apache.org/jira/browse/MESOS-2074 Repository: mesos Description --- Almost all of the functionality in epic MESOS-336. Downloaded files from CommandInfo::URIs can now be cached in a cache directory designated by a slave flag. This only happens when asked for by an extra flag in the URI and is thus backwards-compatible. The cache has a size limit also given by a new slave flag. Cache-resident files are evicted as necessary to make space for newly fetched ones. Concurrent attempts to cache the same URI leads to only one download. The fetcher program remains external for safety reasons, but is now augmented with more elaborate parameters packed into a JSON object to implement specific fetch actions for all of the above. Additional testing includes fetching from (mock) HDFS and coverage of the new features. Diffs - docs/configuration.md 54c4e31ed6dfed3c23d492c19a301ce119a0519b docs/fetcher-cache-internals.md PRE-CREATION docs/fetcher.md PRE-CREATION include/mesos/fetcher/fetcher.proto 311af9aebc6a85dadba9dbeffcf7036b70896bcc include/mesos/mesos.proto 967b1e3bbfb3f6b71d5a15d02cba7ed5ec21816f include/mesos/type_utils.hpp 044637481e5405d4d6f61653a9f9386edd191deb src/Makefile.am 93c7c8a807a33ab639be6289535bbd32022aa85b src/hdfs/hdfs.hpp 968545d9af896f3e72e156484cc58135405cef6b src/launcher/fetcher.cpp 796526f59c25898ef6db2b828b0e2bb7b172ba25 src/slave/constants.hpp fd1c1aba0aa62372ab399bee5709ce81b8e92cec src/slave/containerizer/docker.hpp b25ec55bf3cd30d6e8a804d09d90c632a7d12e3f src/slave/containerizer/docker.cpp f9fc89ad7e3c853c3f9f6dcf9aa68e54dc1888c6 src/slave/containerizer/fetcher.hpp 1db0eaf002c8d0eaf4e0391858e61e0912b35829 src/slave/containerizer/fetcher.cpp 9e9e9d0eb6b0801d53dec3baea32a4cd4acdd5e2 src/slave/containerizer/mesos/containerizer.hpp 5e5f13ed8a71ff9510b40b6032d00fd16d312622 src/slave/containerizer/mesos/containerizer.cpp f2587280dc0e1d566d2b856a80358c7b3896c603 src/slave/flags.hpp d3b1ce117fbb4e0b97852ef150b63f35cc991032 src/slave/flags.cpp d0932b04e3825abb6173efe0d1aee199aa356932 src/slave/slave.cpp c78ee3c9e7fc38ad364e83f4abe267e86bfbbc13 src/tests/docker_containerizer_tests.cpp c9d66b3fbc7d081f36c26781573dca50de823c44 src/tests/fetcher_cache_tests.cpp PRE-CREATION src/tests/fetcher_tests.cpp 4549e6a631e2c17cec3766efaa556593eeac9a1e src/tests/mesos.hpp 19db71217f0a3f1ab17a6fd4408f8251410d731d src/tests/mesos.cpp bc082e8d91deb2c5dd64bbc3f0a8a50fa7d19264 Diff: https://reviews.apache.org/r/30774/diff/ Testing --- make check --- longer Description: --- -Replaces all other reviews for the fetcher cache except those related to stout: 30006, 30033, 30034, 30036, 30037, 30039, 30124, 30173, 30614, 30616, 30618, 30621, 30626. See descriptions of those. In dependency order: 30033: Removes the fetcher env tests since these won't be needed any more when the fetcher uses JSON in a single env var as a parameter. They never tested anything that won't be covered by other tests anyway. 30034: Makes the code structure of all fetcher tests the same. Instead of calling the run method of the fetcher directly, calling through fetch(). Also removes all uses of I/O redirection, which is not really needed for debugging, and thus the next patch can refactor fetch() and run(). (The latter comes in two varieties, which complicates matters without much benefit.) 30036: Extends the CommandInfo::URI protobuf with a boolean caching field that will later cause fetcher cache actions. Also introduces the notion of a cache directory to the fetcher info protobuf. And then propagates these additions throughout the rest of the code base where applicable. This includes passing the slave ID all the way down to the place where the cache dir name is constructed. 30037: Extends the fetcher info protobuf with
