-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21316/
-----------------------------------------------------------
(Updated May 19, 2014, 7:03 a.m.)
Review request for mesos and Benjamin Hindman.
Changes
-------
Not yet ready, but soon. Improved version with some tests - finally! These
tests "mostly" run, some checks fail, need to be debugged, nothing complicated
there. Further tests will be written to check:
- fetching from HTTP
- concurrent fetching and GC of cache files
- various corner cases, i.e. suppressed archive uncompressing
Bugs: MESOS-336
https://issues.apache.org/jira/browse/MESOS-336
Repository: mesos-git
Description
-------
Preview of the first cut at fetcher caching. See MESOS-336 JIRA for explanation
for this approach: keep the cache info in the MesosContainerizerProcess in the
save, leverage actor single-threadedness to deal with concurrency issues
without head ache.
Features so far:
- If URI flag "fetched_externally" (default: false) is set, the fetcher does
what it did in Mesos 0.18 and before.
- If URI flag "cached" (default: false) is not set, the fetcher also fetches
every time as in Mesos 0.18 and before.
- If URI flag "cached" is set, the UIR is only fetched once and all subsequent
fetch attempts copy from the cache file.
- URIs are cached separately per framework (ID).
- Recovery is implemented by simply wiping the entire cache.
- GC for cache files. Global flag sets lifetime after last use. Default is 1
hour.
Potential future features:
- symlinks instead of copying
- extraction directly from URI, without cache file
- combine that with symlinks
- Refreshing, explicit cache invalidation
- ...
Diffs (updated)
-----
include/mesos/mesos.proto 8012873
src/Makefile.am 812ad2c
src/launcher/fetcher.cpp 8c9e20d
src/local/local.cpp 5d26aff
src/slave/constants.hpp c097525
src/slave/constants.cpp 1854b16
src/slave/containerizer/mesos_containerizer.hpp 1f5908a
src/slave/containerizer/mesos_containerizer.cpp 2a4816e
src/slave/containerizer/mesos_fetcher.hpp PRE-CREATION
src/slave/containerizer/mesos_fetcher.cpp PRE-CREATION
src/slave/flags.hpp 0a04ad5
src/slave/slave.hpp a6efad4
src/slave/slave.cpp 3a4ae38
src/tests/containerizer_tests.cpp 9d20853
src/tests/fetcher_tests.cpp PRE-CREATION
src/tests/slave_tests.cpp 458356d
src/tests/utils.hpp 3b80933
Diff: https://reviews.apache.org/r/21316/diff/
Testing
-------
Tests need to be written, i.e. this is not commit-ready, just a preview that
shows everybody how it works. Tested with Mesosaurus, with and without caching
switched on. Seems to work for the easy cases, have not tested all corner cases
yet (e.g. extraction on/off, intermittent gc, ...).
Thanks,
Bernd Mathiske
