[
https://issues.apache.org/jira/browse/MESOS-2057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Niklas Quarfot Nielsen updated MESOS-2057:
------------------------------------------
Target Version/s: 0.23.0 (was: 0.22.0)
> Concurrency control for fetcher cache
> -------------------------------------
>
> Key: MESOS-2057
> URL: https://issues.apache.org/jira/browse/MESOS-2057
> Project: Mesos
> Issue Type: Improvement
> Components: fetcher, slave
> Reporter: Bernd Mathiske
> Assignee: Bernd Mathiske
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> Having added a URI flag to CommandInfo messages (in MESOS-2069) that
> indicates caching, caching files downloaded by the fetcher in a repository,
> now ensure that when a URI is "cached", it is only ever downloaded once for
> the same user on the same slave as long as the slave keeps running.
> This even holds if multiple tasks request the same URI concurrently. If
> multiple requests for the same URI occur, perform only one of them and reuse
> the result. Make concurrent requests for the same URI wait for the one
> download.
> Different URIs from different CommandInfos can be downloaded concurrently.
> No cache eviction, cleanup or failover will be handled for now. Additional
> tickets will be filed for these enhancements. (So don't use this feature in
> production until the whole epic is complete.)
> Note that implementing this does not suffice for production use. This ticket
> contains the main part of the fetcher logic, though. See the epic MESOS-336
> for the rest of the features that lead to a fully functional fetcher cache.
> The proposed general approach is to keep all bookkeeping about what is in
> which stage of being fetched and where it resides in the slave's
> MesosContainerizerProcess, so that all concurrent access is disambiguated and
> controlled by an "actor" (aka libprocess "process").
> Depends on MESOS-2056 and MESOS-2069.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
