[
https://issues.apache.org/jira/browse/MESOS-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bernd Mathiske updated MESOS-2069:
----------------------------------
Description:
Cache files downloaded by the fetcher in a repository. If multiple requests for
the same URI occur, perform only one of them and reuse the result. Make
concurrent requests for the same URI wait for the one download. Do support
multiple concurrent downloads of different URIs, though.
Note that implementing this does not suffice for production use. This ticket
contains the main part of the fetcher logic, though. See the epic MESOS-336 for
the rest of the features that lead to a fully functional fetcher cache.
The proposed general approach is to keep all bookkeeping about what is in which
stage of being fetched and where it resides in the slave's
MesosContainerizerProcess, so that all concurrent access is disambiguated and
controlled by an "actor" (aka libprocess "process").
Also see MESOS-336 for the overall goals for the fetcher cache.
was:Clean the fetcher cache completely upon slave restart/recovery. This
implements correct, albeit not ideal behavior. More efficient schemes that
restore knowledge about cached files or even resume downloads can be added
later.
Labels: fetcher slave (was: newbie)
Remaining Estimate: 168h (was: 6h)
Original Estimate: 168h (was: 6h)
Story Points: 2 (was: 0.2)
Summary: Basic fetcher cache functionality (was: Implement
simple slave recovery behavior for fetcher cache)
> Basic fetcher cache functionality
> ---------------------------------
>
> Key: MESOS-2069
> URL: https://issues.apache.org/jira/browse/MESOS-2069
> Project: Mesos
> Issue Type: Improvement
> Components: fetcher, slave
> Reporter: Bernd Mathiske
> Assignee: Bernd Mathiske
> Labels: fetcher, slave
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Cache files downloaded by the fetcher in a repository. If multiple requests
> for the same URI occur, perform only one of them and reuse the result. Make
> concurrent requests for the same URI wait for the one download. Do support
> multiple concurrent downloads of different URIs, though.
> Note that implementing this does not suffice for production use. This ticket
> contains the main part of the fetcher logic, though. See the epic MESOS-336
> for the rest of the features that lead to a fully functional fetcher cache.
> The proposed general approach is to keep all bookkeeping about what is in
> which stage of being fetched and where it resides in the slave's
> MesosContainerizerProcess, so that all concurrent access is disambiguated and
> controlled by an "actor" (aka libprocess "process").
> Also see MESOS-336 for the overall goals for the fetcher cache.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
