[
https://issues.apache.org/jira/browse/MESOS-336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13968922#comment-13968922
]
Bernd Mathiske commented on MESOS-336:
--------------------------------------
I want what is in MESOS-700, too, as an option for the user. But that scheme
requires the framework to have a checksum ready and where does this checksum
come from? The framework programmer needs to have access to the
URI-downloadable resources and generate the checksum ahead of time. In
contrast, with the above scheme, this extra work is not necessary. Instead, it
is assumed that each URI is (typically) only ever used once and that subsequent
downloads would have exactly the same result. This should be a common scenario
for most frameworks.
Since neither variant is ideal for every scenario that may occur in the wild, I
suggest to have both, i.e. make the checksum optional as explained in the
comments to MESOS-700, and ALSO provide caching if no checksum is given. This
makes provisioning easier if no third party download site is involved.
An important use case for checksums might be upgrading executor software on the
fly. You can force just that by introducing a fresh checksum. But employing
this kind of trigger is only strictly necessary when the content of a third
party URI changes while a framework is running. In all cases where the
downloaded resources are under the control of the framework user, the framework
has alternative control mechanisms to deal with upgrades, e.g. using different
file names.
> Mesos slave should cache executors
> ----------------------------------
>
> Key: MESOS-336
> URL: https://issues.apache.org/jira/browse/MESOS-336
> Project: Mesos
> Issue Type: Improvement
> Components: slave
> Reporter: brian wickman
> Assignee: Bernd Mathiske
> Labels: newbie
>
> The slave should be smarter about how it handles pulling down executors. In
> our environment, executors rarely change but the slave will always pull it
> down from regardless HDFS. This puts undue stress on our HDFS clusters, and
> is not resilient to reduced HDFS availability.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
