[
https://issues.apache.org/jira/browse/MESOS-336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13974167#comment-13974167
]
Bernd Mathiske commented on MESOS-336:
--------------------------------------
Good idea to encode the checksum in the dirs! I am keen on using this scheme
whether we deal with concurrency in the fetcher program or in the slave
program. But I have to admit that I am leaning towards the latter. Looking up a
given cache file is easier to handle in one coherent process (the slave).
Instead of searching on disk, a simple lookup in hashmap yields the whole cache
file path, including dir and thus checksum.
> Mesos slave should cache executors
> ----------------------------------
>
> Key: MESOS-336
> URL: https://issues.apache.org/jira/browse/MESOS-336
> Project: Mesos
> Issue Type: Improvement
> Components: slave
> Reporter: brian wickman
> Assignee: Bernd Mathiske
> Labels: newbie
>
> The slave should be smarter about how it handles pulling down executors. In
> our environment, executors rarely change but the slave will always pull it
> down from regardless HDFS. This puts undue stress on our HDFS clusters, and
> is not resilient to reduced HDFS availability.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
