[
https://issues.apache.org/jira/browse/MESOS-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15101557#comment-15101557
]
Adam B commented on MESOS-1887:
-------------------------------
cc: [~jieyu] who is working on a related update to the fetcher
> Allow to specify programs to use in the fetcher
> -----------------------------------------------
>
> Key: MESOS-1887
> URL: https://issues.apache.org/jira/browse/MESOS-1887
> Project: Mesos
> Issue Type: Improvement
> Components: fetcher, slave
> Reporter: Tobias Weingartner
>
> In environments with multiple different hadoop versions, the hadoop version
> on the slave's {{$PATH}}, or found via {{$HADOOP_HOME}} may not be the always
> globally correct version of hadoop. IE:
> * The slave may be fetching executors from hadoop1
> * The execution may be against/for a hadoop2
> Unfortunately, {{hdfs://}} urls are not specific enough to be able to
> dis-ambiguate the particular version of hadoop you should use to access the
> hadoop/hdfs cluster given in the url.
> Off hand, I can see two possible solutions:
> * extra information from the framework in the task struct to help identify
> the fetchers environment for fetching this particular executor (different
> from the slave's environment, different from a task's environment).
> * fetchers global environment (different from slave and task's
> environemnt), allowing a different hadoop/hdfs cluster to be used to fetch
> _all_ executors from within hdfs
> * a method to replace the fetcher with site specific scripts/programs
> Other?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
