Tobias Weingartner created MESOS-1887:
-----------------------------------------
Summary: Allow to specify programs to use in the fetcher
Key: MESOS-1887
URL: https://issues.apache.org/jira/browse/MESOS-1887
Project: Mesos
Issue Type: Bug
Components: slave
Reporter: Tobias Weingartner
In environments with multiple different hadoop versions, the hadoop version on
the slave's {{$PATH}}, or found via {{$HADOOP_HOME}} may not be the always
globally correct version of hadoop. IE:
* The slave may be fetching executors from hadoop1
* The execution may be against/for a hadoop2
Unfortunately, {{hdfs://}} urls are not specific enough to be able to
dis-ambiguate the particular version of hadoop you should use to access the
hadoop/hdfs cluster given in the url.
Off hand, I can see two possible solutions:
* extra information from the framework in the task struct to help identify
the fetchers environment for fetching this particular executor (different from
the slave's environment, different from a task's environment).
* fetchers global environment (different from slave and task's environemnt),
allowing a different hadoop/hdfs cluster to be used to fetch _all_ executors
from within hdfs
* a method to replace the fetcher with site specific scripts/programs
Other?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
