Re: How does Mesos parse hadoop command??

2015-11-04 Thread Du, Fan
Hallo I reconfigured my mesos slaves default configuration with HADOOP_HOME and JAVA_HOME set as below, then my SparkPi works well for the first time, yes, with joy :) # cat /etc/default/mesos-slave MASTER=`cat /etc/mesos/zk` HADOOP_HOME=/opt/hadoop-2.6.0/ JAVA_HOME=/usr/lib/jvm/java-1.7.0-open

Re: How does Mesos parse hadoop command??

2015-11-04 Thread Elizabeth Lingg
As a follow up, a place you would typically set the JAVA_HOME environment variable would be in /etc/default/mesos-slave on Ubuntu On Wed, Nov 4, 2015 at 11:38 AM, Elizabeth Lingg wrote: > Ah, yes. I have seen this issue. Typically, it is because you have > JAVA_HOME set on your host,but not on y

Re: How does Mesos parse hadoop command??

2015-11-04 Thread Elizabeth Lingg
Ah, yes. I have seen this issue. Typically, it is because you have JAVA_HOME set on your host,but not on your Mesos Agent. If you run a Marathon job and output "env" you will see the JAVA_HOME environment variable is missing. You would need to set it in your agent init configurations as export JAVA

Re: How does Mesos parse hadoop command??

2015-11-04 Thread haosdent
how about add this flag when launch slave --executor_environment_variables='{"HADOOP_HOME": "/opt/hadoop-2.6.0"}' ? On Wed, Nov 4, 2015 at 5:13 PM, Du, Fan wrote: > > > On 2015/11/4 17:09, haosdent wrote: > >> I notice >> ``` >> "user":"root" >> ``` >> Do you make sure could execute `hadoop ver

Re: How does Mesos parse hadoop command??

2015-11-04 Thread Du, Fan
On 2015/11/4 17:09, haosdent wrote: I notice ``` "user":"root" ``` Do you make sure could execute `hadoop version` under root? [root@tylersburg spark-1.5.1-bin-hadoop2.6]# whoami root [root@tylersburg spark-1.5.1-bin-hadoop2.6]# hadoop version Hadoop 2.6.0 Subversion https://git-wip-us.apach

Re: How does Mesos parse hadoop command??

2015-11-04 Thread haosdent
I notice ``` "user":"root" ``` Do you make sure could execute `hadoop version` under root? On Wed, Nov 4, 2015 at 4:56 PM, Du, Fan wrote: > > > On 2015/11/4 16:40, Tim Chen wrote: > >> What OS are you running this with? >> >> And I assume if you run /bin/sh and try to run hadoop it can be found

Re: How does Mesos parse hadoop command??

2015-11-04 Thread Du, Fan
On 2015/11/4 16:40, Tim Chen wrote: What OS are you running this with? And I assume if you run /bin/sh and try to run hadoop it can be found in your PATH as well? I'm using CentOS-7.2 # /bin/sh hadoop version Hadoop 2.6.0 Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r e34

Re: How does Mesos parse hadoop command??

2015-11-04 Thread Tim Chen
What OS are you running this with? And I assume if you run /bin/sh and try to run hadoop it can be found in your PATH as well? Tim On Wed, Nov 4, 2015 at 12:34 AM, Du, Fan wrote: > Hi Mesos experts > > I setup a small mesos cluster with 1 master and 6 slaves, > and deploy hdfs on the same clus

How does Mesos parse hadoop command??

2015-11-04 Thread Du, Fan
Hi Mesos experts I setup a small mesos cluster with 1 master and 6 slaves, and deploy hdfs on the same cluster topology, both with root user role. #cat spark-1.5.1-bin-hadoop2.6/conf/spark-env.sh export MESOS_NATIVE_JAVA_LIBRARY=/usr/local/lib/libmesos.so export JAVA_HOME=/usr/lib/jvm/java-1.7.