Hi, regarding symlinks i followed rather explicit directions:
cd hadoop-2.5.0-cdh5.2.0 mv bin bin-mapreduce2 mv examples examples-mapreduce2 ln -s bin-mapreduce1 bin ln -s examples-mapreduce1 examples pushd etc mv hadoop hadoop-mapreduce2 ln -s hadoop-mapreduce1 hadoop popd pushd share/hadoop rm mapreduce ln -s mapreduce1 mapreduce popd Please tell if the mesos/hadoop readme is not specific enough or wrong ________________________________ From: Elizabeth Lingg <[email protected]> Sent: Friday, May 22, 2015 8:22 PM To: user Subject: Re: mesos hadoop jobtracker - cant start. Just looking at this quickly, are you sure you set up your symlinks correctly, i.e. "Since CDH5 includes both MRv1 and MRv2 (YARN) and is configured for YARN by default, we need update the symlinks to point to the correct directories." If you are interested in running Yarn, MR-2, instead of MR-1, you might be interested in trying out the Myriad project, https://github.com/mesos/myriad. https://github.com/mesos/hadoop is for Hadoop 1/ MR-1. Thanks, Elizabeth On Fri, May 22, 2015 at 6:49 AM, Maciej Strzelecki <[email protected]<mailto:[email protected]>> wrote: I am following the install steps on https://github.com/mesos/hadoop Question one: -------------------- Where should be conf/mapred-site.xml I can only see etc/hadoop/mapred-site.xml , there is no conf/ directory Question two: ------------------- How can i start the hadoop mesos framework? Steps: - mvn package and put target/mesos-hadoop-mr1-0.1.1-SNAPSHOT.jar into hadoop-2.5.0-cdh5.2.0/share/hadoop/common/lib/ - done the link changes described - packaged tar, uploaded to hdfs (hdfs://hdfs/hadoop-2.5.0-cdh5.2.0.tar.gz) - patched the config file (etc/hadoop/mapred-site.xml) - uploaded again (with the changed config file) invoking the hadoop binary from downloaded/unpacked tar: root@mesos-master3 ~/hadoop-2.5.0-cdh5.2.0 # MESOS_NATIVE_JAVA_LIBRARY=/usr/lib/libmesos.so ./bin/hadoop jobtracker Error: Could not find or load main class org.apache.hadoop.mapred.JobTracker my classpath echo: root@mesos-master3 ~/hadoop-2.5.0-cdh5.2.0 # MESOS_NATIVE_JAVA_LIBRARY=/usr/lib/libmesos.so ./bin/hadoop jobtracker /root/hadoop-2.5.0-cdh5.2.0/bin-mapreduce1/../etc/hadoop:/usr/lib/tools.jar:/root/hadoop-2.5.0-cdh5.2.0/bin-mapreduce1/../share/hadoop/mapreduce1/hadoop-core-2.5.0-mr1-cdh5.2.0.jar:/root/hadoop-2.5.0-cdh5.2.0/bin-mapreduce1/../lib/*.jar:/root/hadoop-2.5.0-cdh5.2.0/bin-mapreduce1/../lib/jsp-2.1/*.jar:/root/hadoop-2.5.0-cdh5.2.0/bin-mapreduce1/../etc/hadoop:/root/hadoop-2.5.0-cdh5.2.0/share/hadoop/common/lib/*:/root/hadoop-2.5.0-cdh5.2.0/share/hadoop/common/*:/root/hadoop-2.5.0-cdh5.2.0/share/hadoop/hdfs:/root/hadoop-2.5.0-cdh5.2.0/share/hadoop/hdfs/lib/*:/root/hadoop-2.5.0-cdh5.2.0/share/hadoop/hdfs/*:/root/hadoop-2.5.0-cdh5.2.0/share/hadoop/yarn/lib/*:/root/hadoop-2.5.0-cdh5.2.0/share/hadoop/yarn/*:/root/hadoop-2.5.0-cdh5.2.0/share/hadoop/mapreduce:/root/hadoop-2.5.0-cdh5.2.0/share/hadoop/mapreduce/lib/*:/root/hadoop-2.5.0-cdh5.2.0/share/hadoop/mapreduce/* Error: Could not find or load main class org.apache.hadoop.mapred.JobTracker invoking "hadoop" gives me an error about jobtracker command no longer supported. MESOS_NATIVE_JAVA_LIBRARY=/path/to/libmesos.so hadoop jobtracker root@mesos-master3 ~/hadoop-2.5.0-cdh5.2.0 # MESOS_NATIVE_JAVA_LIBRARY=/usr/lib/libmesos.so hadoop jobtracker DEPRECATED: Use of this script to execute mapred command is deprecated. Instead use the mapred command for it. Sorry, the jobtracker command is no longer supported. You may find similar functionality with the "yarn" shell command. Usage: mapred [--config confdir] COMMAND where COMMAND is one of: pipes run a Pipes job job manipulate MapReduce jobs queue get information regarding JobQueues classpath prints the class path needed for running mapreduce subcommands historyserver run job history servers as a standalone daemon distcp <srcurl> <desturl> copy file or directories recursively archive -archiveName NAME -p <parent path> <src>* <dest> create a hadoop archive hsadmin job history server admin interface Most commands print help when invoked w/o parameters.
