Thanks for the reply, Vinod.. We are running 0.9.0-incubating downloaded from here:
http://www.apache.org/dyn/closer.cgi/incubator/mesos/mesos-0.9.0-incubating/ The Hadoop I'm trying to run is v0.20.205 that's included with this Mesos distribution. But we WANT to run hadoop-2.0.0-mr1-cdh4.1.2. I've got a patch for this, but not sure exactly what I should use as the starting point. Can you give me a few pointers on how to proceed? We want the hadoop-2.0.0-mr-cdh4.1.2 version, and we'll build/run anything required to get it. Craig On Fri, Mar 1, 2013 at 9:16 AM, Vinod Kone <[email protected]> wrote: > Hey Craig, > > Which version of Mesos are you running? We recently (~1 month) revamped > our Hadoop port and unfortunately didn't update the wiki. > > @vinodkone > Sent from my mobile > > On Mar 1, 2013, at 7:48 AM, Craig Vanderborgh <[email protected]> > wrote: > > > I'm getting close. I've got the namenode formatted and HDFS running. I > > started hdfs with $HADOOP_HOME/bin/start-dfs.sh. Basic HDFS operations > > seem to be working. Now it's time to start the jobtracker - I have been > > following the instructions here: > > > > https://github.com/mesos/mesos/wiki/Running-Hadoop-on-Mesos > > > > The instructions say "Launch a JobTracker with bin/hadoop jobtracker (do > > not use bin/start-mapred.sh)... > > > > So here's what happens: > > > > [craigv@sleuth-mesos-01 bin]$ hadoop jobtracker > > 13/03/01 00:23:02 INFO mapred.JobTracker: STARTUP_MSG: > > /************************************************************ > > STARTUP_MSG: Starting JobTracker > > STARTUP_MSG: host = sleuth-mesos-01.lab.webapps.rr.com/71.74.48.186 > > STARTUP_MSG: args = [] > > STARTUP_MSG: version = 0.20.205.1 > > STARTUP_MSG: build = -r ; compiled by 'craigv' on Thu Feb 28 22:59:28 > > UTC 2013 > > ************************************************************/ > > 13/03/01 00:23:02 INFO impl.MetricsConfig: loaded properties from > > hadoop-metrics2.properties > > 13/03/01 00:23:02 INFO impl.MetricsSourceAdapter: MBean for source > > MetricsSystem,sub=Stats registered. > > 13/03/01 00:23:02 INFO impl.MetricsSystemImpl: Scheduled snapshot period > at > > 10 second(s). > > 13/03/01 00:23:02 INFO impl.MetricsSystemImpl: JobTracker metrics system > > started > > 13/03/01 00:23:02 INFO impl.MetricsSourceAdapter: MBean for source > > QueueMetrics,q=default registered. > > 13/03/01 00:23:02 INFO impl.MetricsSourceAdapter: MBean for source ugi > > registered. > > 13/03/01 00:23:02 INFO delegation.AbstractDelegationTokenSecretManager: > > Updating the current master key for generating delegation tokens > > 13/03/01 00:23:02 INFO delegation.AbstractDelegationTokenSecretManager: > > Starting expired delegation token remover thread, > > tokenRemoverScanInterval=60 min(s) > > 13/03/01 00:23:02 INFO delegation.AbstractDelegationTokenSecretManager: > > Updating the current master key for generating delegation tokens > > 13/03/01 00:23:02 INFO mapred.JobTracker: Scheduler configured with > > (memSizeForMapSlotOnJT, memSizeForReduceSlotOnJT, limitMaxMemForMapTasks, > > limitMaxMemForReduceTasks) (-1, -1, -1, -1) > > 13/03/01 00:23:02 INFO util.HostsFileReader: Refreshing hosts > > (include/exclude) list > > 13/03/01 00:23:02 INFO mapred.JobTracker: Starting jobtracker with owner > as > > craigv > > 13/03/01 00:23:02 INFO ipc.Server: Starting SocketReader > > 13/03/01 00:23:02 INFO impl.MetricsSourceAdapter: MBean for source > > RpcDetailedActivityForPort54311 registered. > > 13/03/01 00:23:02 INFO impl.MetricsSourceAdapter: MBean for source > > RpcActivityForPort54311 registered. > > 13/03/01 00:23:02 INFO mortbay.log: Logging to > > org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via > > org.mortbay.log.Slf4jLog > > 13/03/01 00:23:02 INFO http.HttpServer: Added global filtersafety > > (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) > > 13/03/01 00:23:02 INFO http.HttpServer: Port returned by > > webServer.getConnectors()[0].getLocalPort() before open() is -1. Opening > > the listener on 50030 > > 13/03/01 00:23:02 INFO http.HttpServer: listener.getLocalPort() returned > > 50030 webServer.getConnectors()[0].getLocalPort() returned 50030 > > 13/03/01 00:23:02 INFO http.HttpServer: Jetty bound to port 50030 > > 13/03/01 00:23:02 INFO mortbay.log: jetty-6.1.26 > > 13/03/01 00:23:02 INFO mortbay.log: Started > > [email protected]:50030 > > 13/03/01 00:23:02 INFO impl.MetricsSourceAdapter: MBean for source jvm > > registered. > > 13/03/01 00:23:02 INFO impl.MetricsSourceAdapter: MBean for source > > JobTrackerMetrics registered. > > 13/03/01 00:23:02 INFO mapred.JobTracker: JobTracker up at: 54311 > > 13/03/01 00:23:02 INFO mapred.JobTracker: JobTracker webserver: 50030 > > 13/03/01 00:23:02 INFO mapred.JobTracker: Cleaning up the system > directory > > 13/03/01 00:23:02 INFO mapred.JobTracker: History server being > initialized > > in embedded mode > > 13/03/01 00:23:02 INFO mapred.JobHistoryServer: Started job history > server > > at: localhost:50030 > > 13/03/01 00:23:02 INFO mapred.JobTracker: Job History Server web address: > > localhost:50030 > > 13/03/01 00:23:02 INFO mapred.CompletedJobStatusStore: Completed job > store > > is inactive > > 13/03/01 00:23:03 INFO mapred.MesosScheduler: Starting MesosScheduler > > java: ./common/try.hpp:77: T Try<T>::get() const [with T = > > mesos::internal::MasterDetector*]: Assertion `state == SOME' failed. > > Aborted > > > > Close but no cigar. The Mesos code in this proximity: > > > > src/common/try.hpp > > > > Try<T>& operator = (const Try<T>& that) > > { > > if (this != &that) { > > state = that.state; > > if (that.t != NULL) { > > t = new T(*that.t); > > } else { > > t = NULL; > > } > > message = that.message; > > } > > > > return *this; > > } > > > > bool isSome() const { return state == SOME; } > > bool isError() const { return state == ERROR; } > > > > T get() const { assert(state == SOME); return *t; } > > > > std::string error() const { assert(state == ERROR); return message; } > > > > What's wrong here? Obviously it can't start MesosScheduler, but why? > > > > Here is my mapred-site.xml: > > > > <?xml version="1.0"?> > > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> > > > > <!-- Put site-specific property overrides in this file. --> > > > > <configuration> > > <property> > > <name>mapred.job.tracker</name> > > <value>localhost:54311</value> > > </property> --> > > <property> > > <name>mapred.jobtracker.taskScheduler</name> > > <value>org.apache.hadoop.mapred.MesosScheduler</value> > > </property> > > <property> > > <name>mapred.mesos.master</name> > > <value>mesos://sleuth-mesos-01:5050</value> > > </property> > > </configuration> > > > > THANKS! > > Craig >
