Hey Craig/Sebastian. Great to see that you guys are testing out our new Hadoop port!
Once you have built mesos-hadoop, there should be a "hadoop.tar.gz" package inside the hadoop build directory (e.g. mesos/build/hadoop/hadoop-2.0.0-mr1-cdh4.1.2/build) This should contain all the stuff that is need to run the Mesosified Hadoop JobTracker and TaskTracker. In your case, I would suggest just copying this bundle to wherever you want to run the Hadoop JobTracker, unzip it and run "bin/hadoop/jobtracker". Some important info: --> Before running the jobtracker, make sure to edit "conf/mapred-site.xml" to the correct values. --> Specifically, you might want to set "mapred.mesos.master" to the URL of your Mesos Master (e.g. 1.2.3.4:5050). --> You also need to upload the hadoop.tar.gz to a location where Mesos slaves can pull it (e.g. http://<ip>/hadoop.tar.gz), to run the Hadoop TaskTracker. Let me know if that makes sense and if you have any further questions. Cheers, @vinodkone On Tue, Mar 5, 2013 at 10:07 AM, Sebastien Coutu <[email protected]> wrote: > Hi Corey, > > I've did a quick patch for CDH 4.1.2 that was tied to Mesos 0.9.0, it > worked well but had some resource issues with the Hadoop JobTracker but > when I get a few minutes, I'm planning on trying the new Mesos Scheduler. > If you manage get that working first, I'd like to see your feedback on it. > > Thanks, > > Sebastien Coutu > > On Tue, Mar 5, 2013 at 1:01 PM, Craig Vanderborgh < > [email protected]> wrote: > > > Hello, > > > > We want to use "Mesosized" Cloudera CDH1.4.2 so that we can share our > > cluster with Spark. > > > > We now have an operational Mesos 1.12.0 based on trunk plus R9 of the > > hadoop-mr1 patch that's under review in incubation. > > > > STATUS: > > o Mesos 1.12.0 is working in pseudo-distributed mode with one master and > > one slave > > o Spark 0.7.0 is working in this environment, including Spark Streaming > > o The build of hadoop-2.0-mr1 completes and the self tests are working > > > > WHAT NEXT? > > Our plan is to install the unmodified CDH4 RPM's and get a > > pseudo-distributed CDH4.1.2 working. Then, we plan to replace the stock > > CDH4.1.2 M/R 1 with the Mesosized version. Does this seem like the right > > way to proceed, and if so do you have any pointers on how to achieve this > > with the least amount of effort/pain? > > > > Thanks in advance, > > Craig Vanderborgh > > >
