Thanks this is great. I actually ended up using the vanilla Hadoop distribution and it worked just fine. I will try out your tutorial. Side question, is there a solution around making the jobtracker ha?
-- ankur Sent from my iPhone > On Oct 30, 2014, at 5:05 PM, Stratos Dimopoulos > <[email protected]> wrote: > > Hi Ankur, > > I recently went through the process of installing Hadoop on Mesos over > cdh5.1.2 > > I created a post here - shouldn't be much different for your version: > http://strat0sphere.wordpress.com/2014/10/30/hadoop-on-mesos-installation-guide/ > > You can also find an other post about configuring CDH5.1.2 specifically to > use with Mesos: useful: > http://strat0sphere.wordpress.com/2014/10/30/cloudera-hdfs-cdh5-installation-to-use-with-mesos/ > > Have in mind that when using Mesos you don't need to start the jobtracker. > Mesos will do this for you. > You also mentioned that you are trying to start hadoop as ubuntu user. This > is not the right thing to do. Either add root to cloudera's root list or > (recommended) use the root user (mapred? hdfs?) that your cloudera version > considers as root - you have to check the documentation for this. > > Regarding the error you are seeing "Does not contain a valid host:port > authority: local" - I've seen this error when my worker version was different > than the jobtracker version (happened because I was using a hadoop-on-mesos > tar file compiled with a different version than the one my cluster was > using). To fix this you can do the obvious, which is making sure the > installed version is the same with the one you ship to the executors through > HDFS or you can hack this by adding the property > hadoop.skip.worker.version.check to True - In the later case I wish you good > luck... Neverhteless, I am not sure if this error can also appear in other > cases. > > Hope this helps. > > Stratos > > >> On Tue, Oct 28, 2014 at 12:30 PM, Ankur Chauhan <[email protected]> wrote: >> Anyone else have something to add on this? >> -- Ankur Chauhan >> >>> On 28 Oct 2014, at 02:10, Ankur Chauhan <[email protected]> wrote: >>> >>> Hi tom, >>> >>> I was basically following the readme. This gist has the list of commands >>> how i am setting up things >>> https://gist.github.com/ankurcha/a9504b0e423b1a40d756 so first of all if >>> possible if you could help me verify if my process of setting up core-site, >>> hdfs-site and mapred-site is correct. I was starting the node with >>> >>> $ /opt/hadoop/bin/hadoop jobtracker >>> >>> There are two errors that i was working through. It seems that hadoop >>> doesn't like running as root (which is good) but despite starting the >>> process as ubuntu i kept getting >>> >>> Does not contain a valid host:port authority: local >>> >>> -- Ankur >>> >>>> On 28 Oct 2014, at 01:57, Tom Arnfeld <[email protected]> wrote: >>>> >>>> Hi Ankur, >>>> >>>> There aren't any getting started resources other than the documention >>>> there as far as I know. Could you share your hadoop configuration and >>>> perhaps a description of the problems you're having? >>>> >>>> Tom. >>>> >>>> >>>> >>>>> On Tue, Oct 28, 2014 at 8:53 AM, Ankur Chauhan <[email protected]> wrote: >>>>> H, >>>>> >>>>> >>>>> I was trying to setup mesos/hadoop with the latest CDH version (MR1) and >>>>> it seems like the instructions are sort of out of date and I also tried >>>>> the suggestions in https://github.com/mesos/hadoop/issues/25 but after 4 >>>>> hours of flailing around I am still kind of stuck :-/ >>>>> >>>>> It seems like the configuration/installation instructions aren't complete >>>>> and I am just too new to hadoop to figure out what's missing or going >>>>> wrong. Does anyone know of a good resource I can use to get going? >>>>> >>>>> -- Ankur >
