unfortunately without DNS you have to rely on /etc/hosts, so put in entry for all your nodes(nn,snn,dn1,dn2 etc..) on all nodes(/etc/hosts file) and i have that tested for hortonworks(using ambari) and cloudera manager and i am certainly sure it will work for MapR
On Thu, Mar 5, 2015 at 8:47 PM, Alexander Pivovarov <[email protected]> wrote: > what about DNS? > if you have 2 computers (nn and dn) how nn knows dn ip? > > The script puts only this computer ip to /etc/hosts > > On Thu, Mar 5, 2015 at 6:39 PM, max scalf <[email protected]> wrote: > >> Here is a easy way to go about assigning static name to your ec2 >> instance. When you get the launch an EC2-instance from aws console when >> you get to the point of selecting VPC, ip address screen there is a screen >> that says "USER DATA"...put the below in with appropriate host name(change >> CHANGE_HOST_NAME_HERE to whatever you want) and that should be able to get >> you static name. >> >> #!/bin/bash >> >> HOSTNAME_TAG=CHANGE_HOST_NAME_HERE >> cat > /etc/sysconfig/network << EOF >> NETWORKING=yes >> NETWORKING_IPV6=no >> HOSTNAME=${HOSTNAME_TAG} >> EOF >> >> IP=$(curl http://169.254.169.254/latest/meta-data/local-ipv4) >> echo "${IP} ${HOSTNAME_TAG}.localhost ${HOSTNAME_TAG}" >> /etc/hosts >> >> echo ${HOSTNAME_TAG} > /proc/sys/kernel/hostname >> service network restart >> >> >> Also note i was able to do this on couple of spot instance for cheap >> price, only thing is once you shut it down or someone outbids you, you >> loose that instance but its easy/cheap to play around with.... and i have >> used couple of m3.medium for my NN/SNN and couple of them for data nodes... >> >> On Thu, Mar 5, 2015 at 7:19 PM, Jonathan Aquilina < >> [email protected]> wrote: >> >>> I dont know how you would do that to be honest. With EMR you have >>> destinctions master core and task nodes. If you need to change >>> configuration you just ssh into the EMR master node. >>> >>> >>> >>> --- >>> Regards, >>> Jonathan Aquilina >>> Founder Eagle Eye T >>> >>> On 2015-03-06 02:11, Alexander Pivovarov wrote: >>> >>> What is the easiest way to assign names to aws ec2 computers? >>> I guess computer need static hostname and dns name before it can be used >>> in hadoop cluster. >>> On Mar 5, 2015 4:36 PM, "Jonathan Aquilina" <[email protected]> >>> wrote: >>> >>>> When I started with EMR it was alot of testing and trial and error. >>>> HUE is already supported as something that can be installed from the AWS >>>> console. What I need to know is if you need this cluster on all the time or >>>> this is goign ot be what amazon call a transient cluster. Meaning you fire >>>> it up run the job and tear it back down. >>>> >>>> >>>> >>>> --- >>>> Regards, >>>> Jonathan Aquilina >>>> Founder Eagle Eye T >>>> >>>> On 2015-03-06 01:10, Krish Donald wrote: >>>> >>>> Thanks Jonathan, >>>> >>>> I will try to explore EMR option also. >>>> Can you please let me know the configuration which you have used it? >>>> Can you please recommend for me also? >>>> I would like to setup Hadoop cluster using cloudera manager and then >>>> would like to do below things: >>>> >>>> setup kerberos >>>> setup federation >>>> setup monitoring >>>> setup hadr >>>> backup and recovery >>>> authorization using sentry >>>> backup and recovery of individual componenets >>>> performamce tuning >>>> upgrade of cdh >>>> upgrade of CM >>>> Hue User Administration >>>> Spark >>>> Solr >>>> >>>> >>>> Thanks >>>> Krish >>>> >>>> >>>> On Thu, Mar 5, 2015 at 3:57 PM, Jonathan Aquilina < >>>> [email protected]> wrote: >>>> >>>>> krish EMR wont cost you much with all the testing and data we ran >>>>> through the test systems as well as the large amont of data when >>>>> everythign >>>>> was read we paid about 15.00 USD. I honestly do not think that the specs >>>>> there would be enough as java can be pretty ram hungry. >>>>> >>>>> >>>>> >>>>> --- >>>>> Regards, >>>>> Jonathan Aquilina >>>>> Founder Eagle Eye T >>>>> >>>>> On 2015-03-06 00:41, Krish Donald wrote: >>>>> >>>>> Hi, >>>>> >>>>> I am new to AWS and would like to setup Hadoop cluster using cloudera >>>>> manager for 6-7 nodes. >>>>> >>>>> t2.micro on AWS; Is it enough for setting up Hadoop cluster ? >>>>> I would like to use free service as of now. >>>>> >>>>> Please advise. >>>>> >>>>> Thanks >>>>> Krish >>>>> >>>>> >> >
