Do a reverse lookup and use the name you find. There are a few areas of Hadoopo that require reverse name lookup, but in general just create relevant entries (shared across the cluster, e.g. via Ansible if more than just a few nodes) in /etc/hosts.
Not hard. On Thu, Mar 5, 2015 at 6:35 PM, Alexander Pivovarov <[email protected]> wrote: > I found the following solution to this problem > > I registered 2 subdomains (public and local) for each computer on > https://freedns.afraid.org/subdomain/ > e.g. > myhadoop-nn.crabdance.com > myhadoop-nn-local.crabdance.com > > then I added cron job which sends http requests to update public and local > ip on freedns server > hint: public ip is detected automatically > ip address for local name can be set using request parameter > &address=10.x.x.x (don't forget to escape &) > > as a result my nn computer has 2 DNS names with currently assigned ip > addresses , e.g. > myhadoop-nn.crabdance.com 54.203.181.177 > myhadoop-nn-local.crabdance.com 10.220.149.103 > > in hadoop configuration I can use local machine names > to access my cluster outside of AWS I can use public names > > Just curious if AWS provides easier way to name EC2 computers? > > On Thu, Mar 5, 2015 at 5:19 PM, Jonathan Aquilina <[email protected]> > wrote: >> >> I dont know how you would do that to be honest. With EMR you have >> destinctions master core and task nodes. If you need to change configuration >> you just ssh into the EMR master node. >> >> >> >> --- >> Regards, >> Jonathan Aquilina >> Founder Eagle Eye T >> >> On 2015-03-06 02:11, Alexander Pivovarov wrote: >> >> What is the easiest way to assign names to aws ec2 computers? >> I guess computer need static hostname and dns name before it can be used >> in hadoop cluster. >> >> On Mar 5, 2015 4:36 PM, "Jonathan Aquilina" <[email protected]> >> wrote: >>> >>> When I started with EMR it was alot of testing and trial and error. HUE >>> is already supported as something that can be installed from the AWS >>> console. What I need to know is if you need this cluster on all the time or >>> this is goign ot be what amazon call a transient cluster. Meaning you fire >>> it up run the job and tear it back down. >>> >>> >>> >>> --- >>> Regards, >>> Jonathan Aquilina >>> Founder Eagle Eye T >>> >>> On 2015-03-06 01:10, Krish Donald wrote: >>> >>> Thanks Jonathan, >>> >>> I will try to explore EMR option also. >>> Can you please let me know the configuration which you have used it? >>> Can you please recommend for me also? >>> I would like to setup Hadoop cluster using cloudera manager and then >>> would like to do below things: >>> >>> setup kerberos >>> setup federation >>> setup monitoring >>> setup hadr >>> backup and recovery >>> authorization using sentry >>> backup and recovery of individual componenets >>> performamce tuning >>> upgrade of cdh >>> upgrade of CM >>> Hue User Administration >>> Spark >>> Solr >>> >>> >>> Thanks >>> Krish >>> >>> >>> On Thu, Mar 5, 2015 at 3:57 PM, Jonathan Aquilina >>> <[email protected]> wrote: >>>> >>>> krish EMR wont cost you much with all the testing and data we ran >>>> through the test systems as well as the large amont of data when everythign >>>> was read we paid about 15.00 USD. I honestly do not think that the specs >>>> there would be enough as java can be pretty ram hungry. >>>> >>>> >>>> >>>> --- >>>> Regards, >>>> Jonathan Aquilina >>>> Founder Eagle Eye T >>>> >>>> On 2015-03-06 00:41, Krish Donald wrote: >>>> >>>> Hi, >>>> >>>> I am new to AWS and would like to setup Hadoop cluster using cloudera >>>> manager for 6-7 nodes. >>>> >>>> t2.micro on AWS; Is it enough for setting up Hadoop cluster ? >>>> I would like to use free service as of now. >>>> >>>> Please advise. >>>> >>>> Thanks >>>> Krish > >
