I found the following solution to this problem I registered 2 subdomains (public and local) for each computer on https://freedns.afraid.org/subdomain/ e.g. myhadoop-nn.crabdance.com myhadoop-nn-local.crabdance.com
then I added cron job which sends http requests to update public and local ip on freedns server hint: public ip is detected automatically ip address for local name can be set using request parameter &address=10.x.x.x (don't forget to escape &) as a result my nn computer has 2 DNS names with currently assigned ip addresses , e.g. myhadoop-nn.crabdance.com 54.203.181.177 myhadoop-nn-local.crabdance.com 10.220.149.103 in hadoop configuration I can use local machine names to access my cluster outside of AWS I can use public names Just curious if AWS provides easier way to name EC2 computers? On Thu, Mar 5, 2015 at 5:19 PM, Jonathan Aquilina <[email protected]> wrote: > I dont know how you would do that to be honest. With EMR you have > destinctions master core and task nodes. If you need to change > configuration you just ssh into the EMR master node. > > > > --- > Regards, > Jonathan Aquilina > Founder Eagle Eye T > > On 2015-03-06 02:11, Alexander Pivovarov wrote: > > What is the easiest way to assign names to aws ec2 computers? > I guess computer need static hostname and dns name before it can be used > in hadoop cluster. > On Mar 5, 2015 4:36 PM, "Jonathan Aquilina" <[email protected]> > wrote: > >> When I started with EMR it was alot of testing and trial and error. HUE >> is already supported as something that can be installed from the AWS >> console. What I need to know is if you need this cluster on all the time or >> this is goign ot be what amazon call a transient cluster. Meaning you fire >> it up run the job and tear it back down. >> >> >> >> --- >> Regards, >> Jonathan Aquilina >> Founder Eagle Eye T >> >> On 2015-03-06 01:10, Krish Donald wrote: >> >> Thanks Jonathan, >> >> I will try to explore EMR option also. >> Can you please let me know the configuration which you have used it? >> Can you please recommend for me also? >> I would like to setup Hadoop cluster using cloudera manager and then >> would like to do below things: >> >> setup kerberos >> setup federation >> setup monitoring >> setup hadr >> backup and recovery >> authorization using sentry >> backup and recovery of individual componenets >> performamce tuning >> upgrade of cdh >> upgrade of CM >> Hue User Administration >> Spark >> Solr >> >> >> Thanks >> Krish >> >> >> On Thu, Mar 5, 2015 at 3:57 PM, Jonathan Aquilina < >> [email protected]> wrote: >> >>> krish EMR wont cost you much with all the testing and data we ran >>> through the test systems as well as the large amont of data when everythign >>> was read we paid about 15.00 USD. I honestly do not think that the specs >>> there would be enough as java can be pretty ram hungry. >>> >>> >>> >>> --- >>> Regards, >>> Jonathan Aquilina >>> Founder Eagle Eye T >>> >>> On 2015-03-06 00:41, Krish Donald wrote: >>> >>> Hi, >>> >>> I am new to AWS and would like to setup Hadoop cluster using cloudera >>> manager for 6-7 nodes. >>> >>> t2.micro on AWS; Is it enough for setting up Hadoop cluster ? >>> I would like to use free service as of now. >>> >>> Please advise. >>> >>> Thanks >>> Krish >>> >>>
