Hi,Andy Yes. I shutdown EC2 instance every day to save some money for my company. I use shell scripts to reduce manual editing as much as possible. I just edit core-site.xml and yarn-site.xml in single node and then scp it to all the other nodes. This can be automated. You can add to /etc/ssh/ssh-config the two lines below
StrictHostKeyChecking no UserKnownHostsFile /dev/null so that you will not need to type "yes" when connecting to an unknown host. So I re-edited four files core-site.xml and yarn-site.xml, slaves and master on namenode everyday. It take only a few minutes. I did not use Zookeeper to deal with this. Please let me know if you have better approach. On Wed, Nov 14, 2012 at 6:34 AM, Kartashov, Andy <[email protected]>wrote: > Yinghau, > > > > Last week you mentioned that you are running your cluster on EC2 that you > shut them down instance(s) over the weekend. > > > > QTE > > Since I shutdown the EC2 instance every night, I thought that using > 'master','slave1','slave2' will save typing after the full host name change > with reboot. > > UNQTE > > > > Whenever I re-start an instance on Amazon cloud my both internal and > external ip addresses for the instance change… ? > > > > p.s. Has anyone successfully configured Zookeeper to curb this matter? > > > > *From:* yinghua hu [mailto:[email protected]] > *Sent:* Sunday, November 11, 2012 2:06 PM > *To:* [email protected] > *Subject:* Re: error running pi program > > > > Hi, Steve > > > > These are very helpful suggestions! Thanks! > > > > I will try using SSH proxy and see how it goes. > > On Sun, Nov 11, 2012 at 10:02 AM, Steve Loughran <[email protected]> > wrote: > > > > On 10 November 2012 18:30, yinghua hu <[email protected]> wrote: > > Thanks Steve, > > > > Your suggestion is very helpful. But I only have terminal access to the > EC2 NN. To run internet browser there will be a lot of more work. > > > > the SSH proxy sets up something that appears to be a SOCKs proxy -but > which routes to the far end; you set your browser to go through this proxy > (foxyproxy is a good firefox plugin here), and then your browser has access > to all the machines > > > > see http://www.linuxjournal.com/content/use-ssh-create-http-proxy > > > > > > > > Since my cluster will be shut down at night and weekend. I will prefer to > have an IP that does not change. How do you think of using EC2 Elastic IP? > > > > > > not tried that; I have used dyndns.org to self register an IP address > that changes every time you bring up an instance, but whose browser > bookmarks can remain fixed > > > > > > -- > Regards, > > Yinghua > NOTICE: This e-mail message and any attachments are confidential, > subject to copyright and may be privileged. Any unauthorized use, copying > or disclosure is prohibited. If you are not the intended recipient, please > delete and contact the sender immediately. Please consider the environment > before printing this e-mail. AVIS : le présent courriel et toute pièce > jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur > et peuvent être couverts par le secret professionnel. Toute utilisation, > copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le > destinataire prévu de ce courriel, supprimez-le et contactez immédiatement > l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent > courriel > -- Regards, Yinghua
