Didn't get a chance to go thru Andrew's script, but I tested the theory of using fully qualified internal domain names (such as, domU-12-31-38-00-89-AA.compute-1.internal) and that didn't work! Zookeeper is still having problems.
Sounds like others have used Andrew's script successfully. The only difference seems to be that it starts a *dedicated* ZooKeeper quorum. Should have listened to Mark when he suggested that 4 days ago :) Anyway, I will try Andrew's script tomorrow. On Tue, Dec 8, 2009 at 2:36 PM, Andrew Purtell <[email protected]> wrote: > > If HBase has scripts that start ZK for you it should be sure to > > specify fully qualified names in the ZK configuration. > > The scripts build the ZK config dynamically and use the fully qualified > (internal) domain names. > > - Andy > > > > > ________________________________ > From: Patrick Hunt <[email protected]> > To: [email protected] > Cc: Jean-Daniel Cryans <[email protected]> > Sent: Tue, December 8, 2009 3:31:53 PM > Subject: Re: Starting HBase in fully distributed mode... > > I consulted with a ZK user who runs exclusively on EC2. He mentioned > that one potential problem ZK users may be seeing is the following: > > EC2 instances can be on different subdomains, even in the same > geographical region. > > As a result if you configure ZK without fully qualified names the quorum > may fail as DHCP is typically configured to not setup name resolution to > search for hostnames across sub-domains. > > Verify that you are using fully qualified names in your ZK configuration > file. (an alternative is to configure DHCP to resolve across the > sub-domain, but this is not as simple as modifying resolve.conf afaik) > > If HBase has scripts that start ZK for you it should be sure to specify > fully qualified names in the ZK configuration. > > Patrick > > Vaibhav Puranik wrote: > > Something, > > > > You can now boot your instance with an EBS volume. This is different > from > > attaching an EBS volume after booting up an instance. > > This will simplify your setup. Furthermore, this functionality also > offers a > > new command to stop an instance which is different from 'Terminate'. Thus > > you can stop your instance but never loose data. > > > > Regards, > > Vaibhav Puranik > > Gumgum > > > > On Tue, Dec 8, 2009 at 11:11 AM, Something Something < > > [email protected]> wrote: > > > >> Awesome. Does it allow me to create HDFS on *my* EBS volumes? If not, > I > >> probably can change that later. Without EBS volumes all my data would > be > >> lost after I terminate my instances, right? > >> > >> On Tue, Dec 8, 2009 at 10:00 AM, Andrew Purtell <[email protected]> > >> wrote: > >> > >>> Use the HBase EC2 scripts to launch and manage a cluster up in EC2. The > >>> scripts launch the instances themselves, and take care of details such > >>> as setting up security groups, allocating and starting a dedicated ZK > >>> quorum ensemble, adding ZK and master private DNS names to the config > >>> and distributing it to the slaves, etc. > >>> > >>> See subtasks on HBASE-1961 for further developments planned. > >>> > >>> - Andy > >>> > >>> > >>> > >>> > >>> ________________________________ > >>> From: Something Something <[email protected]> > >>> To: [email protected] > >>> Sent: Tue, December 8, 2009 12:40:17 PM > >>> Subject: Re: Starting HBase in fully distributed mode... > >>> > >>> Cool. Didn't know there were scripts in the trunk for HBase. Will > >>> definitely try those. > >>> > >>> Should I first run the scripts for Hadoop (from > <Hadoop>/src/contrib/ec2) > >>> and then run scripts for HBase (from <HBase>/src/contrib/ec2)? Or > would > >>> the > >>> HBase scripts install/format HDFS automatically for me so that I don't > >> need > >>> to install Hadoop separately? > >>> > >>> In other words, what should be my approach: > >>> > >>> 1) Launch 3 instances on EC2. Run HBase scripts against them. > >>> > >>> OR > >>> > >>> 2) Launch 3 instances on EC2. Install Hadoop. Format HDFS. Start > HDFS. > >>> Run HBase scripts against these instances? > >>> > >>> Please let me know. Thanks. > >>> > >>> > >>> On Tue, Dec 8, 2009 at 8:50 AM, Andrew Purtell <[email protected]> > >>> wrote: > >>> > >>>>>> Good advice. This is what our EC2 scripts -- in src/contrib/ec2/ -- > >>>>>> do, and so I can say it works. > >>>>> I don't think there are scripts for HBase on EC2, are there? > >>>> We (HBase) have EC2 scripts -- src/contrib/ec2/, in SVN, in trunk and > >> on > >>>> the 0.20 branch. > >>>> > >>>> - Andy > >>>> > >>>> > >>>> > >>> > >>> > >>> > >>> > > > > > > >
