Finally, I was able to get HBase running on EC2 in fully distributed mode. I started ZooKeeper quorum myself and pointed HBase to it. I was able to create tables using HBase shell, ran a Mapreduce job that writes to these tables, and run queries against these tables. I used HBase shell from all 3 machines, and they all see the same data confirming that the instances are indeed working together.
It seems like under EC2, starting ZooKeeper within HBase doesn't work, but I could be wrong. In any case, Andrew, I would like to get your scripts working in my environment because without your scripts I don't know how I would grow my cluster from 3 instances to say, 30 :) Thank you so much everyone for your help and for sticking with me. On Wed, Dec 9, 2009 at 8:25 PM, Something Something < [email protected]> wrote: > When I run: > > hbase-ec2 launch-cluster testcluster 3 3 > > I keep getting 'lost connection' messages (See below). Tried this 4 > times. Please help. Thanks. > > > ------------------------------------------------------------- > > Creating/checking security groups > Security group testcluster-master exists, ok > Security group testcluster exists, ok > Security group testcluster-zookeeper exists, ok > Starting ZooKeeper quorum ensemble. > Starting an AMI with ID ami-b0cb29d9 (arch i386) in group > testcluster-zookeeper > Waiting for instance i-9db6f4f5 to start: .................. Started > ZooKeeper instance i-9db6f4f5 as domU-12-31-38-01-7D-D1.compute-1.internal > Public DNS name is ec2-174-129-148-5.compute-1.amazonaws.com. > Starting an AMI with ID ami-b0cb29d9 (arch i386) in group > testcluster-zookeeper > Waiting for instance i-2db7f545 to start: ................. Started > ZooKeeper instance i-2db7f545 as domU-12-31-38-01-7D-43.compute-1.internal > Public DNS name is ec2-174-129-157-122.compute-1.amazonaws.com. > Starting an AMI with ID ami-b0cb29d9 (arch i386) in group > testcluster-zookeeper > Waiting for instance i-afb7f5c7 to start: ...................... Started > ZooKeeper instance i-afb7f5c7 as domU-12-31-38-01-78-F3.compute-1.internal > Public DNS name is ec2-174-129-179-14.compute-1.amazonaws.com. > ZooKeeper quorum is > domU-12-31-38-01-7D-D1.compute-1.internal,domU-12-31-38-01-7D-43.compute-1.internal,domU-12-31-38-01-78-F3.compute-1.internal. > Initializing the ZooKeeper quorum ensemble. > ec2-174-129-148-5.compute-1.amazonaws.com > lost connection > ec2-174-129-157-122.compute-1.amazonaws.com > lost connection > ec2-174-129-179-14.compute-1.amazonaws.com > lost connection > > > > > On Wed, Dec 9, 2009 at 12:46 AM, Seth Ladd <[email protected]> wrote: > >> > Sounds like others have used Andrew's script successfully. The only >> > difference seems to be that it starts a *dedicated* ZooKeeper quorum. >> > Should have listened to Mark when he suggested that 4 days ago :) >> > >> > Anyway, I will try Andrew's script tomorrow. >> >> I can vouch that the scripts in svn trunk work. Thanks to Andrew for >> his help! I was able to start a 3 node Zookeeper and 5 node HBase >> cluster on EC2 from just the scripts. >> >> Seth >> > >
