Mis feature, basically a master will tell the regionservers to 'shutdown and flush gracefully' via RPC.
Since we don't ship with any cluster management tools - to make your life easier we have a 'master tells RS to shutdown' path. I wouldn't be against removing it and relying on regular process management (Eg: ssh $host hbase-daemon.sh stop regionserver which uses kill -QUIT) to do cluster shutdown. At SU we have abstracted HBase away from the devs, they use thrift and never do anymore more than hbase shell. -ryan On Wed, Mar 2, 2011 at 5:23 PM, Bill Graham <[email protected]> wrote: > Hi, > > We had a troubling experience today that I wanted to share. Our dev > cluster got completely shut down by a developer by mistake, without > said developer even realizing it. Here's how... > > We have multiple sets of HBase configs checked into SVN that > developers can checkout and point their HBASE_CONF_DIR to to easily > change from developing in local mode or testing against our > distributed dev cluster. > > In local mode someone might do something like this: > > bin/start-hbase.sh > bin/hbase shell > > ... do some work ... > > bin/stop-hbase.sh > > The problem arose when a developer accidentally tried to do this with > their HBASE_CONF_DIR pointing to our dev cluster configs. When this > happens, the first command will add another master to the cluster and > the last command will shut down the entire cluster. I assume this > happens via Zookeeper somehow, since we don't have ssh keys to > remotely start/stop as the user running the processes. > > So the question is, is this a bug or a feature? If it's a feature it > seems like an incredibly dangerous one. Once our live cluster is > running, those configs will also be needed on the client so really bad > things could happen by mistake. > > thanks, > Bill >
