Oh, in fact, what happened was, I had 3 ZK nodes, and one was an observer. When I brought down the one server, I didn't have a quorum (gotta love that word). So I had to manually change my zoo.cfg and remove the :observer flag. Then my ZKs came back up.
So what am I doing wrong? I want to make sure a server can go down my ZKs remain up. :) thanks again! On Wed, Jan 28, 2015 at 4:08 PM, Dillian Murphey <[email protected]> wrote: > Thanks Cameron, ok so I just ran the exercise again. I killed a server. > The other two ZKs go offline and the zookeeper.out log is as so: > > So why does this feel as if I need to manually adjust the config file? > > Thanks. P.S. I am using netflix opensource exhibitor. It may be a factor, > I don't know. The creator seems to brush me off a lot. > > 2015-01-29 00:05:51,271 [myid:] - ERROR [main:QuorumPeerMain@85] - > Invalid config, exiting abnormally > org.apache.zookeeper.server.quorum.QuorumPeerConfig$ConfigException: Error > processing /zookeeper/zookeeper-3.4.6/bin/../conf/zoo.cfg > at > org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:123) > at > org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:101) > at > org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78) > Caused by: java.lang.IllegalArgumentException: Observers w/o quorum is an > invalid configuration > at > org.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties(QuorumPeerConfig.java:284) > at > org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:119) > ... 2 more > Invalid config, exiting abnormally > tail: zookeeper.out: file truncated > 2015-01-29 00:06:11,561 [myid:] - INFO [main:QuorumPeerConfig@103] - > Reading configuration from: /zookeeper/zookeeper-3.4.6/bin/../conf/zoo.cfg > 2015-01-29 00:06:11,566 [myid:] - ERROR [main:QuorumPeerMain@85] - > Invalid config, exiting abnormally > org.apache.zookeeper.server.quorum.QuorumPeerConfig$ConfigException: Error > processing /zookeeper/zookeeper-3.4.6/bin/../conf/zoo.cfg > at > org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:123) > at > org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:101) > at > org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78) > Caused by: java.lang.IllegalArgumentException: Observers w/o quorum is an > invalid configuration > at > org.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties(QuorumPeerConfig.java:284) > at > org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:119) > ... 2 more > Invalid config, exiting abnormally > > On Wed, Jan 28, 2015 at 3:45 PM, Cameron McKenzie <[email protected]> > wrote: > >> Assuming that your 2 remaining servers can communicate with each other the >> cluster should should remain online as it still has a quorum (2 / 3 nodes >> are online). There may be a brief period where the cluster won't accept >> requires while a new leadership election occurs between the 2 remaining >> nodes. >> cheers >> Cam >> >> On Thu, Jan 29, 2015 at 10:40 AM, Dillian Murphey < >> [email protected]> >> wrote: >> >> > If I have 3 ZK instances running on separate servers and I terminate >> one of >> > the servers, what should I expect to happen? >> > >> > Will the remaining 2 zookeepers continue happily and stay online without >> > any interruption? >> > >> > I'm using exhibitor/zookeeper and I'm attempting this scenerio and what >> I'm >> > seeing is my remaining 2 zookeepers go offline. I thought the whole >> point >> > of having a majority was to allow a failure? >> > >> > >> > Thank you. I'm new so be kind. >> > >> > >
