Josh is correct about the behavior of the ZooKeeper cli. As an aside, how big is this cluster? Five ZooKeeper servers shouldn't be needed until you get past ~100 nodes, unless you're just going for more fault tolerance.
Could you update your gist with the changes to accumulo-env.sh? It's much easier to follow there. Could you also re-run the accumulo classpath command on the tserver after updating accumulo-env.sh across the cluster? We already know the proximal cause of the failure: the Hadoop configuration files aren't ending up in the classpath for the tabletservers. I can think of a few things that would show this in the tablet server logs, but you've already described hitting most of them. -Sean On Wed, Mar 19, 2014 at 8:27 AM, Benjamin Parrish < [email protected]> wrote: > So, I am back to no clue now... > > > On Wed, Mar 19, 2014 at 9:13 AM, Josh Elser <[email protected]> wrote: > >> I think by default zkCli.sh will just try to connect to localhost. You >> can change this by providing the quorum string to the script with the >> -server option. >> On Mar 19, 2014 8:29 AM, "Benjamin Parrish" < >> [email protected]> wrote: >> >>> I adjusted accumulo-env.sh to have hard coded values as seen below. >>> >>> Are there any logs that could shed some light on this issue? >>> >>> If it also helps I am using CentOS 6.5, Hadoop 2.2.0, ZooKeeper 3.4.6. >>> >>> I also ran across this, that didn't look right... >>> >>> Welcome to ZooKeeper! >>> 2014-03-19 08:25:53,479 [myid:] - INFO >>> [main-SendThread(localhost:2181):ClientCnxn$SendThread@975] - Opening >>> socket connection to server localhost/127.0.0.1:2181. Will not attempt >>> to authenticat >>> e using SASL (unknown error) >>> 2014-03-19 08:25:53,483 [myid:] - INFO >>> [main-SendThread(localhost:2181):ClientCnxn$SendThread@852] - Socket >>> connection established to localhost/127.0.0.1:2181, initiating session >>> JLine support is enabled >>> [zk: localhost:2181(CONNECTING) 0] 2014-03-19 08:25:53,523 [myid:] - >>> INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1235] - >>> Session establishment complete on server localhost/127.0. >>> 0.1:2181, sessionid = 0x144da4e00d90000, negotiated timeout = 30000 >>> >>> should ZooKeeper try to hit localhost/127.0.0.1? >>> >>> my zoo.cfg looks like this.... >>> tickTime=2000 >>> initLimit=10 >>> syncLimit=5 >>> dataDir=/usr/local/zookeeper/data >>> clientPort=2181 >>> server.1=hadoop-node-1:2888:3888 >>> server.2=hadoop-node-2:2888:3888 >>> server.3=hadoop-node-3:2888:3888 >>> server.4=hadoop-node-4:2888:3888 >>> server.5=hadoop-node-5:2888:3888 >>> >>>
