Finally got it. Came down to a user and ownership issue with my Hadoop, ZooKeeper, Accumulo. Does anyone have a Knowledgebase for this info that lays out a standard of what users should be created, where folders should be created, permissions, ownership, etc. I feel like that would be invaluable information for a newbie setting items up... or maybe I just tried to jump into something head first without being a Linux power user.
On Wed, Mar 19, 2014 at 10:04 AM, Sean Busbey <[email protected]>wrote: > Also, on the off chance that some other part of your system is exporting > an incorrect HADOOP_CONF_DIR, you should still run this confirmation step > from earlier: > > > You can verify this by doing > > > > ssh ${HOST} "bash -c 'echo ${HADOOP_CONF_DIR:-no hadoop conf}'" > > > > as the accumulo user on the master server, where HOST is the tserver. > > For the accumulo-env.sh you posted to impact things, the above command > would have to result in the output "no hadoop conf". > > -Sean > > > On Wed, Mar 19, 2014 at 9:01 AM, Sean Busbey <[email protected]>wrote: > >> Josh is correct about the behavior of the ZooKeeper cli. As an aside, how >> big is this cluster? Five ZooKeeper servers shouldn't be needed until you >> get past ~100 nodes, unless you're just going for more fault tolerance. >> >> Could you update your gist with the changes to accumulo-env.sh? It's much >> easier to follow there. >> >> Could you also re-run the accumulo classpath command on the tserver after >> updating accumulo-env.sh across the cluster? >> >> We already know the proximal cause of the failure: the Hadoop >> configuration files aren't ending up in the classpath for the >> tabletservers. I can think of a few things that would show this in the >> tablet server logs, but you've already described hitting most of them. >> >> -Sean >> >> >> >> On Wed, Mar 19, 2014 at 8:27 AM, Benjamin Parrish < >> [email protected]> wrote: >> >>> So, I am back to no clue now... >>> >>> >>> On Wed, Mar 19, 2014 at 9:13 AM, Josh Elser <[email protected]>wrote: >>> >>>> I think by default zkCli.sh will just try to connect to localhost. You >>>> can change this by providing the quorum string to the script with the >>>> -server option. >>>> On Mar 19, 2014 8:29 AM, "Benjamin Parrish" < >>>> [email protected]> wrote: >>>> >>>>> I adjusted accumulo-env.sh to have hard coded values as seen below. >>>>> >>>>> Are there any logs that could shed some light on this issue? >>>>> >>>>> If it also helps I am using CentOS 6.5, Hadoop 2.2.0, ZooKeeper 3.4.6. >>>>> >>>>> I also ran across this, that didn't look right... >>>>> >>>>> Welcome to ZooKeeper! >>>>> 2014-03-19 08:25:53,479 [myid:] - INFO >>>>> [main-SendThread(localhost:2181):ClientCnxn$SendThread@975] - >>>>> Opening socket connection to server localhost/127.0.0.1:2181. Will >>>>> not attempt to authenticat >>>>> e using SASL (unknown error) >>>>> 2014-03-19 08:25:53,483 [myid:] - INFO >>>>> [main-SendThread(localhost:2181):ClientCnxn$SendThread@852] - Socket >>>>> connection established to localhost/127.0.0.1:2181, initiating session >>>>> JLine support is enabled >>>>> [zk: localhost:2181(CONNECTING) 0] 2014-03-19 08:25:53,523 [myid:] - >>>>> INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1235] - >>>>> Session establishment complete on server localhost/127.0. >>>>> 0.1:2181, sessionid = 0x144da4e00d90000, negotiated timeout = 30000 >>>>> >>>>> should ZooKeeper try to hit localhost/127.0.0.1? >>>>> >>>>> my zoo.cfg looks like this.... >>>>> tickTime=2000 >>>>> initLimit=10 >>>>> syncLimit=5 >>>>> dataDir=/usr/local/zookeeper/data >>>>> clientPort=2181 >>>>> server.1=hadoop-node-1:2888:3888 >>>>> server.2=hadoop-node-2:2888:3888 >>>>> server.3=hadoop-node-3:2888:3888 >>>>> server.4=hadoop-node-4:2888:3888 >>>>> server.5=hadoop-node-5:2888:3888 >>>>> >>>>> >> > -- Benjamin D. Parrish H: 540-597-7860
