I seem to have tracked it down.
It seems the environment variable HBASE_HOME is not set when the masters start up the data nodes, but when I run locally on a datanode, it is set just fine. I had variables set in .bash_profile and moved those exports to the hadoop-env.sh and everything works now. It took a lot of looking at the classpaths with ps -ef | grep Child etc. to figure it all out. Dean From: Hiller, Dean (Contractor) Sent: Sunday, January 02, 2011 4:35 PM To: '[email protected]' Subject: debug of TaskNode using IsolationRunner is different than real run!!! ugh...help please My guess is execution of hadoop script vs. launch of a child task differ in the *.site.xml files they load???? Ie. My mapred-site.xml which contains the Zookeeper quorum is read during IsolationRunner's run but the childTask does not have the same access. Has anyone setup zookeeper and pointed datanodes to zookeeper? Is that done through the mapred-site.xml or zoo.cfg file? Okay, so I have been running into my TaskNode not connecting to Zookeeper(map/reduce involving hbase and hbase connects there not me). I get this in the logs 011-01-02 15:28:26,250 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=60000 watcher=org.apache.hadoop.hbase.client.HConnectionManager$ClientZKWatche r...@165f738 Clearly, it should not be connecting to localhost since I have zookeeper running on a different node than the DataNodes-TaskNodes. Now, the kicker, when I run IsolationRunner, the log is the following now... 11/01/02 16:28:54 INFO zookeeper.ZooKeeper: Initiating client connection, connec tString=DENVER-DEV3.test.jsq.bsg.ad.adp.com:2181 sessionTimeout=60000 watcher=org. apache.hadoop.hbase.client.hconnectionmanager$clientzkwatc...@3b4f0e which is the correct node. I don't get it. This message and any attachments are intended only for the use of the addressee and may contain information that is privileged and confidential. If the reader of the message is not the intended recipient or an authorized representative of the intended recipient, you are hereby notified that any dissemination of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the message and any attachments from your system.
