Hi everyone,
Following is mine test environment:
node 1:namenode, secondarynamenode, jobtracker, hbase master
node 2:datanode, tasktracker
In node 1, I run following COMMANDS in pig shell, but I found map task failed
in tasktracker node with error "HBase is able to connect to ZooKeeper but the
connection closes immediately.". This mean tasktracker did not get current
hbase configuration. But I can find the correct hbase configuration in
jobtracker node. Seems tasktracker node did not get configuration from
jobtracker node, but get configuration from hadoop classpath in tasktracker
node.
I think tasktracker node should get hbase configuration from jobtracker node,
but not from local hadoop classpath. Am I correct?
In tasktracker side, after I add hbase-site.xml to hadoop classpath, the test
case passed.
My question is: how tasktracker node get hbase configuration from tasktracker
side? From jobtracker side(included in *.jar file transferred by jobtracker
node) or local hadoop classpath?
COMMANDS:
REGISTER /home/pig/Rules.jar;
REGISTER '/home/pig/zookeeper.jar';
REGISTER '/home/pig/guava-r06.jar';
REGISTER '/home/pig/hbase-0.90.5.jar';
test = LOAD 'hbase://table' USING
org.apache.pig.backend.hadoop.hbase.HBaseStorage( 'd:sWords','-loadKey true')
AS (ID: bytearray , Words:chararray );
result = FOREACH test GENERATE ID, com.nice.rules.RunRules(Words);
--result = FOREACH AA GENERATE com.nice.rules.RunRules(Words), ID;
--dump result;
store result into 'table' using
org.apache.pig.backend.hadoop.hbase.HBaseStorage('d:drools_cat');
--store result into 'AA_10_categs' using
org.apache.pig.backend.hadoop.hbase.HBaseStorage('d:cat');
